Commit Graph
32 Commits
Author SHA1 Message Date
Thomas Lemoine 4639bb902b removed rawtext to vectorstore as it is useless 2023-03-15 04:36:38 -04:00
henri123lemoine e7979073e7 Updated dataset/semantic search/main for ease of use. 2023-03-14 08:45:33 -04:00
henri123lemoine 344ed281ee Updated the code from dataset.py, semantic_search.py, and main.py, using the code from the testing.ipynb file. 2023-03-14 08:30:53 -04:00
henri123lemoine fbb0c3bbea Updated the text_splitter.py file to remove an unnecessary function, and updated the testing notebook. 2023-03-14 08:18:05 -04:00
Thomas Lemoine 719674541c modified main of textsplitter 2023-03-14 02:21:18 -04:00
Thomas Lemoine 6503a61bd3 created the TextSplitter class 2023-03-14 00:04:05 -04:00
Thomas Lemoine 49b8cd0239 added functionality to textsplitter 2023-03-13 21:19:13 -04:00
Thomas Lemoine e8ee64cd7c raw text can be converted to chunks with signature 2023-03-10 21:09:51 -05:00
Thomas Lemoine ad41e0b08b removed json from requirements.txt 2023-03-10 21:00:39 -05:00
Thomas Lemoine 472902ec91 added requirements.txt 2023-03-09 19:04:42 -05:00
henri123lemoine 8d4a0b9b45 Moved files around, updated semantic search and added prompt search. 2023-03-09 16:20:52 -05:00
henri123lemoine 3b1451bf64 Same 2023-03-09 16:20:33 -05:00
henri123lemoine ced7e927f7 Changing format of folders. 2023-03-09 16:19:40 -05:00
henri123lemoine 47175fe074 Updated readme and testing notebook 2023-02-06 21:22:07 -05:00
henri123lemoine 53965dd15c updated typo in readme 2023-02-06 20:58:27 -05:00
henri123lemoine dac3dc8782 updated readme 2023-02-06 20:56:19 -05:00
henri123lemoine c6dd827c4a added if __name__ 2023-02-06 20:52:52 -05:00
henri123lemoine 82e79354f4 added main.py, do not run 2023-02-06 20:52:21 -05:00
henri123lemoine 59d9f25f65 Added settings.py 2023-02-06 20:51:34 -05:00
henri123lemoine ec03b854cc Added AlignmentSearch class, takes dataset class as input. Does semantic search, question answering and summarization. 2023-02-06 20:50:39 -05:00
henri123lemoine 47cb734bc3 Added helper functions for sentence-splitting and article splitting 2023-02-06 20:49:55 -05:00
henri123lemoine 807a7f066c Added dataset.py from testing.ipynb, gets the data from the jsonl file from wanted sources and gets the embeddings from each paragraph. 2023-02-06 20:48:58 -05:00
henri123lemoine 0155a3cc52 Didn't change much 2023-02-05 01:41:52 -05:00
henri123lemoine 639c11ad1e Added TODO 2023-02-05 01:41:15 -05:00
henri123lemoine f047045208 Added a search+q&a class 2023-02-04 05:11:22 -05:00
henri123lemoine aa39a22bf2 Updated testing.ipynb 2023-02-03 09:53:40 -05:00
henri123lemoine 88d2164d88 Updated the testing considerably. Loading the dataset works pretty well. 2023-02-03 06:37:35 -05:00
henri123lemoine 571d0195f4 OOPed notebook 2023-02-03 01:06:58 -05:00
henri123lemoine 3b603a51fa Updated the testing notebook for extracting the jsonl dataset. 2023-02-01 02:15:10 -05:00
henri123lemoine 44019ee24b Updated README.md description and source. 2023-02-01 02:14:30 -05:00
henri123lemoine f10bc74328 Added .jsonl and config to .gitignore 2023-02-01 02:13:47 -05:00
henri123lemoine a760ace26e Initial commit 2023-01-31 04:15:23 -05:00