Commit Graph
79 Commits
Author SHA1 Message Date
Thomas Lemoine 8044111a1b commented change in authors' type from str to list 2023-06-21 00:45:40 -04:00
Thomas Lemoine 9b8c470438 removed error_count_dict 2023-06-21 00:41:54 -04:00
Thomas Lemoine 12c9db9cd0 load_dataset simpler than langchain's wrapper 2023-06-21 00:40:26 -04:00
Thomas Lemoine 5479ed7e93 minor change to MAX_LEN_PROMPT 2023-06-21 00:38:43 -04:00
Thomas Lemoine f2e160f8e3 removed commented-out code 2023-06-21 00:37:16 -04:00
Thomas Lemoine a755ecc480 removed PATH_TO_DATASET_PKL, stored in dict now 2023-06-21 00:36:14 -04:00
Thomas Lemoine e9ebb25f88 fixed minor errors from testing 2023-06-21 00:34:14 -04:00
Thomas Lemoine a25f217530 simplified dataset code + uses huggingface dataset 2023-06-21 00:30:30 -04:00
Fraser d924e18a5c Remove old files from repo 2023-04-13 16:32:33 -04:00
henri123lemoine 8c1962fa7a Updated main, success 2023-03-30 03:45:06 -04:00
henri123lemoine 41877a58a8 Updated create_dataset, success 2023-03-30 03:44:26 -04:00
henri123lemoine 8d225a18d7 Repaired text_splitter and settings. create_dataset success 2023-03-30 02:26:08 -04:00
Thomas Lemoine a31174de04 fixed variable name 2023-03-30 02:08:47 -04:00
Fraser ac05ddbcb0 Merge remote-tracking branch 'origin/creating_embeddings_dataset' into this new thing I've made 2023-03-29 23:51:35 -04:00
henri123lemoine 19a227f7ab Added save_data function to create_dataset 2023-03-29 22:53:36 -04:00
henri123lemoine a4e6d45029 Updated dataset pickle. 2023-03-29 22:47:24 -04:00
henri123lemoine b19da39eac Updated the settings.py files 2023-03-29 22:42:52 -04:00
henri123lemoine 71ab553afd Added pickle dict to main 2023-03-29 03:38:56 -04:00
Fraser 6066d2487f move dataset 2023-03-29 03:11:03 -04:00
Fraser 8b5b38b813 Merge remote-tracking branch 'origin/creating_embeddings_dataset' into main 2023-03-29 03:05:17 -04:00
henri123lemoine 8892ecb8af Added the dataset containing roughly 1500 embeddings. 2023-03-29 02:58:24 -04:00
Thomas Lemoine 9a2890ef89 reorganized src 2023-03-29 00:53:04 -04:00
henri123lemoine 01b60d6a37 Bug fixes, mostly related to saving to json and metadata indices 2023-03-27 07:39:19 -04:00
henri123lemoine 4d21a30e7b Added moderation to the assistant 2023-03-26 05:41:08 -04:00
henri123lemoine c9d0af84ed Added an informed_assistant.py file. 2023-03-26 05:37:44 -04:00
henri123lemoine fb34d1b95c Modified embeddings.py's outputs to be more informative. 2023-03-25 15:47:39 -04:00
henri123lemoine c3f4c917b5 test dataset.py 2023-03-24 23:46:16 -04:00
henri123lemoine 0572e9d40a Testing OpenAI API Key 2023-03-24 22:36:05 -04:00
henri123lemoine 3e5cfa18f8 Merge remote-tracking branch 'origin/main' into main 2023-03-24 22:23:33 -04:00
henri123lemoine c8a33ff114 Solved dataset.py errors. 2023-03-24 22:23:19 -04:00
Thomas Lemoine a845f90412 modified the textsplitter 2023-03-24 22:22:05 -04:00
henri123lemoine ea5bf2e707 Solved error in dataset.py. 2023-03-24 22:21:29 -04:00
Thomas Lemoine 3dbd0fdb2f updated the dataset 2023-03-24 21:59:49 -04:00
Thomas Lemoine 0f20079a92 removed unused module 2023-03-24 21:24:57 -04:00
Thomas Lemoine 50203f92eb smaller token blocks 2023-03-24 21:22:22 -04:00
Thomas Lemoine 4dc70f7286 removed pinecone 2023-03-24 20:26:52 -04:00
Thomas Lemoine d8c642e576 modified dataset storage of metadata 2023-03-24 18:56:52 -04:00
henri123lemoine e092681346 Merge branch 'main' of https://github.com/henri123lemoine/AlignmentSearch 2023-03-24 17:23:32 -04:00
henri123lemoine ff1a5b5272 Modified text_splitter, added prompts. 2023-03-24 17:22:54 -04:00
henri123lemoine 35da2db2cf Added comments, modified text_splitter. 2023-03-24 17:21:43 -04:00
Thomas Lemoine 3cd271d8d7 introducing pinecone stuff 2023-03-24 17:01:18 -04:00
henri123lemoine c54252e182 Updated folder structure and AlignmentSearch prompt. 2023-03-20 12:48:35 -04:00
Thomas Lemoine 475308c081 refactor Dataset class 2023-03-20 02:14:48 -04:00
Thomas Lemoine e1e6a3e67b refactor of Dataset class 2023-03-20 02:13:39 -04:00
Thomas Lemoine b34b1483c1 lowered dependencies in the notebook 2023-03-17 12:16:21 -04:00
Thomas Lemoine 873c3aeec2 Merge branch 'main' of https://github.com/henri123lemoine/AlignmentSearch 2023-03-16 19:21:46 -04:00
Thomas Lemoine d284375431 Merge branch 'main' of https://github.com/henri123lemoine/AlignmentSearch 2023-03-16 19:20:03 -04:00
henri123lemoine 12137d0ab7 Merge branch 'main' of https://github.com/henri123lemoine/AlignmentSearch 2023-03-16 11:32:57 -04:00
henri123lemoine 0f61660c0c Added tests 2023-03-16 11:29:58 -04:00
Thomas Lemoine 9980deb36b made textsplitter work with tokens, not chars 2023-03-16 07:45:31 -04:00