41 Commits

Author SHA1 Message Date
Yannic Kilcher b6a0eedf81 various fixes to the SFT code 2023-02-11 21:37:35 +01:00
Sotirios Anagnostidis 6a68139b91 os private dataset 2023-02-11 13:20:42 +01:00
Sotirios Anagnostidis ac97943be1 refactor datasets and oa private data selection 2023-02-11 11:49:58 +01:00
Sotirios Anagnostidis 23ee2f24d9 dataset args 2023-02-11 11:02:48 +01:00
Sotirios Anagnostidis 540a96fb0e pre commits 2023-02-11 10:29:56 +01:00
theblackcat102 9e69117ead [fix] Fix other PR merge bug 2023-02-11 01:53:37 +00:00
theblackcat102 bcebbbc49c [merge] Fix conflict 2023-02-11 00:23:25 +00:00
theblackcat102 34347607d4 [fix] add comments for translation data 2023-02-11 00:08:34 +00:00
sanagnos 4ba622de8e Merge branch 'main' into sft-data-sampling 2023-02-09 09:19:17 +01:00
theblackcat102 a1b90bf981 Merge branch 'main' into add-dataset 2023-02-09 01:28:42 +00:00
Mark Worrall 283df8ec84 Get working on multi-gpu 2023-02-08 20:49:25 +00:00
theblackcat102 2c35ff6e50 [fix] patch translated history conversation 2023-02-08 00:20:11 +00:00
theblackcat102 af1c62cd83 [feature] Add missing hindi and spanish prompt for translation 2023-02-07 01:23:54 +00:00
theblackcat102 742161590f [feature] Add rallio new instruction dataset v3 2023-02-06 00:22:30 +00:00
Kian-Meng Ang 1e321a6fca Fix typos (#1143)
Found via `codespell -S .mypy_cache,yarn.lock,*.json,*.ipynb -L
rouge,nam,vie`
2023-02-05 20:18:03 +01:00
theblackcat102 8b2080559c [fix] Custom collate_fn for training 2023-02-03 06:08:01 +00:00
theblackcat102 1041564db7 [feature] mix generation from different tasks 2023-02-03 00:15:29 +00:00
theblackcat102 9be4c921cd [feature] Add OA translated QA 2023-02-01 22:33:37 +00:00
theblackcat102 f8eba68544 [feature] Add mix conversation augmentation 2023-02-01 22:14:11 +00:00
theblackcat102 fdcc629678 [feature] add reddit eli5, asks, askh; bug fix 2023-01-29 03:05:56 +00:00
theblackcat102 b8990d9078 [fix] remove spaces in format_pair 2023-01-23 02:48:47 +00:00
theblackcat102 736f46fb00 [fix] prosocial dialogue format error 2023-01-22 14:00:20 +00:00
theblackcat102 f5b2a34857 [feature] add pythia and limit translation pair 2023-01-22 00:56:17 +00:00
theblackcat102 62a203fd8c [feature] move data formatting into dataset, instead of collator 2023-01-21 03:31:35 +00:00
theblackcat102 22e3ab1a89 [fix] linter fix 2023-01-20 07:23:02 +00:00
theblackcat102 c255148dc6 [fix] Merge main branch update 2023-01-20 06:18:43 +00:00
theblackcat102 6cd62e3d48 [fix] Fix missing russian and update readme 2023-01-20 06:16:26 +00:00
theblackcat102 74cb9aaa5a [feature] added translation, rallio instruct tuning dataset, prosocial for safety, new summary dataset 2023-01-20 03:02:07 +00:00
Sotirios Anagnostidis 5d9591d82c added soda dialogue dataset 2023-01-19 14:25:19 +01:00
theblackcat102 6f6c590e57 [fix] Disable task specific evaluation 2023-01-14 06:47:21 +00:00
theblackcat102 1546111094 [feature] added GSM8k and code refactoring 2023-01-14 06:24:47 +00:00
theblackcat102 3966024871 [fix] Fix summarizer bug and QA typo issue 2023-01-14 05:49:22 +00:00
theblackcat102 9451aff6cc [fix] @ekurtulus major logic bug in summarization 2023-01-14 03:49:19 +00:00
Sotirios Anagnostidis c8f47eef9f precommits 2023-01-11 22:58:17 +01:00
Sotirios Anagnostidis 4a3ea0b033 refactoring, now running 2023-01-11 22:42:04 +01:00
ekurtulus 5b77dd2e9f better 2023-01-11 11:37:27 +03:00
Sotirios Anagnostidis 148244455c refactor 2023-01-06 21:29:38 +01:00
Sotirios Anagnostidis 88ee3b3264 merge deepspeed 2023-01-06 21:28:26 +01:00
Sotirios Anagnostidis c20dfaad5b pre-commits 2023-01-03 22:45:34 +01:00
Sotirios Anagnostidis 3a10e9412d Question-Answer special tokens 2023-01-03 22:02:32 +01:00
Sotirios Anagnostidis 675b0866b0 SFT training 2023-01-03 01:31:56 +01:00