f0ffa0d7a6
Update Zephyr configs to account for UltraFeedback & TRL fixes ( #88 )
...
* Add files
* Add checkpointing
* Add checkpointing to SFT
* Add loss type
* Fix setup|
* Clean SFT
* Add lora config
* Rename config
* Remove max eval samples
* Add kwargs tp push to hub
* Add DPO configs
* Fix dpo configs
* Extend chat template test to multi-turn
* Add warmup
* Refactor
* Fix LoRA -> QLoRA
* Fix configs
* Specify chat template
* Add sample logging
* Fix push to hub hanging
* Add reentrant
* Fix quality
* Add transformer logging
* Tweak grad acc
* Add null type
* Add doc
2024-01-10 17:42:24 +11:00
c69ae4b8a5
Check that default_chat_template is also None ( #83 )
...
* Check that `default_chat_template` is also None before overwriting chat template
* add unit test to `get_tokenizer` to ensure default behaviour of chat template is not changed
---------
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com >
2024-01-08 17:54:23 +11:00
edf67d1d93
Tweaks
2023-11-10 11:15:45 +00:00
64f1834e01
Add config tests
2023-11-10 10:00:05 +00:00
2ed5a45d25
Add model utils tests
2023-11-10 09:42:15 +00:00
610a1a2de4
Add unit tests for data mixer
2023-11-10 08:37:53 +00:00
87033c09b3
Add tests folder
2023-10-26 09:54:20 +00:00
lewtun