d17fd7cd3b
Add auto_insert_empty_system_msg config flag ( #123 )
...
* Make system messages optional
Also use the `maybe_insert_system_message` in dpo setting
* add `auto_insert_empty_system_msg` flag
* add `auto_insert_empty_system_msg`
* add auto_insert_empty_system_msg
* Update src/alignment/configs.py
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com >
* make style
---------
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com >
2024-02-28 20:05:44 +01:00
lewtun and GitHub
87cc800498
Apply quantization during DPO QLoRA ( #115 )
...
* Add QLoRA fix
* Update script
2024-02-05 16:50:17 +01:00
lewtun and GitHub
f0ffa0d7a6
Update Zephyr configs to account for UltraFeedback & TRL fixes ( #88 )
...
* Add files
* Add checkpointing
* Add checkpointing to SFT
* Add loss type
* Fix setup|
* Clean SFT
* Add lora config
* Rename config
* Remove max eval samples
* Add kwargs tp push to hub
* Add DPO configs
* Fix dpo configs
* Extend chat template test to multi-turn
* Add warmup
* Refactor
* Fix LoRA -> QLoRA
* Fix configs
* Specify chat template
* Add sample logging
* Fix push to hub hanging
* Add reentrant
* Fix quality
* Add transformer logging
* Tweak grad acc
* Add null type
* Add doc
2024-01-10 17:42:24 +11:00
Kirill and GitHub
98fe28fb14
Clean deprecated max_samples arguments ( #89 )
2024-01-05 09:06:47 +11:00
Nathan Azrak and GitHub
3f368a0748
Add check that parameters are not intended to be offloaded ( #51 )
...
* Add check that parameters are not intended to be offloaded
* Only push model to device if quantization config is set.
2023-12-04 09:10:41 +01:00
Lewis Tunstall
e2e8ab945d
Refactor imports
2023-11-10 13:38:45 +00:00
edbeeching
7f1a14e0d4
adds auto adapter merge to dpo script
2023-11-10 14:15:44 +01:00
Lewis Tunstall
33a0ce3afd
Add more doc
2023-11-09 13:39:03 +00:00
Lewis Tunstall
ee10c4efd9
Make DPO work!
2023-11-08 22:58:34 +00:00
Lewis Tunstall
967eab4cfb
Add skeleton
2023-11-08 13:21:57 +00:00
