wassname/SimPO
1
0
Fork 0
mirror of https://github.com/wassname/SimPO.git synced 2026-06-27 19:29:20 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Yu Meng
2024-07-09 14:31:23 -04:00
committed by GitHub
parent 3235adf17a
commit b1be711212
1 changed files with 5 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+5 -1
View File
@@ -144,7 +144,7 @@ python -m pip install flash-attn --no-build-isolation
## Training Scripts
We provide four training config files for the four training setups reported in our paper. The training config is set for 8xH100 GPUs. You may need to adjust `num_processes` and `per_device_train_batch_size` based on your computation environment.
We provide four training config files for the four training setups reported in our paper. The training config is set for 4xH100 GPUs. You may need to adjust `num_processes` and `per_device_train_batch_size` based on your computation environment.
* Mistral-Base:
```shell
@@ -162,6 +162,10 @@ ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/dee
```shell
ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/deepspeed_zero3.yaml scripts/run_simpo.py training_configs/llama-3-8b-instruct-simpo.yaml
```
* Llama3-Instruct v0.2:
```shell
ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/deepspeed_zero3.yaml scripts/run_simpo.py training_configs/llama-3-8b-instruct-simpo-v2.yaml
```
## Evaluation