From b1be7112123e515639ac506b5534dedca8bbbcc9 Mon Sep 17 00:00:00 2001 From: Yu Meng Date: Tue, 9 Jul 2024 14:31:23 -0400 Subject: [PATCH] Update README.md --- README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index dae3033..5413cfb 100644 --- a/README.md +++ b/README.md @@ -144,7 +144,7 @@ python -m pip install flash-attn --no-build-isolation ## Training Scripts -We provide four training config files for the four training setups reported in our paper. The training config is set for 8xH100 GPUs. You may need to adjust `num_processes` and `per_device_train_batch_size` based on your computation environment. +We provide four training config files for the four training setups reported in our paper. The training config is set for 4xH100 GPUs. You may need to adjust `num_processes` and `per_device_train_batch_size` based on your computation environment. * Mistral-Base: ```shell @@ -162,6 +162,10 @@ ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/dee ```shell ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/deepspeed_zero3.yaml scripts/run_simpo.py training_configs/llama-3-8b-instruct-simpo.yaml ``` +* Llama3-Instruct v0.2: +```shell +ACCELERATE_LOG_LEVEL=info accelerate launch --config_file accelerate_configs/deepspeed_zero3.yaml scripts/run_simpo.py training_configs/llama-3-8b-instruct-simpo-v2.yaml +``` ## Evaluation