wassname/alignment-handbook
1
0
Fork 0
mirror of https://github.com/wassname/alignment-handbook.git synced 2026-06-27 17:29:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

Missing config params on SFT (#31)

Browse Source 
* fix warmup with total number of steps

* Explicitely tell to use 80GB Gpus

* Revert "fix warmup with total number of steps"

This reverts commit 760e477efdbf7f67be766a0d43b0c3b2ac26947a.
This commit is contained in:
Thomas Capelle
2023-11-21 12:00:09 +01:00
committed by GitHub
parent c9d9035f95
commit f025057ce4
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
recipes/zephyr-7b-beta/README.md
+1 -1
View File
@@ -9,7 +9,7 @@ As described in the Zephyr [technical report](https://huggingface.co/papers/2310
See below for commands to train these models using either DeepSpeed ZeRO-3 or LoRA.
## Full training examples
You will require 8 GPUs (80GB of VRAM) to train the full model.
```shell
# Step 1 - SFT
ACCELERATE_LOG_LEVEL=info accelerate launch --config_file recipes/accelerate_configs/deepspeed_zero3.yaml scripts/run_sft.py recipes/zephyr-7b-beta/sft/config_full.yaml