[merge] Fix conflict

model/reward/instructor/README.md

@@ -2,20 +2,22 @@
 
 Trainer code based on huggingface. Compatible with deepspeed or accelerate
 
-Requirements
-
-```
-wandb
-evaluate
-datasets
-transformers
-torch==1.12
-```
-
-Start training reward model
+Install Python requirements
 
 ```bash
-python trainer.py configs/electra-base-dis-webgpt.yml
+pip install -r requirements.txt
+```
+
+Write or inherit a `configs/<config-name>.yml` file to store training
+configuration details.
+
+> The configuration file must have _at least_ all the keys present in
+> [`configs/dummy.yml`](configs/dummy.yml)
+
+Run training procedure
+
+```bash
+python trainer.py configs/<config-name>.yml
 ```
 
 Additional axis labeling, this outputs a 4 summary quality evaluation metrics

model/reward/instructor/configs/dummy.yml

@@ -0,0 +1,21 @@
+model_name: X
+tokenizer_name: X
+max_length: X
+num_train_epochs: X
+warmup_steps: X
+scheduler: X
+learning_rate: X
+deepspeed: X
+fp16: X
+local_rank: X
+gradient_checkpointing: X
+gradient_accumulation_steps: X
+per_device_train_batch_size: X
+per_device_eval_batch_size: X
+weight_decay: X
+max_grad_norm: X
+eval_steps: X
+save_steps: X
+wandb_entity: X
+datasets:
+  - X