From 65a7ac97d57885a44b2e02f02072192f822980b0 Mon Sep 17 00:00:00 2001 From: Yu Meng Date: Tue, 9 Jul 2024 16:40:57 -0400 Subject: [PATCH 1/4] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index eb0d30c..688637a 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,7 @@ For DPO, we use the following hyperparameters for training. | Setting | β | Learning Rate | |------------------------|------|---------------| | Mistral-Base | 0.01 | 5e-7 | -| Mistral-Instruct | 0.01 | 2e-7 | +| Mistral-Instruct | 0.01 | 5e-7 | | Llama3-Base | 0.01 | 5e-7 | | Llama3-Instruct | 0.01 | 7e-7 | | Llama3-Instruct v0.2 | 0.01 | 3e-7 | From 26685dd9c65765c11d258e507fdc010559422fd3 Mon Sep 17 00:00:00 2001 From: Yu Meng Date: Tue, 9 Jul 2024 16:51:53 -0400 Subject: [PATCH 2/4] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 688637a..fba6ec1 100644 --- a/README.md +++ b/README.md @@ -194,6 +194,7 @@ Please cite our paper if you find the repo helpful in your work: @article{meng2024simpo, title={{SimPO}: Simple Preference Optimization with a Reference-Free Reward}, author={Meng, Yu and Xia, Mengzhou and Chen, Danqi}, + journal={arXiv preprint arXiv:2405.14734}, year={2024} } ``` From 3e5532e0b2494c9f65090a285420ef3429f06202 Mon Sep 17 00:00:00 2001 From: Yu Meng Date: Tue, 9 Jul 2024 18:05:29 -0400 Subject: [PATCH 3/4] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index fba6ec1..411f0f5 100644 --- a/README.md +++ b/README.md @@ -60,7 +60,7 @@ We have observed that, in some cases, adding an additional SFT loss can help imp ## Released Models ### v0.1 -Below is the complete list of models evaluated in our preprint. We used the [HuggingFaceH4/ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) dataset to train the Mistral Base and Llama3 Base models, the [princeton-nlp/mistral-instruct-ultrafeedback](https://huggingface.co/princeton-nlp/mistral-instruct-ultrafeedback) dataset to train the Mistral Instruct models, and the [princeton-nlp/llama3-ultrafeedback](https://huggingface.co/princeton-nlp/llama3-ultrafeedback) dataset to train the Llama3 Instruct models. The latter two datasets are annotated by the [llm-blender/PairRM](https://huggingface.co/llm-blender/PairRM) model. +Below is the complete list of models evaluated in our preprint. We used the [HuggingFaceH4/ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) dataset to train the Mistral Base and Llama3 Base models, the [princeton-nlp/mistral-instruct-ultrafeedback](https://huggingface.co/datasets/princeton-nlp/mistral-instruct-ultrafeedback) dataset to train the Mistral Instruct models, and the [princeton-nlp/llama3-ultrafeedback](https://huggingface.co/datasets/princeton-nlp/llama3-ultrafeedback) dataset to train the Llama3 Instruct models. The latter two datasets are annotated by the [llm-blender/PairRM](https://huggingface.co/llm-blender/PairRM) model. models | | AE2 LC | AE2 WR | AH | |------------------------------|-----------------------------------------------------------------------------------------------------------|:------:|:------:|:----:| From e7186a81342486b3bf0d4fed8360ec35eb7db02a Mon Sep 17 00:00:00 2001 From: Yu Meng Date: Wed, 10 Jul 2024 00:33:48 -0400 Subject: [PATCH 4/4] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 411f0f5..3f19486 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,7 @@ We used the following hyperparameters for training the released models (note tha | Llama3-Instruct | 2.5 | 0.55 | 1e-6 | | Llama3-Instruct v0.2 | 10 | 0.3 | 1e-6 | -For DPO, we use the following hyperparameters for training. +For DPO, the best hyperparameters for each setting are as follows. | Setting | β | Learning Rate | |------------------------|------|---------------| | Mistral-Base | 0.01 | 5e-7 |