wassname/SimPO
1
0
Fork 0
mirror of https://github.com/wassname/SimPO.git synced 2026-06-27 17:28:54 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Yu Meng
2024-06-02 18:25:09 -04:00
committed by GitHub
parent e36b6d5946
commit 62088ca319
1 changed files with 4 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
eval/README.md
+4 -1
View File
@@ -6,7 +6,10 @@ benchmark’s evaluation protocol. For AlpacaEval 2, we report both the raw win
length-controlled win rate (LC). The LC metric is specifically designed to be robust against model verbosity. For Arena-Hard, we report the win rate (WR) against the baseline model.
### AlpacaEval 2
We provides generation configurations for the released models in the `alpacaeval2/configs` directory, and the corresponding generation templates can be found in `alpacaeval2/templates`. To evaluate the models on AlpacaEval 2, please use the [`alpaca-eval`](https://github.com/tatsu-lab/alpaca_eval) package.
We provide generation configurations for the released models in the `alpacaeval2/configs` directory, and the corresponding generation templates can be found in `alpacaeval2/templates`. To evaluate the models on AlpacaEval 2, please use the [`alpaca-eval`](https://github.com/tatsu-lab/alpaca_eval) package.
### Arena-Hard
We provide generation configurations for the released models in the `arenahard/configs` directory, and the corresponding generation templates can be found in `arenahard/templates`. To evaluate the models on Arena-Hard, please use the [`arena-hard-auto`](https://github.com/lm-sys/arena-hard-auto) package.
### MT-Bench
We provide the reference answers generated by GPT-4 Turbo in the `mt-bench` directory which are more accurate than the original GPT-4 generated answers. To evaluate the models on MT-Bench, please use the [`FastChat LLM Judge`](https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge#mt-bench) package and replace the [reference answers generated by GPT-4](https://github.com/lm-sys/FastChat/blob/main/fastchat/llm_judge/data/mt_bench/reference_answer/gpt-4.jsonl).