wassname/Open-Assistant
1
0
Fork 0
mirror of https://github.com/wassname/Open-Assistant.git synced 2026-09-11 11:50:17 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #212 from bitplane/prettier-markdown

Browse Source 
Format markdown with prettier --prose-wrap=always
This commit is contained in:
Yannic Kilcher
2023-01-01 22:11:11 +01:00
committed by GitHub
parent 22863ba954 c3c7a1701a
commit 4841550cd4
22 changed files with 449 additions and 209 deletions
Download Patch File Download Diff File Expand all files Collapse all files
model/reward/instructor/README.md
+4 -2
View File
@@ -18,7 +18,8 @@ Start training reward model
python trainer.py configs/electra-base-dis-webgpt.yml
```
Additional axis labeling, this outputs a 4 summary quality evaluation metrics (score are normalized to 0-1 )
Additional axis labeling, this outputs a 4 summary quality evaluation metrics
(score are normalized to 0-1 )
```bash
python summary_quality_trainer.py configs/test-bloomz-560m-quality.yml
@@ -36,7 +37,8 @@ The four summary are :
## Dataset
For now we only supports webgpt and summary dataset from OpenAI. Once open-asisstant dataset are available it will be added here.
For now we only supports webgpt and summary dataset from OpenAI. Once
open-asisstant dataset are available it will be added here.
## Model
model/reward/instructor/TODO.md
+9 -4
View File
@@ -4,16 +4,21 @@ Some other reward features we can use
1. Summaries from human feedback
- use `confidence` score into the RM learning, ensure the output rank score correlates with confidence
- use `confidence` score into the RM learning, ensure the output rank score
correlates with confidence
- each labeling has a labeling `note`, basically comments by labeler, not sure what else we can use
- each labeling has a labeling `note`, basically comments by labeler, not sure
what else we can use
- ~~Use the score for "overall", "accuracy", "coverage", "coherence" from axis/evals to train an addition model (rank additional aspect of the policy model)~~
- ~~Use the score for "overall", "accuracy", "coverage", "coherence" from
axis/evals to train an addition model (rank additional aspect of the policy
model)~~
- this should be placed under experimental_dataset.py
2. Add support for anthropic dataset
- anthropic dataset is more like a conversation tree which is much complex than simply question-answer schema
- anthropic dataset is more like a conversation tree which is much complex than
simply question-answer schema
- this is basically a MCTS from alphazero.