fix formatting on docs/supervised_data again to make linter happy again because the formatting rules changed in between

This commit is contained in:
Marek Kadlcik
2023-01-03 21:25:54 +01:00
parent 43d82dfaf5
commit c7ff4c6a97
+44 -22
View File
@@ -1,55 +1,77 @@
# Supervised datasets
For discussion about usage of supervised data see issue <https://github.com/LAION-AI/Open-Assistant/issues/186>.
For discussion about usage of supervised data see issue
<https://github.com/LAION-AI/Open-Assistant/issues/186>.
## Motivation
An important part of making the assistant useful is to teach it to understand and follow instructions, and to perform large set of tasks well.
An important part of making the assistant useful is to teach it to understand
and follow instructions, and to perform large set of tasks well.
While RLHF seems like the main ingredient, using existing supervised data might help.
While RLHF seems like the main ingredient, using existing supervised data might
help.
There are two large-scale projects in the area of instruction-following / multitask learning: Promptsource and Natural Instructions -
these projects crowdsourced templates and turned existing NLP datasets into instruction-following seq2seq form in natural langauge.
They include both long-output training examples like generating a sentence that is a likely consequence of sentence in the prompt, and
short-output, like rating prediction from review. (Pre-)training on such datasets should help model understand and follow instructions
and teach it many abilities neccessary to perform a large set of tasks correctly. However, these data are not dialog-like - they do not
There are two large-scale projects in the area of instruction-following /
multitask learning: Promptsource and Natural Instructions - these projects
crowdsourced templates and turned existing NLP datasets into
instruction-following seq2seq form in natural langauge. They include both
long-output training examples like generating a sentence that is a likely
consequence of sentence in the prompt, and short-output, like rating prediction
from review. (Pre-)training on such datasets should help model understand and
follow instructions and teach it many abilities neccessary to perform a large
set of tasks correctly. However, these data are not dialog-like - they do not
look like a normal conversation.
There are also supervised dialog datasets such as Blended Skill Talk or SODA. In constrast to instruction-following datasets, dialog data
is not as focused on "academic tasks" or correctness, but encourage the model to respond naturally like a person would.
There are also supervised dialog datasets such as Blended Skill Talk or SODA. In
constrast to instruction-following datasets, dialog data is not as focused on
"academic tasks" or correctness, but encourage the model to respond naturally
like a person would.
### Promptsource
- GitHub: <https://github.com/bigscience-workshop/promptsource>
- paper: [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://arxiv.org/abs/2110.08207)
- paper:
[Multitask Prompted Training Enables Zero-Shot Task Generalization](https://arxiv.org/abs/2110.08207)
- project for preparing templates and working with them
- they generated a dataset using the templates:
- <https://huggingface.co/datasets/bigscience/P3>
- <https://huggingface.co/datasets/bigscience/xP3> (with multilingual data but English prompt)
- <https://huggingface.co/datasets/bigscience/xP3mt> (with multilingual data and machine-translated prompt)
- they trained zero-shot models (= models for following instructions in the input)
- based on T5 architecture (encoder-decoder) called T0 family (and MT0 for multilingual)
- <https://huggingface.co/datasets/bigscience/xP3> (with multilingual data but
English prompt)
- <https://huggingface.co/datasets/bigscience/xP3mt> (with multilingual data
and machine-translated prompt)
- they trained zero-shot models (= models for following instructions in the
input)
- based on T5 architecture (encoder-decoder) called T0 family (and MT0 for
multilingual)
- and based on GPT architecture (decoder-only) called BloomZ family
- Huggingface demo: [T0](https://huggingface.co/bigscience/T0pp), [MT0](https://huggingface.co/bigscience/mt0-large), [BloomZ](https://huggingface.co/bigscience/bloomz),
- Huggingface demo: [T0](https://huggingface.co/bigscience/T0pp),
[MT0](https://huggingface.co/bigscience/mt0-large),
[BloomZ](https://huggingface.co/bigscience/bloomz),
- GitHub repo for T0: <https://github.com/bigscience-workshop/t-zero>
- GitHub repo for BloomZ and MT0: <https://github.com/bigscience-workshop/xmtf>
- GitHub repo for BloomZ and MT0:
<https://github.com/bigscience-workshop/xmtf>
### Natural instructions
- GitHub: <https://github.com/allenai/natural-instructions>
- paper: [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://arxiv.org/abs/2204.07705)
- they crowdsource directly the data prepared for instruction following (and learning from a few examples)
- paper:
[Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://arxiv.org/abs/2204.07705)
- they crowdsource directly the data prepared for instruction following (and
learning from a few examples)
- the GitHub repo = the dataset. It contains jsons
- they trained zero-shot and in-context few-shot models (in multiple sizes):
- mT5 architecture (encoder-decoder, multilingual pretraining)
- Huggingface demo few-shot: <https://huggingface.co/allenai/tk-instruct-3b-def-pos>
- Huggingface demo zero-shot: <https://huggingface.co/allenai/tk-instruct-3b-def>
- Huggingface demo few-shot:
<https://huggingface.co/allenai/tk-instruct-3b-def-pos>
- Huggingface demo zero-shot:
<https://huggingface.co/allenai/tk-instruct-3b-def>
### Blended Skill Talk
- used by Facebook in Blenderbot project
- HuggingFace dataset: <https://huggingface.co/datasets/blended_skill_talk>
- example model trained on it: <https://huggingface.co/facebook/blenderbot_small-90M>
- example model trained on it:
<https://huggingface.co/facebook/blenderbot_small-90M>
### SODA