From 35cc68d7f7cf2dc471488076f5aabe91f84acf22 Mon Sep 17 00:00:00 2001 From: Marek Kadlcik <10684818+markcheeky@users.noreply.github.com> Date: Tue, 3 Jan 2023 21:22:54 +0100 Subject: [PATCH] fix formatting to make linter happy --- docs/supervised_datasets.md | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/docs/supervised_datasets.md b/docs/supervised_datasets.md index 38d4ba2c..c23e06f1 100644 --- a/docs/supervised_datasets.md +++ b/docs/supervised_datasets.md @@ -2,7 +2,6 @@ For discussion about usage of supervised data see issue . - ## Motivation An important part of making the assistant useful is to teach it to understand and follow instructions, and to perform large set of tasks well. @@ -20,6 +19,7 @@ There are also supervised dialog datasets such as Blended Skill Talk or SODA. In is not as focused on "academic tasks" or correctness, but encourage the model to respond naturally like a person would. ### Promptsource + - GitHub: - paper: [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://arxiv.org/abs/2110.08207) - project for preparing templates and working with them @@ -29,31 +29,29 @@ is not as focused on "academic tasks" or correctness, but encourage the model to - (with multilingual data and machine-translated prompt) - they trained zero-shot models (= models for following instructions in the input) - based on T5 architecture (encoder-decoder) called T0 family (and MT0 for multilingual) - - and based on GPT architecture (decoder-only) called BloomZ family - - Huggingface demo: [T0](https://huggingface.co/bigscience/T0pp), [MT0](https://huggingface.co/bigscience/mt0-large), [BloomZ](https://huggingface.co/bigscience/bloomz), + - and based on GPT architecture (decoder-only) called BloomZ family + - Huggingface demo: [T0](https://huggingface.co/bigscience/T0pp), [MT0](https://huggingface.co/bigscience/mt0-large), [BloomZ](https://huggingface.co/bigscience/bloomz), - GitHub repo for T0: - GitHub repo for BloomZ and MT0: - ### Natural instructions + - GitHub: - paper: [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://arxiv.org/abs/2204.07705) - they crowdsource directly the data prepared for instruction following (and learning from a few examples) -- the GitHub repo = the dataset. It contains jsons +- the GitHub repo = the dataset. It contains jsons - they trained zero-shot and in-context few-shot models (in multiple sizes): - mT5 architecture (encoder-decoder, multilingual pretraining) - Huggingface demo few-shot: - Huggingface demo zero-shot: - ### Blended Skill Talk + - used by Facebook in Blenderbot project - HuggingFace dataset: - example model trained on it: - ### SODA + - GitHub: - paper: - -