From 1c9d19769358a0ec3e409db736a4c66a4c02b921 Mon Sep 17 00:00:00 2001 From: Sourab Mangrulkar <13534540+pacman100@users.noreply.github.com> Date: Sat, 18 Feb 2023 12:37:02 +0530 Subject: [PATCH] add `EleutherAI/gpt-neox-20b` to support matrix --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 0ae4415..45fbf93 100644 --- a/README.md +++ b/README.md @@ -222,6 +222,7 @@ An example is provided in `~examples/causal_language_modeling/peft_lora_clm_acce | OPT | ✅ | ✅ | ✅ | ✅ | | GPT-Neo | ✅ | ✅ | ✅ | ✅ | | GPT-J | ✅ | ✅ | ✅ | ✅ | +| GPT-NeoX-20B | ✅ | ✅ | ✅ | ✅ | ### Conditional Generation | Model | LoRA | Prefix Tuning | P-Tuning | Prompt Tuning | @@ -279,6 +280,8 @@ The same principle applies to our [segmentation models](https://huggingface.co/m | Model | LoRA | Prefix Tuning | P-Tuning | Prompt Tuning | | --------- | ---- | ---- | ---- | ---- | | SegFormer | ✅ | | | | + + ## Caveats: 1. Below is an example of using PyTorch FSDP for training. However, it doesn't lead to