wassname/peft
1
0
Fork 0
mirror of https://github.com/wassname/peft.git synced 2026-09-12 12:40:18 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #160 from zphang/llama

Browse Source 
Support for LLaMA models
This commit is contained in:
Sourab Mangrulkar
2023-03-14 11:18:29 +05:30
committed by GitHub
parent 3b3fc47f84 baa2a4d53f
commit 1c11bc067f
2 changed files with 10 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+9 -8
View File
@@ -217,14 +217,15 @@ An example is provided in `~examples/causal_language_modeling/peft_lora_clm_acce
## Models support matrix
### Causal Language Modeling
| Model | LoRA | Prefix Tuning | P-Tuning | Prompt Tuning |
| --------- | ---- | ---- | ---- | ---- |
| GPT-2 | ✅ | ✅ | ✅ | ✅ |
| Bloom | ✅ | ✅ | ✅ | ✅ |
| OPT | ✅ | ✅ | ✅ | ✅ |
| GPT-Neo | ✅ | ✅ | ✅ | ✅ |
| GPT-J | ✅ | ✅ | ✅ | ✅ |
| GPT-NeoX-20B | ✅ | ✅ | ✅ | ✅ |
| Model | LoRA | Prefix Tuning | P-Tuning | Prompt Tuning |
|--------------| ---- | ---- | ---- | ---- |
| GPT-2 | ✅ | ✅ | ✅ | ✅ |
| Bloom | ✅ | ✅ | ✅ | ✅ |
| OPT | ✅ | ✅ | ✅ | ✅ |
| GPT-Neo | ✅ | ✅ | ✅ | ✅ |
| GPT-J | ✅ | ✅ | ✅ | ✅ |
| GPT-NeoX-20B | ✅ | ✅ | ✅ | ✅ |
| LLaMA | ✅ | ✅ | ✅ | ✅ |
### Conditional Generation
| Model | LoRA | Prefix Tuning | P-Tuning | Prompt Tuning |