mirror of
https://github.com/wassname/peft.git
synced 2026-10-02 12:30:46 +08:00
43 lines
1.2 KiB
Markdown
43 lines
1.2 KiB
Markdown
# 🤗 pets
|
|
Parameter-Efficient Tuning at Scale with 🤗 Accelerate
|
|
|
|
Supported moethods:
|
|
1. Prefix Tuning
|
|
2. P-Tuning
|
|
3. Prompt Tuning
|
|
4. LoRA [in progress]
|
|
|
|
## Models support matrix
|
|
|
|
### Sequence Classification
|
|
| | Prefix Tuning | P-Tuning | Prompt Tuning | LoRA |
|
|
| --------- | ---- | ---- | ---- | ---- |
|
|
| BERT | ✅ | ✅ | ✅ | |
|
|
| RoBERTa | ✅ | ✅ | ✅ | |
|
|
| GPT-2 | ✅ | ✅ | ✅ | |
|
|
| Bloom | ✅ | ✅ | ✅ | |
|
|
| OPT | ✅ | ✅ | ✅ | |
|
|
| GPT-Neo | ✅ | ✅ | ✅ | |
|
|
| GPT-J | ✅ | ✅ | ✅ | |
|
|
| Deberta | | | | |
|
|
| Deberta-v2 | | | | |
|
|
|
|
### Causal Language Modeling
|
|
| | Prefix Tuning | P-Tuning | Prompt Tuning | LoRA |
|
|
| --------- | ---- | ---- | ---- | ---- |
|
|
| GPT-2 | | | | |
|
|
| Bloom | | | | |
|
|
| OPT | | | | |
|
|
| GPT-Neo | | | | |
|
|
| GPT-J | | | | |
|
|
| BART | | | | |
|
|
|
|
### Conditional Generation
|
|
| | Prefix Tuning | P-Tuning | Prompt Tuning | LoRA |
|
|
| --------- | ---- | ---- | ---- | ---- |
|
|
| T5 | | | | |
|
|
| BART | | | | |
|
|
|
|
|
|
|