# 🤗 pets
Parameter-Efficient Tuning at Scale with 🤗 Accelerate

Supported methods:
1. Prefix Tuning
2. P-Tuning
3. Prompt Tuning
4. LoRA [in backlog]

## Models support matrix

### Sequence Classification
|            | Prefix Tuning | P-Tuning  | Prompt Tuning | LoRA  | 
| --------- | ---- | ---- | ---- | ---- |
| BERT           | ✅  | ✅  | ✅  |   |  
| RoBERTa        | ✅  | ✅  | ✅  |   |
| GPT-2          | ✅  | ✅  | ✅  |   | 
| Bloom          | ✅  | ✅  | ✅  |   |   
| OPT            | ✅  | ✅  | ✅  |   |
| GPT-Neo        | ✅  | ✅  | ✅  |   |
| GPT-J          | ✅  | ✅  | ✅  |   |
| Deberta        |   |   |   |   | 
| Deberta-v2     |   |   |   |   |

### Causal Language Modeling
|            | Prefix Tuning | P-Tuning  | Prompt Tuning | LoRA  | 
| --------- | ---- | ---- | ---- | ---- |
| GPT-2          | ✅  | ✅  | ✅  |   |
| Bloom          | ✅  | ✅  | ✅  |   |
| OPT            | ✅  | ✅  | ✅  |   |
| GPT-Neo        | ✅  | ✅  | ✅  |   |
| GPT-J          | ✅  | ✅  | ✅  |   |

### Conditional Generation
|            | Prefix Tuning | P-Tuning  | Prompt Tuning | LoRA  | 
| --------- | ---- | ---- | ---- | ---- |
| T5        | ✅  | ✅  | ✅  |   |
| BART      | ✅  | ✅  | ✅  |   |