mirror of
https://github.com/wassname/peft.git
synced 2026-09-10 12:20:21 +08:00
addressing remaining comments
This commit is contained in:
@@ -24,21 +24,3 @@ Supported methods include:
|
||||
2. Prefix Tuning: [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://aclanthology.org/2021.acl-long.353/), [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://arxiv.org/pdf/2110.07602.pdf)
|
||||
3. P-Tuning: [GPT Understands, Too](https://arxiv.org/pdf/2103.10385.pdf)
|
||||
4. Prompt Tuning: [The Power of Scale for Parameter-Efficient Prompt Tuning](https://arxiv.org/pdf/2104.08691.pdf)
|
||||
|
||||
## Getting started
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForSeq2SeqLM
|
||||
from peft import get_peft_config, get_peft_model, LoraConfig, TaskType
|
||||
|
||||
model_name_or_path = "bigscience/mt0-large"
|
||||
tokenizer_name_or_path = "bigscience/mt0-large"
|
||||
|
||||
peft_config = LoraConfig(task_type=TaskType.SEQ_2_SEQ_LM, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1)
|
||||
|
||||
model = AutoModelForSeq2SeqLM.from_pretrained(model_name_or_path)
|
||||
model = get_peft_model(model, peft_config)
|
||||
model.print_trainable_parameters()
|
||||
# output: trainable params: 2359296 || all params: 1231940608 || trainable%: 0.19151053100118282
|
||||
```
|
||||
|
||||
@@ -29,9 +29,6 @@ from peft import LoraConfig, TaskType
|
||||
peft_config = LoraConfig(task_type=TaskType.SEQ_2_SEQ_LM, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1)
|
||||
```
|
||||
|
||||
Here, `task_type` is the type of task you are training your model for.
|
||||
For available task types, please refer [TaskType](package_reference/config#peft.config.TaskType).
|
||||
|
||||
2. Load the base model you want to fine-tune.
|
||||
|
||||
```python
|
||||
Reference in New Issue
Block a user