Merge pull request #63 from huggingface/vision-examples

add: vision examples to readme.

README.md

@@ -256,7 +256,30 @@ Example is provided in `~examples/causal_language_modeling/peft_lora_clm_acceler
 | Deberta        | ✅  |     |   |   | 
 | Deberta-v2     | ✅  |     |   |   |
 
+### Text-to-Image Generation
 
+|   Model         | LoRA | Prefix Tuning  | P-Tuning | Prompt Tuning  | 
+| --------- | ---- | ---- | ---- | ----  |
+| Stable Diffusion           | ✅  |   |   |   |  
+
+
+### Image Classification
+
+|   Model         | LoRA | Prefix Tuning  | P-Tuning | Prompt Tuning  | 
+| --------- | ---- | ---- | ---- | ----  |
+| ViT           | ✅  |   |   |   | 
+| Swin           | ✅  |   |   |   | 
+
+___Note that we have tested LoRA for https://huggingface.co/docs/transformers/model_doc/vit and [https://huggingface.co/docs/transformers/model_doc/swin] for fine-tuning on image classification. However, it should be possible to use LoRA for any compatible model [provided](https://huggingface.co/models?pipeline_tag=image-classification&sort=downloads&search=vit) by 🤗 Transformers. Check out the respective
+examples to learn more. If you run into problems, please open an issue.___
+
+Same principle applies to our [segmentation models](https://huggingface.co/models?pipeline_tag=image-segmentation&sort=downloads) as well. 
+
+### Semantic Segmentation
+
+|   Model         | LoRA | Prefix Tuning  | P-Tuning | Prompt Tuning  | 
+| --------- | ---- | ---- | ---- | ----  |
+| SegFormer           | ✅  |   |   |   | 
 ## Caveats:
 
 1. Below is an example of using PyTorch FSDP for training. However, it doesn't lead to 

examples/semantic_segmentation/semantic_segmentation_peft_lora.ipynb

@@ -404,7 +404,7 @@
     "This involves two steps:\n",
     "\n",
     "* Defining a config with `LoraConfig`\n",
-    "* Wrapping the original `model` with `PeftModel` with the config defined in the step above. "
+    "* Wrapping the original `model` with `get_peft_model()` with the config defined in the step above. "
    ]
   },
   {
@@ -431,7 +431,7 @@
     }
    ],
    "source": [
-    "from peft import LoraConfig, PeftModel\n",
+    "from peft import LoraConfig, get_peft_model\n",
     "\n",
     "config = LoraConfig(\n",
     "    r=32,\n",
@@ -441,7 +441,7 @@
     "    bias=\"lora_only\",\n",
     "    modules_to_save=[\"decode_head\"],\n",
     ")\n",
-    "lora_model = PeftModel(model, config)\n",
+    "lora_model = get_peft_model(model, config)\n",
     "print_trainable_parameters(lora_model)"
    ]
   },