[Model] Add base class for LoRA-supported models (#5018)

This commit is contained in:
Cyrus Leung authored and GitHub committed 2024-06-27 16:03:04 +08:00
1 parent d12af207d2
commit 96354d6a29
20 files changed
+270 -75

No files matched your search

+3
View File
@@ -4,6 +4,9 @@ Using LoRA adapters
===================
This document shows you how to use `LoRA adapters <https://arxiv.org/abs/2106.09685>`_ with vLLM on top of a base model.
LoRA adapters can be used with any vLLM model that implements :class:`~vllm.model_executor.models.interfaces.SupportsLoRA`.
Adapters can be efficiently served on a per request basis with minimal overhead. First we download the adapter(s) and save
them locally with