mirror of
https://github.com/wassname/vllm.git
synced 2026-10-02 12:50:41 +08:00
[Model] Add base class for LoRA-supported models (#5018)
This commit is contained in:
1 parent
d12af207d2
commit
96354d6a29
20 files changed
+270
-75
No files matched your search
@@ -4,6 +4,9 @@ Using LoRA adapters
|
||||
===================
|
||||
|
||||
This document shows you how to use `LoRA adapters <https://arxiv.org/abs/2106.09685>`_ with vLLM on top of a base model.
|
||||
|
||||
LoRA adapters can be used with any vLLM model that implements :class:`~vllm.model_executor.models.interfaces.SupportsLoRA`.
|
||||
|
||||
Adapters can be efficiently served on a per request basis with minimal overhead. First we download the adapter(s) and save
|
||||
them locally with
|
||||
|
||||
|
||||
Reference in new issue
Block a user