wassname/vllm
1
0
Fork 0
mirror of https://github.com/wassname/vllm.git synced 2026-09-09 11:40:01 +08:00
Code Issues Packages Projects Releases Wiki Activity

Add Automatic Prefix Caching (#2762)

Browse Source 
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
This commit is contained in:
Sage Moore
2024-03-02 00:50:01 -08:00
committed by GitHub
co-authored by ElizaWszola Michael Goin
parent baee28c46c
commit ce4f5a29fb
18 changed files with 615 additions and 289 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/models/engine_args.rst
+4
View File
@@ -81,6 +81,10 @@ Below, you can find an explanation of every engine argument for vLLM:
Token block size for contiguous chunks of tokens.
.. option:: --enable-prefix-caching
Enables automatic prefix caching
.. option:: --seed <seed>
Random seed for operations.