wassname/vllm
1
0
Fork 0
mirror of https://github.com/wassname/vllm.git synced 2026-06-27 17:32:55 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Hardware][Intel] Optimize CPU backend and add more performance tips (#4971)

Browse Source 
Co-authored-by: Jianan Gu <jianan.gu@intel.com>
This commit is contained in:
Li, Jiang
2024-06-14 00:33:14 +08:00
committed by GitHub
parent bd43973522
commit 80aa7e91fc
6 changed files with 165 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -65,7 +65,7 @@ vLLM is flexible and easy to use with:
- Tensor parallelism support for distributed inference
- Streaming outputs
- OpenAI-compatible API server
- Support NVIDIA GPUs and AMD GPUs
- Support NVIDIA GPUs, AMD GPUs, and Intel CPUs
- (Experimental) Prefix caching support
- (Experimental) Multi-lora support