wassname/vllm
1
0
Fork 0
mirror of https://github.com/wassname/vllm.git synced 2026-08-11 11:28:04 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228)

Browse Source 
Signed-off-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
This commit is contained in:
Sanket Kale
2024-11-25 18:32:39 -08:00
committed by GitHub
co-authored by Sanket Kale mgoin
parent 45ac4ff270
commit a6760f6456
9 changed files with 678 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/offline_inference.py
+1 -1
View File
@@ -19,4 +19,4 @@ outputs = llm.generate(prompts, sampling_params)
for output in outputs:
prompt = output.prompt
generated_text = output.outputs[0].text
print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")