wassname/vllm
1
0
Fork 0
mirror of https://github.com/wassname/vllm.git synced 2026-06-27 19:49:51 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,712 Commits 1 Branch 0 Tags
bfbc0b32c6dbea5aea6b14724acaca515d23a28a
Commit Graph

690 Commits

Author SHA1 Message Date
Joe Runde bfbc0b32c6 [Frontend] Add backend-specific options for guided decoding (#13505) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-02-20 15:07:58 -05:00
Harry Mellor 992e5c3d34 Merge similar examples in offline_inference into single basic example (#12737) 2025-02-20 04:53:51 -08:00
Jee Jee Li 512368e34a [Misc] Qwen2.5 VL support LoRA (#13261) 2025-02-19 18:37:55 -08:00
Wilson Wu 01c184b8f3 Fix copyright year to auto get current year (#13561) 2025-02-19 16:55:34 +00:00
youkaichao ad5a35c21b [doc] clarify multi-node serving doc (#13558) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-19 22:32:17 +08:00
youkaichao 52ce14d31f [doc] clarify profiling is only for developers (#13554) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-19 20:55:58 +08:00
Roger Wang fd84857f64 [Doc] Add clarification note regarding paligemma (#13511) 2025-02-18 22:24:03 -08:00
Harry Mellor 00b69c2d27 [Misc] Remove dangling references to --use-v2-block-manager (#13492) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-02-19 03:37:26 +00:00
youkaichao 7b203b7694 [misc] fix debugging code (#13487) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-18 09:37:11 -08:00
Harry Mellor 2358ca527b [Doc]: Improve feature tables (#13224) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-02-18 18:52:39 +08:00
Isotr0py 67ef8f666a [Model] Enable quantization support for transformers backend (#12960) 2025-02-17 19:52:47 -08:00
Cyrus Leung 7b623fca0b [VLM] Check required fields before initializing field config in DictEmbeddingItems (#13380) 2025-02-17 01:36:07 -08:00
yankooo f857311d13 Fix spelling error in index.md (#13369) 2025-02-17 06:53:20 +00:00
shangmingc 46cdd59577 [Feature][Spec Decode] Simplify the use of Eagle Spec Decode (#12304) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-02-16 19:32:26 -08:00
凌 da833b0aee [Docs] Change myenv to vllm. Update python_env_setup.inc.md (#13325) 2025-02-16 16:04:21 +00:00
Roger Wang b7d309860e [V1] Update doc and examples for H2O-VL (#13349) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-02-16 10:35:54 +00:00
Cyrus Leung 367cb8ce8c [Doc] [2/N] Add Fuyu E2E example for multimodal processor (#13331) 2025-02-15 07:06:23 -08:00
Nicolò Lucchesi 579d7a63b2 [Bugfix][Docs] Fix offline Whisper (#13274) 2025-02-14 21:32:37 -08:00
Nicolò Lucchesi d84cef76eb [Frontend] Add /v1/audio/transcriptions OpenAI API endpoint (#12909) 2025-02-13 07:23:45 -08:00
Cyrus Leung 1bc3b5e71b [VLM] Separate text-only and vision variants of the same model architecture (#13157) 2025-02-13 06:19:15 -08:00
Cyrus Leung c9d3ecf016 [VLM] Merged multi-modal processor for Molmo (#12966) 2025-02-13 04:34:00 -08:00
Russell Bryant d46d490c27 [Frontend] Move CLI code into vllm.cmd package (#12971) 2025-02-12 23:12:21 -08:00
Cody Yu 60c68df6d1 [Build] Automatically use the wheel of the base commit with Python-only build (#13178) 2025-02-12 23:10:28 -08:00
Harry Mellor deb6c1c6b4 [Doc] Improve OpenVINO installation doc (#13102) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-02-11 18:02:46 +00:00
Farzad Abdolhosseini 08b2d845d6 [Model] Ultravox Model: Support v0.5 Release (#12912) 
Signed-off-by: Farzad Abdolhosseini <farzad@fixie.ai>
 2025-02-10 22:02:48 +00:00
Cyrus Leung 51f0b5f7f6 [Bugfix] Clean up and fix multi-modal processors (#13012) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-10 10:45:21 +00:00
Yuan Tang 243137143c [Doc] Add link to tool_choice tracking issue in tool_calling.md (#13003) 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
 2025-02-10 06:09:33 +00:00
Jee Jee Li 86222a3dab [VLM] Merged multi-modal processor for GLM4V (#12449) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-02-08 20:32:16 +00:00
Cyrus Leung 8a69e0e20e [CI/Build] Auto-fix Markdown files (#12941) 2025-02-08 04:25:15 -08:00
Jun Duan 256a2d29dc [Doc] Correct HF repository for TeleChat2 models (#12949) 2025-02-08 01:42:15 -08:00
TJian eaa92d4437 [ROCm] [Feature] [Doc] [Dockerfile] [BugFix] Support Per-Token-Activation Per-Channel-Weight FP8 Quantization Inferencing (#12501) 2025-02-07 08:13:43 -08:00
Jitse Klomp afe74f7a96 [Doc] double quote cmake package in build.inc.md (#12840) 2025-02-06 09:17:55 -08:00
Sumit Vij d88506dda4 [Model] LoRA Support for Ultravox model (#11253) 2025-02-05 19:54:13 -08:00
Cyrus Leung 75404d041b [VLM] Update compatibility with transformers 4.49 2025-02-05 19:09:45 -08:00
Roger Wang bf3b79efb8 [VLM] Qwen2.5-VL 2025-02-05 13:31:38 -08:00
Russell Bryant 9a5b1554b4 [Docs] Drop duplicate [source] links 2025-02-05 13:30:50 -08:00
Michael Goin c53dc466b1 [Doc] Remove performance warning for auto_awq.md (#12743) 2025-02-04 22:43:11 -08:00
Isotr0py 815079de8e [VLM] merged multimodal processor and V1 support for idefics3 (#12660) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-02-04 20:00:51 +08:00
Cyrus Leung d1ca7df84d [VLM] Merged multi-modal processor for InternVL-based models (#12553) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-02-04 16:44:52 +08:00
Thomas Parnell bb392af434 [Doc] Replace ibm-fms with ibm-ai-platform (#12709) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-02-04 07:05:04 +00:00
Arthur a1a2aaadb9 [Model]: Add transformers backend support (#11330) 
# Adds support for `transformers` as a backend

Following https://github.com/huggingface/transformers/pull/35235, a
bunch of models should already be supported, we are ramping up support
for more models.

Thanks @Isotr0py for the TP support, and @hmellor for his help as well!
This includes: 
- `trust_remote_code=True` support: any model on the hub, if it
implements attention the correct way can be natively supported!!
- tensor parallel support

---------

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <41363108+Isotr0py@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-02-03 21:30:38 +08:00
youkaichao e64330910b [doc][misc] clarify VLLM_HOST_IP for multi-node inference (#12667) 
As more and more people are trying deepseek models with multi-node
inference, https://github.com/vllm-project/vllm/issues/7815 becomes more
frequent. Let's give clear message to users.

Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-03 09:32:18 +08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Kunshang Ji f256ebe4df [Hardware][Intel GPU] add XPU bf16 support (#12392) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-02-02 10:17:26 +00:00
Brian Dellabetta 44bbca78d7 [Doc] int4 w4a16 example (#12585) 
Based on a request by @mgoin , with @kylesayrs we have added an example
doc for int4 w4a16 quantization, following the pre-existing int8 w8a8
quantization example and the example available in
[`llm-compressor`](https://github.com/vllm-project/llm-compressor/blob/main/examples/quantization_w4a16/llama3_example.py)

FIX #n/a (no issue created)

@kylesayrs and I have discussed a couple additional improvements for the
quantization docs. We will revisit at a later date, possibly including:
- A section for "choosing the correct quantization scheme/ compression
technique"
- Additional vision or audio calibration datasets

---------

Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-01-31 15:38:48 -08:00
Harry Mellor 60808bd4c7 [Doc] Improve installation signposting (#12575) 
- Make device tab names more explicit
- Add comprehensive list of devices to
https://docs.vllm.ai/en/latest/getting_started/installation/index.html
- Add `attention` blocks to the intro of all devices that don't have
pre-built wheels/images

---------

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-31 15:38:35 -08:00
Cody Yu 60bcef000e [Docs][V1] Prefix caching design (#12598) 
- Create v1 design document section in docs.
- Add prefix caching design doc.

@WoosukKwon @ywang96

---------

Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-01-31 12:30:46 -08:00
Cody Yu 847f883232 [Git] Automatically sign-off commits (#12595) 
It's very annoying when I forgot to add `-s` in `git commit` to
sign-off, because I then need to `git rebase HEAD~1 --signoff` and `git
push -f` to fix the DCO. This PR adds a hook to sign off commits
automatically when `-s` is missing to solve this problem. The only
change from the user side is now users have to install 2 hooks, so
instead of just

```
pre-commit install
```

Now we need to

```
pre-commit install --hook-type pre-commit --hook-type commit-msg
```

Note that even if users still only install the pre-commit hook, they
won't get any error in `git commit`. Just the sign-off hook won't run.

cc @hmellor @youkaichao

---------

Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
 2025-01-31 12:30:33 -08:00
Harry Mellor e3f7ff65e7 Add favicon to docs (#12611) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-31 09:20:34 -08:00
Harry Mellor a2769032ca Set ?device={device} when changing tab in installation guides (#12560) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-30 00:05:42 -08:00