Commit Graph - vllm - Gitea: Git with a cup of tea

wassname/vllm

Fork 0

mirror of https://github.com/wassname/vllm.git synced 2026-07-17 11:34:17 +08:00

11fd7ea639 [Pixtral-Large] Pixtral actually has no bias in vision-lang adapter (#10449) Patrick von PlatenandGitHub 2024-11-19 18:33:06 +01:00
f028dff33d [BugFix] Fix hermes tool parser output error stream arguments in some cases (#10395) (#10398) COSMOPlatandGitHub 2024-11-19 21:42:50 +08:00
b4614656b8 [CI][CPU] adding numa node number as container name suffix (#10441) YuanandGitHub 2024-11-19 21:16:43 +08:00
25f9c78961 [misc][plugin] improve plugin loading (#10443) youkaichaoandGitHub 2024-11-19 02:43:21 -08:00
5390d6664f [Doc] Add the start of an arch overview page (#10368) Russell BryantandGitHub 2024-11-19 04:52:11 -05:00
382b6a4852 [Misc] Avoid misleading warning messages (#10438) Jee Jee LiandGitHub 2024-11-19 16:54:58 +08:00
272e31c0bd [Bugfix] Guard for negative counter metrics to prevent crash (#10430) Travis JohnsonandGitHub 2024-11-18 21:57:10 -07:00
74f8c2cf5f Add openai.beta.chat.completions.parse example to structured_outputs.rst (#10433) Michael GoinandGitHub 2024-11-18 23:37:46 -05:00
8c1fb50705 [Platform][Refactor] Extract func get_default_attn_backend to Platform (#10358) Mengqing CaoandGitHub 2024-11-19 11:22:26 +08:00
7eb719df13 [Bugfix]Fix Phi-3 BNB online quantization (#10417) Jee Jee LiandGitHub 2024-11-19 11:21:42 +08:00
284203f171 [ci/build] Have dependabot ignore all patch update (#10436) Kevin H. LuuandGitHub 2024-11-18 15:04:25 -10:00
90a6c759ca [misc] partial prefix & random input generation benchmark (#9929) Ricky XuandGitHub 2024-11-18 15:39:14 -08:00
2298e69b5f [ci][bugfix] fix kernel tests (#10431) youkaichaoandGitHub 2024-11-18 15:29:37 -08:00
a03ea40792 [3/N][torch.compile] consolidate custom op logging (#10399) youkaichaoandGitHub 2024-11-18 15:14:59 -08:00
96d999fbe8 [Kernel] Initial Machete W4A8 support + Refactors (#9855) Lucas WilkinsonandGitHub 2024-11-18 14:59:29 -05:00
c2170a5b39 [Kernel] Explicitly specify other value in tl.load calls (#9014) Angus WangandGitHub 2024-11-18 11:39:40 -08:00
6b2d25efc7 [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) Yan MaandGitHub 2024-11-19 02:18:05 +08:00
281cc4b3cd [Model][Bugfix] Support TP for PixtralHF ViT (#10405) Michael GoinandGitHub 2024-11-18 13:04:14 -05:00
4f686d139f Fix open_collective value in FUNDING.yml (#10426) Andrew NesbittandGitHub 2024-11-18 17:52:42 +00:00
31894a2155 [Doc] Add documentation for Structured Outputs (#9943) ismael-dmandGitHub 2024-11-18 18:52:12 +01:00
7851b45196 [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) youkaichaoandGitHub 2024-11-18 07:20:06 -08:00
4186be8111 [Doc] Update doc for LoRA support in GLM-4V (#10425) B-201andGitHub 2024-11-18 23:08:30 +08:00
e7ebb662d7 [Model] Remove transformers attention porting in VITs (#10414) Isotr0pyandGitHub 2024-11-18 21:45:21 +08:00
5be4e52b65 [Model][LoRA]LoRA support added for glm-4v (#10418) B-201andGitHub 2024-11-18 20:57:10 +08:00
01aae1cc68 [Model] Remove redundant softmax when using PoolingType.STEP (#10415) MaybewussandGitHub 2024-11-18 18:05:36 +08:00
c7dec926f6 [VLM] Report multi_modal_placeholders in output (#10407) lkchenandGitHub 2024-11-18 00:06:16 -08:00
51bb12d17b [4/N][torch.compile] clean up set_torch_compile_backend (#10401) youkaichaoandGitHub 2024-11-17 23:57:20 -08:00
47826cacf0 [Bugfix] Ignore ray reinit error when current platform is ROCm or XPU (#10375) ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟andGitHub 2024-11-18 05:29:26 +02:00
c4e464333e [Misc] Add uninitialized params tracking for AutoWeightsLoader (#10327) Isotr0pyandGitHub 2024-11-18 09:07:46 +08:00
d1557e66d3 [Misc] Enhance offline_inference to support user-configurable paramet… (#10392) wchen61andGitHub 2024-11-17 19:32:40 +08:00
80d85c5d7b [Bugfix] Fix mrope_position_delta in non-last prefill chunk (#10403) 电脑星人andGitHub 2024-11-17 16:50:24 +08:00
76aab90ab6 [Hardware] [HPU]add mark_step for hpu (#10239) Kunshang JiandGitHub 2024-11-17 16:44:44 +08:00
8d74b5aee9 [platforms] refactor cpu code (#10402) youkaichaoandGitHub 2024-11-16 23:14:23 -08:00
cf349c4a97 [Bugfix][CPU] Fix CPU embedding runner with tensor parallel (#10394) Isotr0pyandGitHub 2024-11-17 15:12:04 +08:00
905d0f0af4 [CI/Build] Fix IDC hpu [Device not found] issue (#10384) Chendi.XueandGitHub 2024-11-17 00:58:22 -06:00
643ecf7b11 [V1] Refactor model executable interface for all text-only language models (#10374) Roger WangandGitHub 2024-11-16 21:18:46 -08:00
4fd9375028 [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) youkaichaoandGitHub 2024-11-16 18:02:14 -08:00
661a34fd4f [V1] Add code owners for V1 (#10397) Woosuk KwonandGitHub 2024-11-16 10:45:26 -08:00
361c29e174 [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) 电脑星人andGitHub 2024-11-17 02:10:00 +08:00
b98d89efd4 [Misc] Medusa supports custom bias (#10361) Sky LeeandGitHub 2024-11-17 00:33:01 +08:00
8b6725b0cf [Misc] Update benchmark to support image_url file or http (#10287) Jaehyun AnandGitHub 2024-11-16 19:15:40 +09:00
1d75472626 [BugFix] [Kernel] Fix GPU SEGV occuring in fused_moe kernel (#10385) rasmithandGitHub 2024-11-16 03:55:05 -06:00
2f427c2d16 [misc][plugin] improve log messages (#10386) youkaichaoandGitHub 2024-11-16 01:23:20 -08:00
755b85359b [doc] add doc for the plugin system (#10372) youkaichaoandGitHub 2024-11-15 21:46:27 -08:00
32e46e000f [Frontend] Automatic detection of chat content format from AST (#9919) Cyrus LeungandGitHub 2024-11-16 13:35:40 +08:00
4f168f69a3 [Docs] Misc updates to TPU installation instructions (#10165) Michael GreenandGitHub 2024-11-15 21:26:17 +00:00
3e8d14d8a1 [Doc] Move PR template content to docs (#10159) Russell BryantandGitHub 2024-11-15 16:20:20 -05:00
a067f85e08 [Frontend] Add --version flag to CLI (#10369) Russell BryantandGitHub 2024-11-15 16:13:53 -05:00
c76ac49d26 [Docs] Add Nebius as sponsors (#10371) Simon MoandGitHub 2024-11-15 12:47:40 -08:00
a6221a144a [Misc] bump mistral common version (#10367) Simon MoandGitHub 2024-11-15 09:48:07 -08:00
79ee45b428 [Misc] Bump up test_fused_moe tolerance (#10364) ElizaWszolaandGitHub 2024-11-15 17:31:18 +01:00
691a3ec047 [Bugfix] Ensure special tokens are properly filtered out for guided structured output with MistralTokenizer (#10363) Guillaume CalmettesandGitHub 2024-11-15 15:50:40 +01:00
3a763ba0c3 [core][misc] keep compatibility for old-style classes (#10356) youkaichaoandGitHub 2024-11-15 05:55:51 -08:00
f2056f726d [Misc] Fix some help info of arg_utils to improve readability (#10362) shangmingcandGitHub 2024-11-15 20:40:30 +08:00
1d65ec7eeb [Bugfix] Fix fully sharded LoRA bug (#10352) Jee Jee LiandGitHub 2024-11-15 18:34:58 +08:00
26908554b2 [Doc] Remove float32 choice from --lora-dtype (#10348) Xin YangandGitHub 2024-11-15 02:22:57 -08:00
b311efd0bd [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) Cyrus LeungandGitHub 2024-11-15 17:34:17 +08:00
3d158cdc8d Add default value to avoid Falcon crash (#5363) (#10347) wchen61andGitHub 2024-11-15 16:52:20 +08:00
02dbf30e9a [Build] skip renaming files for release wheels pipeline (#9671) Simon MoandGitHub 2024-11-14 23:31:52 -08:00
2ac6d0e75b [Misc] Consolidate pooler config overrides (#10351) Cyrus LeungandGitHub 2024-11-15 14:59:00 +08:00
2ec8827288 [Bugfix] Qwen-vl output is inconsistent in speculative decoding (#10350) Sky LeeandGitHub 2024-11-15 13:40:10 +08:00
b40cf6402e [Model] Support Qwen2 embeddings and use tags to select model tests (#10184) Cyrus LeungandGitHub 2024-11-15 12:23:09 +08:00
2885ba0e24 [Misc] Change RedundantReshapesPass and FusionPass logging from info to debug (#10308) Tyler Michael SmithandGitHub 2024-11-14 21:44:26 -05:00
bf2ddc6610 [bugfix] Fix static asymmetric quantization case (#10334)
Luka Govedič GitHub Daniël de Kok
2024-11-14 20:35:11 -05:00
972112d82f [Bugfix] Fix unable to load some models (#10312) Cyrus LeungandGitHub 2024-11-15 08:55:54 +08:00
11cd1ae6ad [Tool parsing] Improve / correct mistral tool parsing (#10333) Patrick von PlatenandGitHub 2024-11-15 01:42:49 +01:00
554af9228d [Bugfix] use AF_INET6 for OpenAI Compatible Server with ipv6 (#9583) Zijin XiaoandGitHub 2024-11-15 08:38:53 +08:00
b2e0ad3b59 [Perf] Reduce peak memory usage of llama (#10339) Murali AndoorveeduandGitHub 2024-11-14 16:38:20 -08:00
4a18fd14ba Support Roberta embedding models (#9387)
Maximilien de Bayser GitHub Flavia Beo
2024-11-14 18:23:29 -03:00
1dbae0329c [Docs] Publish meetup slides (#10331) Woosuk KwonandGitHub 2024-11-14 08:19:38 -08:00
675d603400 [CI/Build] Make shellcheck happy (#10285) Cyrus LeungandGitHub 2024-11-14 17:47:53 +08:00
03025c023f [CI/Build] Fix CPU CI online inference timeout (#10314) Isotr0pyandGitHub 2024-11-14 16:45:32 +08:00
29f3ef26a3 [ci][distributed] disable hanging tests (#10317) youkaichaoandGitHub 2024-11-14 00:23:39 -08:00
294bf467ba [Model] Add BNB quantization support for Idefics3 (#10310)
B-201 GitHub Jee Jee Li
2024-11-14 14:31:44 +08:00
52b48c1ead [BugFix]: properly deserialize tool_calls iterator before processing by mistral-common when MistralTokenizer is used (#9951) Guillaume CalmettesandGitHub 2024-11-14 05:48:16 +01:00
f67ce05d0b [Frontend] Pythonic tool parser (#9859) Mike DepinetandGitHub 2024-11-13 20:14:34 -08:00
e0853b6508 [Misc] format.sh: Simplify tool_version_check (#10305) Russell BryantandGitHub 2024-11-13 22:12:35 -05:00
504ac53d18 [misc] error early for old-style class (#10304) youkaichaoandGitHub 2024-11-13 18:55:39 -08:00
15bb8330aa [Bugfix] Fix tensor parallel for qwen2 classification model (#10297) Isotr0pyandGitHub 2024-11-14 10:54:59 +08:00
ac49b59d8b [Bugfix] bitsandbytes models fail to run pipeline parallel (#10200) HoangCongDucandGitHub 2024-11-14 00:56:39 +08:00
0b8bb86bf1 [1/N] Initial prototype for multi-modal processor (#10044) Cyrus LeungandGitHub 2024-11-13 20:39:03 +08:00
bb7991aa29 [V1] Add missing tokenizer options for Detokenizer (#10288) Roger WangandGitHub 2024-11-13 03:02:56 -08:00
d909acf9fe [Model][LoRA]LoRA support added for idefics3 (#10281) B-201andGitHub 2024-11-13 17:25:59 +08:00
b6dde33019 [Core] Flashinfer - Remove advance step size restriction (#10282) Pavani MajetyandGitHub 2024-11-13 00:29:32 -08:00
1b886aa104 [Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944)
Austin Veselka GitHub FurtherAI
2024-11-13 02:28:13 -06:00
3945c82346 [Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (#10221) 电脑星人andGitHub 2024-11-13 15:07:22 +08:00
032fcf16ae [Doc] Fix typo in arg_utils.py (#10264) Xin YangandGitHub 2024-11-12 21:54:52 -08:00
56a955e774 Bump to compressed-tensors v0.8.0 (#10279) Dipika SikkaandGitHub 2024-11-13 00:54:10 -05:00
bbd3e86926 [V1] Support VLMs with fine-grained scheduling (#9871)
Woosuk Kwon GitHub Roger Wang
2024-11-12 20:53:13 -08:00
0d4ea3fb5c [core][distributed] use tcp store directly (#10275) youkaichaoandGitHub 2024-11-12 17:36:08 -08:00
112fa0bbe5 [V1] Fix CI tests on V1 engine (#10272) Woosuk KwonandGitHub 2024-11-12 16:17:20 -08:00
377b74fe87 Revert "[ci][build] limit cmake version" (#10271) youkaichaoandGitHub 2024-11-12 15:06:48 -08:00
18081451f9 [doc] improve debugging doc (#10270) youkaichaoandGitHub 2024-11-12 14:43:52 -08:00
96ae0eaeb2 [doc] fix location of runllm widget (#10266) youkaichaoandGitHub 2024-11-12 14:34:39 -08:00
1f55e05713 [V1] Enable Inductor when using piecewise CUDA graphs (#10268) Woosuk KwonandGitHub 2024-11-12 13:39:56 -08:00
8a06428c70 [LoRA] Adds support for bias in LoRA (#5733)
Umesh GitHub Umesh Deshpande
2024-11-12 11:08:40 -08:00
b41fb9d3b1 [Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982) sroy745andGitHub 2024-11-12 10:53:57 -08:00
7c65527918 [V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to EngineCoreRequest (#10245) Woosuk KwonandGitHub 2024-11-12 08:57:14 -08:00
47db6ec831 [Frontend] Add per-request number of cached token stats (#10174) zifeitongandGitHub 2024-11-12 08:42:28 -08:00
176fcb1c71 [Bugfix] Fix QwenModel argument (#10262) Jie Fu (傅杰)andGitHub 2024-11-13 00:36:51 +08:00

... 14 15 16 17 18 ...

Commit Graph Select branches Hide Pull Requests main Mono Color

Commit Graph

Select branches

Hide Pull Requests

main