wassname/vllm
1
0
Fork 0
mirror of https://github.com/wassname/vllm.git synced 2026-07-13 15:37:06 +08:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • baeded2569 [Attention] Deepseek v3 MLA support with FP8 compute (#12601) Lucas Wilkinson 2025-02-01 00:52:51 -05:00
  • 3e1c76cf3a Fix: Respect sparsity_config.ignore in Cutlass Integration (#12517) Rahul Tuli 2025-01-31 23:41:59 -06:00
  • cfa134d247 [Bugfix/CI] Fixup benchmark_moe.py (#12562) Tyler Michael Smith 2025-02-01 00:41:35 -05:00
  • 35b7a05507 [ci] Upgrade transformers to 4.48.2 in CI dependencies (#12599) Kevin H. Luu 2025-01-31 21:22:23 -08:00
  • 1867c258bd Fix target matching for fused layers with compressed-tensors (#12617) Eldar Kurtic 2025-02-01 06:07:46 +01:00
  • cb3e73e4c8 [BugFix] fix wrong output when using lora and num_scheduler_steps=8 (#11161) fade_away 2025-02-01 12:52:07 +08:00
  • b1340f9d55 [V1] Bugfix: Validate Model Input Length (#12600) Robert Shaw 2025-01-31 21:32:04 -05:00
  • 44bbca78d7 [Doc] int4 w4a16 example (#12585) Brian Dellabetta 2025-01-31 17:38:48 -06:00
  • 60808bd4c7 [Doc] Improve installation signposting (#12575) Harry Mellor 2025-01-31 23:38:35 +00:00
  • fc542144c4 [Feature] Fix guided decoding blocking bitmask memcpy (#12563) Ryan Nguyen 2025-01-31 18:37:30 -05:00
  • eb5741ad42 [Kernel][Quantization] Integrate block-quantized CUTLASS kernels for DeepSeekV3 (#12587) Tyler Michael Smith 2025-01-31 18:29:11 -05:00
  • 145c2ff648 [Bugfix] Revert MoE Triton Config Default (#12629) Robert Shaw 2025-01-31 18:28:47 -05:00
  • 415f19474d [release] Add input step to ask for Release version (#12631) Kevin H. Luu 2025-01-31 13:39:36 -08:00
  • 89003c4082 [v1][Bugfix] Add extra_keys to block_hash for prefix caching (#12603) Chen Zhang 2025-02-01 05:13:04 +08:00
  • 60bcef000e [Docs][V1] Prefix caching design (#12598) Cody Yu 2025-01-31 12:30:46 -08:00
  • 847f883232 [Git] Automatically sign-off commits (#12595) Cody Yu 2025-01-31 12:30:33 -08:00
  • 325f679f32 [BugFix] Fix Torch.Compile For DeepSeek (#12594) Robert Shaw 2025-01-31 15:06:39 -05:00
  • e3f7ff65e7 Add favicon to docs (#12611) Harry Mellor 2025-01-31 17:20:34 +00:00
  • 7a8987dac5 [Bugfix] Gracefully handle huggingface hub http error (#12571) Roger Wang 2025-01-31 00:19:35 -08:00
  • cabaf4eff3 [Attention] MLA decode optimizations (#12528) Lucas Wilkinson 2025-01-31 02:49:37 -05:00
  • a1fc18c030 [ROCm][AMD][Model] llama 3.2 support upstreaming (#12421) Aleksandr Malyshev 2025-01-30 20:24:28 -08:00
  • 9798b2fb00 [Kernel] Update cutlass_scaled_mm to support 2d group (blockwise) scaling (#11868) Lucas Wilkinson 2025-01-30 21:33:00 -05:00
  • 4078052f09 [V1][Log] Add max request concurrency log to V1 (#12569) Michael Goin 2025-01-30 18:07:19 -05:00
  • bd2107e30a [CPU][PPC] Updated torch, torchvision, torchaudio dependencies (#12555) Nishidha 2025-01-31 02:59:39 +05:30
  • 9b0c4bab36 [Kernel] Triton Configs for Fp8 Block Quantization (#11589) Robert Shaw 2025-01-30 14:53:22 -05:00
  • 41bf5612f5 [Misc] fix typo: add missing space in lora adapter error message (#12564) Beim 2025-01-31 04:39:22 +13:00
  • a2769032ca Set ?device={device} when changing tab in installation guides (#12560) Harry Mellor 2025-01-30 08:05:42 +00:00
  • f17f1d4608 [V1][Metrics] Add GPU cache usage % gauge (#12561) Mark McLoughlin 2025-01-30 02:31:01 +00:00
  • 1c1bb0bbf2 [Misc][MoE] add Deepseek-V3 moe tuning support (#12558) Divakar Verma 2025-01-29 18:47:30 -06:00
  • e0cc5f259a [V1][BugFix] Free encoder cache for aborted requests (#12545) Woosuk Kwon 2025-01-29 13:47:33 -08:00
  • 73aa6cfdf7 Revert "[Build/CI] Fix libcuda.so linkage" (#12552) Tyler Michael Smith 2025-01-29 16:12:24 -05:00
  • 27b78c73ca [Kernel] add triton fused moe kernel for gptq/awq (#12185) Jinzhen Lin 2025-01-29 22:07:09 +08:00
  • b02fd288b2 [Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. (#11787) Pavani Majety 2025-01-29 01:46:12 -08:00
  • ff7424f491 [Frontend] Support override generation config in args (#12409) Yanyi Liu 2025-01-29 17:41:01 +08:00
  • d93bf4da85 [Model] Refactoring of MiniCPM-V and add MiniCPM-o-2.6 support for vLLM (#12069) Alphi 2025-01-29 17:24:59 +08:00
  • 036ca94c25 [Bugfix] handle alignment of arguments in convert_sparse_cross_attention_mask_to_dense (#12347) Travis Johnson 2025-01-29 01:54:35 -07:00
  • ef001d98ef Fix the pydantic logging validator (#12420) Maximilien de Bayser 2025-01-29 04:53:13 -03:00
  • 5f671cb4c3 [V1] Improve Error Message for Unsupported Config (#12535) Robert Shaw 2025-01-28 23:56:56 -05:00
  • bd02164cf9 Bugfix for whisper quantization due to fake k_proj bias (#12524) Michael Goin 2025-01-28 23:49:03 -05:00
  • 46fb056749 [V1][Metrics] Add TTFT and TPOT histograms (#12530) Mark McLoughlin 2025-01-29 04:11:16 +00:00
  • dd6a3a02cb [Doc] Convert docs to use colon fences (#12471) Harry Mellor 2025-01-29 03:38:29 +00:00
  • a7e3eba66f [Frontend] Support reasoning content for deepseek r1 (#12473) Ce Gao 2025-01-29 11:38:08 +08:00
  • fbb5bd4cef [TPU] Add example for profiling TPU inference (#12531) Michael Goin 2025-01-28 22:16:47 -05:00
  • 80fcc3ed1c [Kernel] Pipe attn_logits_soft_cap through paged attention TPU kernels (#12482) fenghuizhang 2025-01-28 14:36:44 -08:00
  • c386c43ca3 [V1][Metrics] Add per-request prompt/generation_tokens histograms (#12516) Mark McLoughlin 2025-01-28 22:07:22 +00:00
  • f26d790718 Do not run suggestion pre-commit hook multiple times (#12521) Harry Mellor 2025-01-28 20:05:27 +00:00
  • 0f657bdc52 Replace missed warning_once for rerank API (#12472) Michael Goin 2025-01-28 14:06:32 -05:00
  • 3fd1fb63ef [V1][Metrics] Hook up IterationStats for Prometheus metrics (#12478) Mark McLoughlin 2025-01-28 16:38:38 +00:00
  • 925d2f1908 [Doc] Fix typo for x86 CPU installation (#12514) Jun Duan 2025-01-28 11:37:10 -05:00
  • 8f58a51358 [VLM] Merged multi-modal processor and V1 support for Qwen-VL (#12504) Cyrus Leung 2025-01-29 00:25:05 +08:00
  • 2079e43bee [Core] Make raw_request optional in ServingCompletion (#12503) Sebastian Schoennenbeck 2025-01-28 11:56:45 +01:00
  • e29d4358ef [V1] Include Engine Version in Logs (#12496) Robert Shaw 2025-01-28 03:27:41 -05:00
  • 8cbc424975 Update README.md with V1 alpha release (#12495) Roger Wang 2025-01-28 00:22:41 -08:00
  • dd66fd2b01 [CI] fix pre-commit error (#12494) Mengqing Cao 2025-01-28 14:11:05 +08:00
  • 0f465ab533 [FEATURE] Enables offline /score for embedding models (#12021) Gabriel Marinho 2025-01-28 00:30:13 -03:00
  • 23a7cbc88b [CI/Build] Fixed the xla nightly issue report in #12451 (#12453) Hossein Sarshar 2025-01-27 22:18:07 -05:00
  • 426a5c3625 Fix bad path in prometheus example (#12481) Michael Goin 2025-01-27 20:56:31 -05:00
  • ddee88d0ff [Neuron][Kernel] NKI-based flash-attention kernel with paged KV cache (#11277) Liangfu Chen 2025-01-27 17:31:16 -08:00
  • 823ab79633 Update pre-commit hooks (#12475) Harry Mellor 2025-01-28 00:23:08 +00:00
  • 6116ca8cd7 [Feature] [Spec decode]: Enable MLPSpeculator/Medusa and prompt_logprobs with ChunkedPrefill (#10132) Nicolò Lucchesi 2025-01-27 22:38:35 +01:00
  • 2bc3fbba0c [FlashInfer] Upgrade to 0.2.0 (#11194) Bowen Wang 2025-01-28 02:19:24 +08:00
  • 3f1fc7425a [V1][CI/Test] Do basic test for top-p & top-k sampling (#12469) Woosuk Kwon 2025-01-27 09:40:04 -08:00
  • 01ba927040 [V1][Metrics] Add initial Prometheus logger (#12416) Mark McLoughlin 2025-01-27 17:26:28 +00:00
  • 103bd17ac5 [Build] Only build 9.0a for scaled_mm and sparse kernels (#12339) Lucas Wilkinson 2025-01-27 10:40:00 -05:00
  • ce69f7f754 [Bugfix] Fix gpt2 GGUF inference (#12467) Isotr0py 2025-01-27 18:31:49 +08:00
  • 624a1e4711 [V1][Minor] Minor optimizations for update_from_output (#12454) Woosuk Kwon 2025-01-27 01:09:27 -08:00
  • 372bf0890b [Bugfix] Fix missing seq_start_loc in xformers prefill metadata (#12464) Isotr0py 2025-01-27 15:25:30 +08:00
  • 5204ff5c3f [Bugfix] Fix Granite 3.0 MoE model loading (#12446) Cyrus Leung 2025-01-27 13:26:44 +08:00
  • 0cc6b383d7 [Frontend] Support scores endpoint in run_batch (#12430) Pooya Davoodi 2025-01-26 20:30:17 -08:00
  • 28e0750847 [V1] Avoid list creation in input preparation (#12457) Woosuk Kwon 2025-01-26 19:57:56 -08:00
  • 582cf78798 [DOC] Add link to vLLM blog (#12460) Yuan Tang 2025-01-26 21:46:19 -06:00
  • 0034b09ceb [Frontend] Rerank API (Jina- and Cohere-compatible API) (#12376) Kyle Mistele 2025-01-26 20:58:45 -06:00
  • 72bac73067 [Build/CI] Fix libcuda.so linkage (#12424) Tyler Michael Smith 2025-01-26 16:18:19 -05:00
  • 68f11149d8 [Bugfix][Kernel] Fix perf regression caused by PR #12405 (#12434) Lucas Wilkinson 2025-01-26 14:09:34 -05:00
  • 72f4880425 [Bugfix/CI] Fix broken kernels/test_mha.py (#12450) Tyler Michael Smith 2025-01-26 13:39:03 -05:00
  • aa2cd2c43d [Bugfix] Disable w16a16 2of4 sparse CompressedTensors24 (#12417) Tyler Michael Smith 2025-01-26 06:59:58 -05:00
  • 9ddc35220b [Frontend] generation_config.json for maximum tokens(#12242) Matthew Hendrey 2025-01-26 06:59:25 -05:00
  • a5255270c3 [Misc] Revert FA on ViT #12355 and #12435 (#12445) Roger Wang 2025-01-26 03:56:34 -08:00
  • 0ee349b553 [V1][Bugfix] Fix assertion when mm hashing is turned off (#12439) Roger Wang 2025-01-26 00:47:42 -08:00
  • fa63e710c7 [V1][Perf] Reduce scheduling overhead in model runner after cuda sync (#12094) Keyun Tong 2025-01-26 00:42:37 -08:00
  • 2a0309a646 [Misc][Bugfix] FA3 support to ViT MHA layer (#12435) Roger Wang 2025-01-25 21:00:31 -08:00
  • 324960a95c [TPU][CI] Update torchxla version in requirement-tpu.txt (#12422) Siyuan Liu 2025-01-24 23:23:03 -08:00
  • f1fc0510df [Misc] Add FA2 support to ViT MHA layer (#12355) Isotr0py 2025-01-25 15:07:35 +08:00
  • bf21481dde [ROCm][MoE] MI300 tuned configs Mixtral-8x(7B,22B) | fp16, fp8 (#12408) Divakar Verma 2025-01-24 22:17:19 -06:00
  • fb30ee92ee [Bugfix] Fix BLIP-2 processing (#12412) Cyrus Leung 2025-01-25 11:42:42 +08:00
  • 221d388cc5 [Bugfix][Kernel] Fix moe align block issue for mixtral (#12413) ElizaWszola 2025-01-24 20:49:28 -05:00
  • 3132a933b6 [Bugfix][Kernel] FA3 Fix - RuntimeError: This flash attention build only supports pack_gqa (for build size reasons). (#12405) Lucas Wilkinson 2025-01-24 15:20:59 -05:00
  • df5dafaa5b [Misc] Remove deprecated code (#12383) Cyrus Leung 2025-01-25 03:45:20 +08:00
  • ab5bbf5ae3 [Bugfix][Kernel] Fix CUDA 11.8 being broken by FA3 build (#12375) Lucas Wilkinson 2025-01-24 10:27:59 -05:00
  • 3bb8e2c9a2 [Misc] Enable proxy support in benchmark script (#12356) Junichi Sato 2025-01-24 23:58:26 +09:00
  • e784c6b998 [ci/build] sync default value for wheel size (#12398) youkaichao 2025-01-24 17:54:29 +08:00
  • 9a0f3bdbe5 [Hardware][Gaudi][Doc] Add missing step in setup instructions (#12382) Mohit Deopujari 2025-01-24 01:43:49 -08:00
  • c7c9851036 [ci/build] fix wheel size check (#12396) youkaichao 2025-01-24 17:31:25 +08:00
  • 3c818bdb42 [Misc] Use VisionArena Dataset for VLM Benchmarking (#12389) Roger Wang 2025-01-24 00:22:04 -08:00
  • 6dd94dbe94 [perf] fix perf regression from #12253 (#12380) youkaichao 2025-01-24 11:34:27 +08:00
  • 0e74d797ce [V1] Increase default batch size for H100/H200 (#12369) Woosuk Kwon 2025-01-23 19:19:55 -08:00
  • 55ef66edf4 Update compressed-tensors version (#12367) Dipika Sikka 2025-01-23 22:19:42 -05:00
  • 5e5630a478 [Bugfix] Path join when building local path for S3 clone (#12353) omer-dayan 2025-01-24 05:06:07 +02:00
  • d3d6bb13fb Set weights_only=True when using torch.load() (#12366) Russell Bryant 2025-01-23 21:17:30 -05:00
  • 24b0205f58 [V1][Frontend] Coalesce bunched RequestOutputs (#12298) Nick Hill 2025-01-23 17:17:41 -08:00