Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• c5cffcd0cd [Docs] Update spec decode + structured output in compat matrix (#12373) Russell BryantandGitHub 2025-01-23 20:15:52 -05:00
• 682b55bc07 [Docs] Add meetup slides (#12345) Woosuk KwonandGitHub 2025-01-23 14:10:03 -08:00
• 9726ad676d [Misc] Fix OpenAI API Compatibility Issues in Benchmark Script (#12357) Junichi SatoandGitHub 2025-01-24 07:02:13 +09:00
• eb5cb5e528 [BugFix] Fix parameter names and process_after_weight_loading for W4A16 MoE Group Act Order (#11528) 4 people 2025-01-23 16:40:33 -05:00
• 2cbeedad09 [Docs] Document Phi-4 support (#12362) Isotr0pyandGitHub 2025-01-24 03:18:51 +08:00
• 2c85529bfc [TPU] Update TPU CI to use torchxla nightly on 20250122 (#12334) Siyuan LiuandGitHub 2025-01-23 10:50:16 -08:00
• e97f802b2d [FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) 3 people 2025-01-23 13:04:03 -05:00
• 6e650f56a1 [torch.compile] decouple compile sizes and cudagraph sizes (#12243) youkaichaoandGitHub 2025-01-24 02:01:30 +08:00
• 3f50c148fd [core] add wake_up doc and some sanity check (#12361) youkaichaoandGitHub 2025-01-24 02:00:50 +08:00
• 8c01b8022c [Bugfix] Fix broken internvl2 inference with v1 (#12360) Isotr0pyandGitHub 2025-01-24 01:20:33 +08:00
• 99d01a5e3d [V1] Simplify M-RoPE (#12352) 3 people 2025-01-23 07:13:23 -08:00
• d07efb31c5 [Doc] Troubleshooting errors during model inspection (#12351) Cyrus LeungandGitHub 2025-01-23 22:46:58 +08:00
• 978b45f399 [Kernel] Flash Attention 3 Support (#12093) Lucas WilkinsonandGitHub 2025-01-23 09:45:48 -05:00
• c5b4b11d7f [Bugfix] Fix k_proj's bias for whisper self attention (#12342) Isotr0pyandGitHub 2025-01-23 18:15:33 +08:00
• 8ae5ff2009 [Hardware][Gaudi][BugFix] Fix dataclass error due to triton package update (#12338) liuzhenweiandGitHub 2025-01-23 16:35:46 +08:00
• 511627445e [doc] explain common errors around torch.compile (#12340) youkaichaoandGitHub 2025-01-23 14:56:02 +08:00
• f0ef37233e [V1] Add uncache_blocks (#12333) Cody YuandGitHub 2025-01-22 20:19:21 -08:00
• 7551a34032 [Docs] Document vulnerability disclosure process (#12326) Russell BryantandGitHub 2025-01-22 22:44:09 -05:00
• 01a55941f5 [Docs] Update FP8 KV Cache documentation (#12238) 3 people 2025-01-22 22:18:09 -05:00
• 8d7aa9de71 [Bugfix] Fixing AMD LoRA CI test. (#12329) Alexei-V-Ivanov-AMDandGitHub 2025-01-22 20:53:02 -06:00
• 68c4421b6d [AMD][Quantization] Add TritonScaledMMLinearKernel since int8 is broken for AMD (#12282) rasmithandGitHub 2025-01-22 18:10:37 -06:00
• aea94362c9 [Frontend][V1] Online serving performance improvements (#12287) Nick HillandGitHub 2025-01-22 14:22:12 -08:00
• 7206ce4ce1 [Core] Support reset_prefix_cache (#12284) Cody YuandGitHub 2025-01-22 10:52:27 -08:00
• 96f6a7596f [Bugfix] Fix HPU multiprocessing executor (#12167) Konrad ZaworaandGitHub 2025-01-22 19:07:07 +01:00
• 84bee4bd5c [Misc] Improve the readability of BNB error messages (#12320) Jee Jee LiandGitHub 2025-01-23 00:56:54 +08:00
• fc66dee76d [Misc] Fix the error in the tip for the --lora-modules parameter (#12319) RobinandGitHub 2025-01-23 00:48:41 +08:00
• 6609cdf019 [Doc] Add docs for prompt replacement (#12318) Cyrus LeungandGitHub 2025-01-22 22:56:29 +08:00
• 16366ee8bb [Bugfix][VLM] Fix mixed-modality inference backward compatibility for V0 (#12313) Roger WangandGitHub 2025-01-22 05:06:36 -08:00
• 528dbcac7d [Model][Bugfix]: correct Aria model output (#12309) zhou fanandGitHub 2025-01-22 19:39:19 +08:00
• cd7b6f0857 [VLM] Avoid unnecessary tokenization (#12310) Cyrus LeungandGitHub 2025-01-22 19:08:31 +08:00
• 68ad4e3a8d [Core] Support fully transparent sleep mode (#11743) youkaichaoandGitHub 2025-01-22 14:39:32 +08:00
• 4004f144f3 [Build] update requirements of no-device (#12299) Mengqing CaoandGitHub 2025-01-22 14:29:31 +08:00
• 66818e5b63 [core] separate builder init and builder prepare for each batch (#12253) youkaichaoandGitHub 2025-01-22 14:13:52 +08:00
• 222a9dc350 [Benchmark] More accurate TPOT calc in benchmark_serving.py (#12288) Nick HillandGitHub 2025-01-21 21:46:14 -08:00
• cbdc4ad5a5 [Ci/Build] Fix mypy errors on main (#12296) Cyrus LeungandGitHub 2025-01-22 12:06:54 +08:00
• 016e3676e7 [CI] add docker volume prune to neuron CI (#12291) Liangfu ChenandGitHub 2025-01-21 18:47:49 -08:00
• 64ea24d0b3 [ci/lint] Add back default arg for pre-commit (#12279) Kevin H. LuuandGitHub 2025-01-21 17:15:27 -08:00
• df76e5af26 [VLM] Simplify post-processing of replacement info (#12269) Cyrus LeungandGitHub 2025-01-22 08:48:13 +08:00
• 09ccc9c8f7 [Documentation][AMD] Add information about prebuilt ROCm vLLM docker for perf validation purpose (#12281) Hongxia YangandGitHub 2025-01-21 18:49:22 -05:00
• 69196a9bc7 [BUGFIX] When skip_tokenize_init and multistep are set, execution crashes (#12277) 3 people 2025-01-21 15:30:46 -08:00
• 2acba47d9b [bugfix] moe tuning. rm is_navi() (#12273) Divakar VermaandGitHub 2025-01-21 16:47:32 -06:00
• 9c485d9e25 [Core] Free CPU pinned memory on environment cleanup (#10477) Jani MonosesandGitHub 2025-01-21 21:56:41 +02:00
• fa9ee08121 [Misc] Set default backend to SDPA for get_vit_attn_backend (#12235) wangxiyuanandGitHub 2025-01-22 03:52:11 +08:00
• 347eeebe3b [Misc] Remove experimental dep from tracing.py (#12007) Adrian ColeandGitHub 2025-01-21 11:51:55 -08:00
• 18fd4a8331 [Bugfix] Multi-sequence broken (#11898) Andy LoandGitHub 2025-01-21 19:51:35 +00:00
• 132a132100 [v1][stats][1/n] Add RequestStatsUpdate and RequestStats types (#10907) Ricky XuandGitHub 2025-01-21 11:51:13 -08:00
• 1e60f87bb3 [Kernel] fix moe_align_block_size error condition (#12239) Jinzhen LinandGitHub 2025-01-22 02:30:28 +08:00
• 9705b90bcf [Bugfix] fix race condition that leads to wrong order of token returned (#10802) Jannis SchönleberandGitHub 2025-01-21 18:47:04 +01:00
• 3aec49e56f [ci/build] update nightly torch for gh200 test (#12270) youkaichaoandGitHub 2025-01-21 23:03:17 +08:00
• c64612802b [Platform] improve platforms getattr (#12264) Mengqing CaoandGitHub 2025-01-21 22:42:41 +08:00
• 9a7c3a0042 Remove pytorch comments for outlines + compressed-tensors (#12260) Thomas ParnellandGitHub 2025-01-21 14:49:08 +01:00
• b197a5ccfd [V1][Bugfix] Fix data item ordering in mixed-modality inference (#12259) Roger WangandGitHub 2025-01-21 05:18:43 -08:00
• c81081fece [torch.compile] transparent compilation with more logging (#12246) youkaichaoandGitHub 2025-01-21 19:32:55 +08:00
• a94eee4456 [Bugfix] Fix mm_limits access for merged multi-modal processor (#12252) Cyrus LeungandGitHub 2025-01-21 18:09:39 +08:00
• f2e9f2a3be [Misc] Remove redundant TypeVar from base model (#12248) Cyrus LeungandGitHub 2025-01-21 16:40:39 +08:00
• 1f1542afa9 [Misc]Add BNB quantization for PaliGemmaForConditionalGeneration (#12237) Jee Jee LiandGitHub 2025-01-21 15:49:08 +08:00
• 96912550c8 [Misc] Rename MultiModalInputsV2 -> MultiModalInputs (#12244) Cyrus LeungandGitHub 2025-01-21 15:31:19 +08:00
• 2fc6944c5e [ci/build] disable failed and flaky tests (#12240) youkaichaoandGitHub 2025-01-21 13:25:03 +08:00
• 5fe6bf29d6 [BugFix] Fix GGUF tp>1 when vocab_size is not divisible by 64 (#12230) Nicolò LucchesiandGitHub 2025-01-21 05:23:14 +01:00
• d4b62d4641 [AMD][Build] Porting dockerfiles from the ROCm/vllm fork (#11777) Gregory ShtrasbergandGitHub 2025-01-20 23:22:23 -05:00
• ecf67814f1 Add quantization and guided decoding CODEOWNERS (#12228) Michael GoinandGitHub 2025-01-20 20:23:40 -05:00
• 750f4cabfa [Kernel] optimize moe_align_block_size for cuda graph and large num_experts (e.g. DeepSeek-V3) (#12222) 4 people 2025-01-21 08:42:16 +08:00
• 06a760d6e8 [bugfix] catch xgrammar unsupported array constraints (#12210) Cheng Kuan Yong JasonandGitHub 2025-01-21 08:42:02 +08:00
• da7512215f [misc] add cuda runtime version to usage data (#12190) 3 people 2025-01-21 08:31:01 +08:00
• af69a6aded fix: update platform detection for M-series arm based MacBook processors (#12227) IşıkandGitHub 2025-01-20 22:23:28 +00:00
• 7bd3630067 [Misc] Update CODEOWNERS (#12229) Roger WangandGitHub 2025-01-20 14:19:09 -08:00
• 96663699b2 [CI] Pass local python version explicitly to pre-commit mypy.sh (#12224) Chen ZhangandGitHub 2025-01-20 23:49:18 +08:00
• 18572e3384 [Bugfix] Fix HfExampleModels.find_hf_info (#12223) Cyrus LeungandGitHub 2025-01-20 23:35:36 +08:00
• 86bfb6dba7 [Misc] Pass attention to impl backend (#12218) wangxiyuanandGitHub 2025-01-20 23:25:28 +08:00
• 5f0ec3935a [V1] Remove _get_cache_block_size (#12214) Chen ZhangandGitHub 2025-01-20 21:54:16 +08:00
• c222f47992 [core][bugfix] configure env var during import vllm (#12209) youkaichaoandGitHub 2025-01-20 19:35:59 +08:00
• 170eb35079 [misc] print a message to suggest how to bypass commit hooks (#12217) youkaichaoandGitHub 2025-01-20 18:06:24 +08:00
• b37d82791e [Model] Upgrade Aria to transformers 4.48 (#12203) Cyrus LeungandGitHub 2025-01-20 17:58:48 +08:00
• 3127e975fb [CI/Build] Make pre-commit faster (#12212) Cyrus LeungandGitHub 2025-01-20 17:36:24 +08:00
• 4001ea1266 [CI/Build] Remove dummy CI steps (#12208) Cyrus LeungandGitHub 2025-01-20 16:41:57 +08:00
• 5c89a29c22 [misc] add placeholder format.sh (#12206) youkaichaoandGitHub 2025-01-20 16:04:49 +08:00
• 59a0192fb9 [Core] Interface for accessing model from VllmRunner (#10353) Cyrus LeungandGitHub 2025-01-20 15:00:59 +08:00
• 83609791d2 [Model] Add Qwen2 PRM model support (#12202) Isotr0pyandGitHub 2025-01-20 14:59:46 +08:00
• 0974c9bc5c [Bugfix] Fix incorrect types in LayerwiseProfileResults (#12196) Yuan TangandGitHub 2025-01-20 01:59:20 -05:00
• d2643128f7 [DOC] Add missing docstring in LLMEngine.add_request() (#12195) Yuan TangandGitHub 2025-01-20 01:59:00 -05:00
• c5c06209ec [DOC] Fix typo in docstring and assert message (#12194) Yuan TangandGitHub 2025-01-20 01:58:29 -05:00
• 3ea7b94523 Move linting to pre-commit (#11975) Harry MellorandGitHub 2025-01-20 06:58:01 +00:00
• 51ef828f10 [torch.compile] fix sym_tensor_indices (#12191) youkaichaoandGitHub 2025-01-20 11:37:50 +08:00
• df450aa567 [Bugfix] Fix num_heads value for simple connector when tp enabled (#12074) shangmingcandGitHub 2025-01-20 10:56:43 +08:00
• bbe5f9de7d [Model] Support for fairseq2 Llama (#11442) 3 people 2025-01-19 19:40:40 +01:00
• 81763c58a0 [V1] Add V1 support of Qwen2-VL (#12128) 4 people 2025-01-19 03:52:13 -08:00
• edaae198e7 [Misc] Add BNB support to GLM4-V model (#12184) Isotr0pyandGitHub 2025-01-19 19:49:22 +08:00
• 936db119ed benchmark_serving support --served-model-name param (#12109) 3 people 2025-01-19 17:59:56 +08:00
• e66faf4809 [torch.compile] store inductor compiled Python file (#12182) youkaichaoandGitHub 2025-01-19 16:27:26 +08:00
• 630eb5b5ce [Bugfix] Fix multi-modal processors for transformers 4.48 (#12187) Cyrus LeungandGitHub 2025-01-19 11:16:34 +08:00
• 4e94951bb1 [BUGFIX] Move scores to float32 in case of running xgrammar on cpu (#12152) Michal AdamczykandGitHub 2025-01-19 04:12:05 +01:00
• 7a8a48d51e [V1] Collect env var for usage stats (#12115) Simon MoandGitHub 2025-01-18 19:07:15 -08:00
• 32eb0da808 [Misc] Support register quantization method out-of-tree (#11969) yancongandGitHub 2025-01-19 08:13:16 +08:00
• 6d0e3d3724 [core] clean up executor class hierarchy between v1 and v0 (#12171) youkaichaoandGitHub 2025-01-18 14:35:15 +08:00
• 02798ecabe [Model] Port deepseek-vl2 processor, remove dependency (#12169) Isotr0pyandGitHub 2025-01-18 13:59:39 +08:00
• 813f249f02 [Docs] Fix broken link in SECURITY.md (#12175) Russell BryantandGitHub 2025-01-17 23:35:21 -05:00
• da02cb4b27 [core] further polish memory profiling (#12126) youkaichaoandGitHub 2025-01-18 12:25:08 +08:00
• c09503ddd6 [AMD][CI/Build][Bugfix] use pytorch stale wheel (#12172) Hongxia YangandGitHub 2025-01-17 22:15:53 -05:00
• 2b83503227 [misc] fix cross-node TP (#12166) youkaichaoandGitHub 2025-01-18 10:53:27 +08:00
• 7b98a65ae6 [torch.compile] disable logging when cache is disabled (#12043) youkaichaoandGitHub 2025-01-18 04:29:31 +08:00