Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• fd3a62a122 [perf-benchmark] Fix dependency for steps in benchmark pipeline (#11710) Kevin H. Luu 2025-01-03 13:38:37 +07:00
• 07064cb1d4 [Bugfix] Check chain_speculative_sampling before calling it (#11673) Lu Fang 2025-01-02 16:58:56 -08:00
• 2f1e8e8f54 Update default max_num_batch_tokens for chunked prefill (#11694) Sachin Varghese 2025-01-02 19:25:53 -05:00
• 68d37809b9 [Misc] Minimum requirements for SageMaker compatibility (#11576) Nathan Azrak 2025-01-03 10:59:25 +11:00
• 5dba257506 Resolve race conditions in Marlin kernel (#11493) wchen61 2025-01-03 06:58:56 +08:00
• 187e32997c [Bugfix] Change kv scaling factor by param json on nvidia gpu (#11688) bjmsongandbjmsong 2025-01-03 05:11:39 +08:00
• b55ed6ef8a [V1][Minor] Optimize token_ids_cpu copy (#11692) Woosuk Kwon 2025-01-03 04:04:58 +09:00
• 2f385183f3 [Bugfix] Free cross attention block table for preempted-for-recompute sequence group. (#10013) Kathy Yu 2025-01-02 10:28:09 -08:00
• 84c35c374a According to vllm.EngineArgs, the name should be distributed_executor_backend (#11689) Chunyang Wen 2025-01-03 02:14:16 +08:00
• 8c38ee7007 [VLM] Merged multi-modal processor for LLaVA-NeXT (#11682) Cyrus Leung 2025-01-03 00:39:27 +08:00
• b6087a6bee [mypy] Pass type checking in vllm/inputs (#11680) Tobias Pitters 2025-01-02 17:18:15 +01:00
• 23c1b10a4c [VLM][Bugfix] Multi-modal processor compatible with V1 multi-input (#11674) Cyrus Leung 2025-01-02 17:00:00 +08:00
• a115ac46b5 [VLM] Move supported limits and max tokens to merged multi-modal processor (#11669) Cyrus LeungandIsotr0py 2025-01-01 23:44:42 +08:00
• 73001445fb [V1] Implement Cascade Attention (#11635) Woosuk Kwon 2025-01-01 21:56:46 +09:00
• 6d70198b17 [Doc] Fix typo (#11666) Kazuhiro Serizawa 2025-01-01 17:10:10 +09:00
• f962f426bc [Misc] Replace space with - in the file names (#11667) Lu Fang 2024-12-31 23:39:30 -08:00
• 11d8a091c6 [Misc] Optimize Qwen2-VL LoRA test (#11663) Jee Jee Li 2025-01-01 14:42:23 +08:00
• 365801fedd [VLM] Add max-count checking in data parser for single image models (#11661) Cyrus LeungandRoger Wang 2025-01-01 14:15:21 +08:00
• 4db72e57f6 [Bugfix][Refactor] Unify model management in frontend (#11660) Joe Runde 2024-12-31 18:21:51 -08:00
• 0c6f998554 [Benchmark] Add benchmark script for CPU offloading (#11533) Yihua ChengandKuntaiDu 2024-12-31 18:10:55 -06:00
• e7c7c5e822 [V1][VLM] V1 support for selected single-image models. (#11632) 3 people 2024-12-31 13:17:22 -08:00
• 8c3230d8c1 [V1] Simpify vision block hash for prefix caching by removing offset from hash (#11646) Chen Zhang 2024-12-31 16:56:01 +08:00
• 2c5718809b [Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. (#11565) sakunkun 2024-12-31 14:29:04 +08:00
• 82c49d3260 [Misc][LoRA] Support Rank Stabilized LoRA (RSLoRA) (#6909) John GiorgiandJee Jee Li 2024-12-31 01:15:58 -05:00
• 74fa1d123c [Bugfix] Fix OpenAI parallel sampling when using xgrammar (#11637) Michael Goin 2024-12-30 22:43:54 -05:00
• a2a40bcd0d [Model][LoRA]LoRA support added for MolmoForCausalLM (#11439) 3 people 2024-12-31 02:33:06 +01:00
• ccb1aabcca [benchmark] Remove dependency for H100 benchmark step (#11572) Kevin H. Luu 2024-12-30 12:27:07 -08:00
• 36e7670045 [Bugfix] Validate and concatenate image embeddings in MiniCPMVBaseModel (#11631) whyiug 2024-12-31 02:51:04 +08:00
• 5886aa496e [V1] [6/N] API Server: Better Shutdown (#11586) Robert Shaw 2024-12-30 10:51:02 -05:00
• 8d9b6721e7 [VLM] Abstract out multi-modal data parsing in merged processor (#11620) Cyrus Leung 2024-12-30 23:01:35 +08:00
• b12e87f942 [platforms] enable platform plugins (#11602) youkaichao 2024-12-30 20:24:45 +08:00
• 5dbf854553 [CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels (#11618) Li, Jiang 2024-12-30 18:17:04 +08:00
• 970d6d0776 [Build][Kernel] Update CUTLASS to v3.6.0 (#11607) Tyler Michael Smith 2024-12-30 04:22:13 -05:00
• 628ec6c17b [Docker] bump up neuron sdk v2.21 (#11593) Liangfu Chen 2024-12-29 21:46:14 -08:00
• 3682e33f9f [v1] fix compilation cache (#11598) youkaichao 2024-12-30 12:24:12 +08:00
• 0aa38d16f5 Remove print statement in DeepseekScalingRotaryEmbedding (#11604) Michael Goin 2024-12-29 15:16:46 -05:00
• faef77c0d6 [Misc] KV cache transfer connector registry (#11481) Kuntai Du 2024-12-29 10:08:09 -06:00
• dba4d9dec6 [v1][bugfix] fix cudagraph with inplace buffer assignment (#11596) youkaichao 2024-12-29 17:03:49 +08:00
• 32b4c63f02 [Doc] Convert list tables to MyST (#11594) Cyrus Leung 2024-12-29 15:56:22 +08:00
• 4fb8e329fd [V1] [5/N] API Server: unify Detokenizer and EngineCore input (#11545) Robert Shaw 2024-12-28 15:51:57 -05:00
• 328841d002 [bugfix] interleaving sliding window for cohere2 model (#11583) youkaichao 2024-12-29 00:55:42 +08:00
• d427e5cfda [Doc] Minor documentation fixes (#11580) Cyrus Leung 2024-12-28 21:53:59 +08:00
• 42bb201fd6 [V1][Minor] Set pin_memory=False for token_ids_cpu tensor (#11581) Woosuk Kwon 2024-12-28 22:33:12 +09:00
• 59d6bb4c86 [Hardware][AMD]: Replace HIPCC version with more precise ROCm version (#11515) hj-wei 2024-12-28 19:17:35 +08:00
• b7dcc003dc [Model] Remove hardcoded image tokens ids from Pixtral (#11582) Roger Wang 2024-12-28 02:54:23 -08:00
• d34be24bb1 [Model] Support InternLM2 Reward models (#11571) Isotr0pyandCyrus Leung 2024-12-28 14:14:10 +08:00
• b5cbe8eeb3 [Bugfix] Last token measurement fix (#11376) Rajveer BachkaniwalaandRoger Wang 2024-12-27 22:34:46 -05:00
• df04dffade [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) Robert Shaw 2024-12-27 20:45:08 -05:00
• a60731247f [Doc] Update mllama example based on official doc (#11567) Chen Zhang 2024-12-28 08:31:10 +08:00
• ac79799403 [Bugfix] Fix for ROCM compressed tensor support (#11561) Selali 2024-12-27 12:12:11 -08:00
• dde1fa18c9 [Misc] Improve BNB loader to handle mixture of sharded and merged weights with same suffix (#11566) Isotr0py 2024-12-28 03:45:13 +08:00
• 0240402c46 [Misc]Add BNB quantization for MolmoForCausalLM (#11551) Jee Jee Li 2024-12-28 02:48:24 +08:00
• 55509c2114 [MODEL] LoRA support for Jamba model (#11209) ErezSC42 2024-12-27 19:58:21 +02:00
• 101418096f [VLM] Support caching in merged multi-modal processor (#11396) Cyrus Leung 2024-12-28 01:22:48 +08:00
• 5ce4627a7e [Doc] Add xgrammar in doc (#11549) Chen1022 2024-12-27 21:05:10 +08:00
• 7af553ea30 [Misc] Abstract the logic for reading and writing media content (#11527) Cyrus Leung 2024-12-27 19:21:23 +08:00
• 2c9b8ea2b0 [Bugfix] Fix TeleChat2ForCausalLM weights mapper (#11546) Jee Jee Li 2024-12-27 18:39:15 +08:00
• d003f3ea39 Update deploying_with_k8s.md with AMD ROCm GPU example (#11465) AlexHe99andCyrus Leung 2024-12-27 18:00:04 +08:00
• 6c6f7fe8a8 [Platform] Move model arch check to platform (#11503) Mengqing Cao 2024-12-27 16:45:25 +08:00
• 2339d59f92 [BugFix] Fix quantization for all other methods (#11547) Robert Shaw 2024-12-27 01:23:29 -05:00
• 1b875a0ef3 [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (#11534) Robert Shaw 2024-12-27 00:19:21 -05:00
• eb881ed006 [misc] fix typing (#11540) youkaichao 2024-12-27 11:05:08 +08:00
• 46d4359450 [CI] Fix broken CI (#11543) Robert Shaw 2024-12-26 21:49:16 -05:00
• 81b979f2a8 [V1] Fix yapf (#11538) Woosuk Kwon 2024-12-27 09:47:10 +09:00
• 371d04d39b [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (#11394) Woosuk Kwon 2024-12-27 09:32:38 +09:00
• 0c0c2015c5 Update openai_compatible_server.md (#11536) Robert ShawandSimon Mo 2024-12-26 19:26:18 -05:00
• 82d24f7aac [Docs] Document Deepseek V3 support (#11535) Simon Mo 2024-12-26 16:21:56 -08:00
• f49777ba62 Deepseek v3 (#11502) 3 people 2024-12-26 16:09:44 -08:00
• 55fb97f7bd [2/N] API Server: Avoid ulimit footgun (#11530) Robert Shaw 2024-12-26 18:43:05 -05:00
• 2072924d14 [Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) 4 people 2024-12-26 18:33:30 -05:00
• 720b10fdc6 [1/N] API Server (Remove Proxy) (#11529) Robert Shaw 2024-12-26 18:03:43 -05:00
• b85a977822 [Doc] Add video example to openai client for multimodal (#11521) Isotr0pyandCyrus Leung 2024-12-27 01:31:29 +08:00
• eec906d811 [Misc] Add placeholder module (#11501) Cyrus Leung 2024-12-26 21:12:51 +08:00
• f57ee5650d [Model] Modify MolmoForCausalLM MLP (#11510) Jee Jee Li 2024-12-26 21:12:05 +08:00
• dcb1a944d4 [V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681) sroy745andWoosuk Kwon 2024-12-26 02:02:58 -08:00
• 7492a36207 [Doc] Add QVQ and QwQ to the list of supported models (#11509) Roger WangandCyrus Leung 2024-12-26 01:44:32 -08:00
• aa25985bd1 [Misc][LoRA] Fix LoRA weight mapper (#11495) Jee Jee Li 2024-12-26 15:52:48 +08:00
• dbeac95dbb Mypy checking for vllm/compilation (#11496) Lucas Tuckerandlucast2021 2024-12-25 23:04:07 -06:00
• 51a624bf02 [Misc] Move some multimodal utils to modality-specific modules (#11494) Cyrus Leung 2024-12-26 12:23:20 +08:00
• 6ad909fdda [Doc] Improve GitHub links (#11491) Cyrus Leung 2024-12-26 06:49:26 +08:00
• b689ada91e [Frontend] Enable decord to load video from base64 (#11492) Cyrus Leung 2024-12-26 00:33:55 +08:00
• fc601665eb [Misc] Update disaggregation benchmark scripts and test logs (#11456) Jiaxin Shan 2024-12-24 22:58:48 -08:00
• 9832e5572a [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (#11472) Rui Qiao 2024-12-24 19:49:46 -08:00
• 3f3e92e1f2 [Model] Automatic conversion of classification and reward models (#11469) Cyrus Leung 2024-12-25 02:22:22 +08:00
• 409475a827 [Bugfix] Fix issues in CPU build Dockerfile. Fixes #9182 (#11435) Yuan Tang 2024-12-24 11:53:28 -05:00
• 196c34b0ac [Misc] Move weights mapper (#11443) Jee Jee Li 2024-12-24 21:05:25 +08:00
• 5c7963249d [attn][tiny fix] fix attn backend in MultiHeadAttention (#11463) Mengqing Cao 2024-12-24 20:39:36 +08:00
• 461cde2080 [OpenVINO] Fixed installation conflicts (#11458) Ilya Lavrenov 2024-12-24 15:38:21 +04:00
• 7a5286cc04 [Bugfix][Hardware][CPU] Fix CPU input_positions creation for text-only inputs with mrope (#11434) Isotr0py 2024-12-24 17:59:51 +08:00
• b1b1038fbd [Bugfix] Fix Qwen2-VL LoRA weight loading (#11430) Jee Jee Li 2024-12-24 17:56:10 +08:00
• 9edca6bf8f [Frontend] Online Pooling API (#11457) Cyrus Leung 2024-12-24 17:54:30 +08:00
• 4f074fbf53 [Misc]Suppress irrelevant exception stack trace information when CUDA… (#11438) dpxaandshiquan 2024-12-24 16:43:39 +08:00
• a491d6f535 [V1] TP Ray executor (#11107) Rui Qiao 2024-12-23 15:00:12 -08:00
• 32aa2059ad [Docs] Convert rST to MyST (Markdown) (#11145) Rafael Vasquez 2024-12-23 17:35:38 -05:00
• 94d545a1a1 [Doc] Fix typo in the help message of '--guided-decoding-backend' (#11440) yansh97 2024-12-24 04:20:44 +08:00
• 60fb4f3bcf [Bugfix] Add kv cache scales to gemma2.py (#11269) Michael Goin 2024-12-23 14:30:45 -05:00
• 63afbe9215 [CI] Expand OpenAI test_chat.py guided decoding tests (#11048) Michael Goin 2024-12-23 13:35:38 -05:00
• 8cef6e02dc [Misc] add w8a8 asym models (#11075) Dipika Sikka 2024-12-23 13:33:20 -05:00
• b866cdbd05 [Misc] Add assertion and helpful message for marlin24 compressed models (#11388) Dipika Sikka 2024-12-23 13:23:38 -05:00
• 2e726680b3 [Bugfix] torch nightly version in ROCm installation guide (#11423) Yuan Tang 2024-12-23 12:20:22 -05:00