Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• f93bf2b189 [Bugfix][CI][CPU] add missing datasets package to requirements-cpu.txt (#11159) Li, JiangandGitHub 2024-12-13 16:50:35 +08:00
• 7cd7409142 PaliGemma 2 support (#11142) Jani MonosesandGitHub 2024-12-13 09:40:07 +02:00
• be39e3cd18 [core] clean up cudagraph batchsize padding logic (#10996) youkaichaoandGitHub 2024-12-12 22:57:50 -08:00
• 34f1a806d5 [Bugfix][V1] Fix 'NoneType' object has no attribute 'hash_value' (#11157) Cody YuandGitHub 2024-12-12 22:30:06 -08:00
• 00c1bde5d8 [ROCm][AMD] Disable auto enabling chunked prefill on ROCm (#11146) Gregory ShtrasbergandGitHub 2024-12-13 00:31:26 -05:00
• 3989a79824 [Bugfix] Update starcoder2 to remap k/v scale names for kv_cache quantization (#11148) Dipika SikkaandGitHub 2024-12-13 00:07:20 -05:00
• 1efce68605 [Bugfix] Use runner_type instead of task in GritLM (#11144) Pooya DavoodiandGitHub 2024-12-12 20:09:53 -08:00
• 30870b4f66 [torch.compile] Dynamic fp8 + rms_norm fusion (#10906) 3 people 2024-12-12 22:19:23 -05:00
• 78ed8f57d8 [Misc][V1] Fix type in v1 prefix caching (#11151) Cody YuandGitHub 2024-12-12 16:57:40 -08:00
• db6c264a1e [Bugfix] Fix value unpack error of simple connector for KVCache transfer. (#11058) shangmingcandGitHub 2024-12-13 05:19:17 +08:00
• 9f3974a319 Fix logging of the vLLM Config (#11143) Jeremy ArnoldandGitHub 2024-12-12 14:05:57 -06:00
• 2c97eca1ff [Misc] Validate grammar and fail early (#11119) Cody YuandGitHub 2024-12-12 10:34:26 -08:00
• 5d712571af [Bugfix] Quick fix to make Pixtral-HF load correctly again after 39e227c7ae. (#11024) Jeff CookandGitHub 2024-12-12 11:09:20 -07:00
• d4d5291cc2 fix(docs): typo in helm install instructions (#11141) Ramon ZiaiandGitHub 2024-12-12 18:36:32 +01:00
• 4816d20aa4 [V1] Fix torch profiling for offline inference (#11125) Roger WangandGitHub 2024-12-12 07:51:53 -08:00
• 85362f028c [Misc][LoRA] Ensure Lora Adapter requests return adapter name (#11094) 3 people 2024-12-12 01:25:16 -08:00
• 62de37a38e [core][distributed] initialization from StatelessProcessGroup (#10986) youkaichaoandGitHub 2024-12-12 01:04:19 -08:00
• 8195824206 [Hardware][Intel-Gaudi] Enable LoRA support for Intel Gaudi (HPU) (#10565) Sanju C SudhakaranandGitHub 2024-12-12 13:39:28 +05:30
• f092153fbe [V1] Use more persistent buffers to optimize input preparation overheads (#11111) Woosuk KwonandGitHub 2024-12-11 23:14:20 -08:00
• 1da8f0e1dd [Model] Add support for embedding model GritLM (#10816) Pooya DavoodiandGitHub 2024-12-11 22:39:16 -08:00
• ccede2b264 [Core] cleanup zmq ipc sockets on exit (#11115) Russell BryantandGitHub 2024-12-11 22:12:24 -05:00
• 24a36d6d5f Update link to LlamaStack remote vLLM guide in serving_with_llamastack.rst (#11112) Yuan TangandGitHub 2024-12-11 21:39:21 -05:00
• 8fb26dac61 [Docs] Add media kit (#11121) Simon MoandGitHub 2024-12-11 17:33:11 -08:00
• 7439a8b5fc [Bugfix] Multiple fixes to tool streaming with hermes and mistral (#10979) ClaytonandGitHub 2024-12-11 17:10:12 -08:00
• 4e11683368 [V1] VLM preprocessor hashing (#11020) 4 people 2024-12-11 19:55:30 -05:00
• 452a723bf2 [V1][Core] Remove should_shutdown to simplify core process termination (#11113) Tyler Michael SmithandGitHub 2024-12-11 18:34:54 -05:00
• d1e21a979b [CI/Build] Split up VLM tests (#11083) Cyrus LeungandGitHub 2024-12-12 06:18:16 +08:00
• 72ff3a9686 [core] Bump ray to use _overlap_gpu_communication in compiled graph tests (#10410) 3 people 2024-12-11 11:36:35 -08:00
• 66aaa7722d [torch.compile] remove graph logging in ci (#11110) youkaichaoandGitHub 2024-12-11 10:59:50 -08:00
• d643c2aba1 [V1] Use input_ids as input for text-only models (#11032) Woosuk KwonandGitHub 2024-12-11 10:49:23 -08:00
• 91642db952 [torch.compile] use depyf to dump torch.compile internals (#10972) youkaichaoandGitHub 2024-12-11 10:43:05 -08:00
• fd22220687 [Doc] Installed version of llmcompressor for int8/fp8 quantization (#11103) 3 people 2024-12-11 23:43:24 +08:00
• b2f775456e [CI/Build] Enable prefix caching test for AMD (#11098) hissu-hyvarinenandGitHub 2024-12-11 17:23:37 +02:00
• cad5c0a6ed [Doc] Update docs to refer to pooling models (#11093) Cyrus LeungandGitHub 2024-12-11 21:36:27 +08:00
• 8f10d5e393 [Misc] Split up pooling tasks (#10820) Cyrus LeungandGitHub 2024-12-11 17:28:00 +08:00
• 40766ca1b8 [Bugfix]: Clamp -inf logprob values in prompt_logprobs (#11073) Rafael VasquezandGitHub 2024-12-11 04:27:39 -05:00
• 2e32f5d28d [Bugfix] Fix Idefics3 fails during multi-image inference (#11080) B-201andGitHub 2024-12-11 17:27:07 +08:00
• 61b1d2f6ae [Core] v1: Use atexit to handle engine core client shutdown (#11076) Russell BryantandGitHub 2024-12-11 04:26:36 -05:00
• 9974fca047 [ci/build] Fix entrypoints test and pin outlines version (#11088) Kevin H. LuuandGitHub 2024-12-11 01:01:53 -08:00
• 3fb4b4f163 [ci/build] Fix AMD CI dependencies (#11087) Kevin H. LuuandGitHub 2024-12-11 00:39:53 -08:00
• 2e33fe4191 [CI/Build] Check transformers v4.47 (#10991) Cyrus LeungandGitHub 2024-12-11 13:02:02 +08:00
• e39400a4b6 Fix streaming for granite tool call when <|tool_call|> is present (#11069) Maximilien de BayserandGitHub 2024-12-11 01:51:40 -03:00
• ffa48c9146 [Model] PP support for Mamba-like models (#10992) Mor ZusmanandGitHub 2024-12-11 04:53:37 +02:00
• d5c5154fcf [Misc] LoRA + Chunked Prefill (#9057) Aurick QiaoandGitHub 2024-12-10 21:09:20 -05:00
• 9a93973708 [Bugfix] Fix Mamba multistep (#11071) Tyler Michael SmithandGitHub 2024-12-10 19:16:22 -05:00
• 134810b3d9 [V1][Bugfix] Always set enable_chunked_prefill = True for V1 (#11061) Woosuk KwonandGitHub 2024-12-10 14:41:23 -08:00
• 75f89dc44c [torch.compile] add a flag to track batchsize statistics (#11059) youkaichaoandGitHub 2024-12-10 12:40:52 -08:00
• e739194926 [Core] Update to outlines >= 0.1.8 (#10576) Russell BryantandGitHub 2024-12-10 15:08:16 -05:00
• 250ee65d72 [BUG] Remove token param #10921 (#11022) Flávia BéoandGitHub 2024-12-10 14:38:15 -03:00
• 9b9cef3145 [Bugfix] Backport request id validation to v0 (#11036) Joe RundeandGitHub 2024-12-10 09:38:23 -07:00
• d05f88679b [Misc][LoRA] Add PEFTHelper for LoRA (#11003) Jee Jee LiandGitHub 2024-12-10 19:12:01 +08:00
• beb16b2c81 [Bugfix] Handle <|tool_call|> token in granite tool parser (#11039) Travis JohnsonandGitHub 2024-12-10 03:27:11 -07:00
• fe2e10c71b Add example of helm chart for vllm deployment on k8s (#9199) Maxime FourniouxandGitHub 2024-12-10 10:19:27 +01:00
• 82c73fd510 [Bugfix] cuda error running llama 3.2 (#11047) Gene Der SuandGitHub 2024-12-09 23:41:11 -08:00
• bfd610430c Update README.md (#11034) Diego MarinhoandGitHub 2024-12-10 18:08:10 +11:00
• e35879c276 [Bugfix] Fix xgrammar failing to read a vocab_size from LlavaConfig on PixtralHF. (#11043) Jeff CookandGitHub 2024-12-09 23:54:22 -07:00
• ebf778061d monitor metrics of tokens per step using cudagraph batchsizes (#11031) youkaichaoandGitHub 2024-12-09 22:35:36 -08:00
• 28b3a1c7e5 [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) Tyler Michael SmithandGitHub 2024-12-10 01:28:14 -05:00
• bc192a2b09 [Pixtral] Improve loading (#11040) Patrick von PlatenandGitHub 2024-12-10 07:09:32 +01:00
• 980ad394a8 [Frontend] Use request id from header (#10968) Joe RundeandGitHub 2024-12-09 22:46:29 -07:00
• 391d7b2763 [Bugfix] Fix usage of deprecated decorator (#11025) Cyrus LeungandGitHub 2024-12-10 13:45:47 +08:00
• d1f6d1c8af [Model] Add has_weight to RMSNorm and re-enable weights loading tracker for Mamba (#10739) Isotr0pyandGitHub 2024-12-10 10:23:07 +08:00
• 6d525288c1 [Docs] Add dedicated tool calling page to docs (#10554) 3 people 2024-12-09 20:15:34 -05:00
• 6faec54505 [V1] Do not store None in self.generators (#11038) Woosuk KwonandGitHub 2024-12-09 15:08:19 -08:00
• 5ed5d5f128 Build tpu image in release pipeline (#10936) 3 people 2024-12-09 15:07:48 -08:00
• b63ba84832 [ROCm][bugfix] scpecilative decoding worker class (#11035) Gregory ShtrasbergandGitHub 2024-12-09 17:00:29 -05:00
• 9c6459e4cb [Neuron] Upgrade neuron to 2.20.2 (#11016) 3 people 2024-12-09 22:53:24 +01:00
• 1a2f8fb828 [v1] fix use compile sizes (#11000) youkaichaoandGitHub 2024-12-09 13:47:24 -08:00
• cbcbdb1ceb [Bugfix][Hardware][Gaudi] Bump vllm_hpu_extension version (#11028) Konrad ZaworaandGitHub 2024-12-09 22:21:06 +01:00
• a811dd6608 [Model] merged input processor for Phi-3-Vision models (#10977) 3 people 2024-12-10 04:55:10 +08:00
• ca871491ed [Misc][LoRA] Abstract PunicaWrapper (#10955) Jee Jee LiandGitHub 2024-12-10 04:54:44 +08:00
• 3b61cb450d [V1] Further reduce CPU overheads in flash-attn (#10989) Woosuk KwonandGitHub 2024-12-09 12:38:46 -08:00
• edc4fa3188 [ci/build] Recompile CI dependencies list with Python 3.12 (#11013) Kevin H. LuuandGitHub 2024-12-09 11:46:58 -08:00
• 25b79d9fd3 [V1] Input Batch Relocation (#10962) 3 people 2024-12-09 12:33:41 -05:00
• aea2fc38c3 [Platform] Move async output check to platform (#10768) wangxiyuanandGitHub 2024-12-10 01:24:46 +08:00
• e691b26f6f [Core] Require xgrammar >= 0.1.6 (#11021) Russell BryantandGitHub 2024-12-09 11:44:27 -05:00
• c690357928 [V1] Fix Detokenizer loading in AsyncLLM (#10997) Roger WangandGitHub 2024-12-09 08:27:10 -08:00
• d1c2e15eb3 [torch.compile] add dynamo time tracking (#11005) youkaichaoandGitHub 2024-12-08 23:09:04 -08:00
• af7c4a92e6 [Doc][V1] Add V1 support column for multimodal models (#10998) Roger WangandGitHub 2024-12-08 22:29:16 -08:00
• 46004e83a2 [misc] clean up and unify logging (#10999) youkaichaoandGitHub 2024-12-08 17:28:27 -08:00
• 43b05fa314 [torch.compile][misc] fix comments (#10993) youkaichaoandGitHub 2024-12-08 11:18:18 -08:00
• a11f326528 [V1] Initial support of multimodal models for V1 re-arch (#10699) Roger WangandGitHub 2024-12-08 04:50:51 -08:00
• fd57d2b534 [torch.compile] allow candidate compile sizes (#10984) youkaichaoandGitHub 2024-12-08 03:05:21 -08:00
• 7be15d9356 [core][misc] remove use_dummy driver for _run_workers (#10920) youkaichaoandGitHub 2024-12-07 12:06:08 -08:00
• 1b62745b1d [core][executor] simplify instance id (#10976) youkaichaoandGitHub 2024-12-07 09:33:45 -08:00
• 78029b34ed [BugFix][Kernel]: fix illegal memory access in causal_conv1d when conv_states is None (#10928) zhou fanandGitHub 2024-12-08 01:21:18 +08:00
• c889d5888b [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975) Cyrus LeungandGitHub 2024-12-08 01:20:49 +08:00
• 39e227c7ae [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) Cyrus LeungandGitHub 2024-12-08 01:10:05 +08:00
• 1c768fe537 [Doc] Explicitly state that InternVL 2.5 is supported (#10978) Cyrus LeungandGitHub 2024-12-08 00:58:02 +08:00
• bf0e382e16 [Model] Composite weight loading for multimodal Qwen2 (#10944) Cyrus LeungandGitHub 2024-12-07 22:22:52 +08:00
• b26b4cd03c [Misc][LoRA] Refactor and clean MergedQKVParallelLinearWithLora implementation (#10958) Isotr0pyandGitHub 2024-12-07 18:33:49 +08:00
• f13cf9ad50 [Build] Fix for the Wswitch-bool clang warning (#10060) Gregory ShtrasbergandGitHub 2024-12-07 04:03:44 -05:00
• 955fa9533a [3/N] Support and implement merged input processor for LLaVA model (#10676) 3 people 2024-12-07 16:50:58 +08:00
• acf092d348 [Bugfix] Fix test-pipeline.yaml (#10973) Jee Jee LiandGitHub 2024-12-07 12:08:54 +08:00
• 69d357ba12 [Core] Cleanup startup logging a bit (#10961) Russell BryantandGitHub 2024-12-06 21:30:23 -05:00
• dcdc3fafe5 [ci] fix broken tests (#10956) youkaichaoandGitHub 2024-12-06 11:25:47 -08:00
• c05cfb67da [misc] fix typo (#10960) youkaichaoandGitHub 2024-12-06 11:25:20 -08:00
• 7406274041 [Doc] add KubeAI to serving integrations (#10837) Sam StoelingaandGitHub 2024-12-06 09:03:56 -08:00
• 8b59631855 [Core] Support Lark grammars for XGrammar (#10870) Michael GoinandGitHub 2024-12-06 10:34:29 -05:00
• a1887f2c96 [torch.compile] fix deprecated code (#10948) youkaichaoandGitHub 2024-12-06 03:01:23 -08:00