Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• 9a88f89799 custom allreduce + torch.compile (#10121) Sage Mooreandyoukaichao 2024-11-26 00:00:16 -06:00
• 519e8e4182 [v1] EngineArgs for better config handling for v1 (#10382) Ricky Xu 2024-11-25 21:09:43 -08:00
• a6760f6456 [Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228) 3 people 2024-11-26 08:02:39 +05:30
• 45ac4ff270 [bugfix] fix aria model and add torch.compile (#10645) youkaichao 2024-11-25 18:32:09 -08:00
• 6e9ff050c8 [misc] do not read HOST_IP (#10644) youkaichao 2024-11-25 17:04:50 -08:00
• 9db713a1dc [Model] Add OLMo November 2024 model (#10503) Shane A 2024-11-25 14:26:40 -08:00
• 1b583cfefa [Doc] Fix typos in docs (#10636) Cyrus Leung 2024-11-26 02:15:45 +08:00
• cf73f0c95e [Model] Enable optional prefix when loading embedding models (#10639) Cyrus Leung 2024-11-26 02:14:33 +08:00
• b1d920531f [Model]: Add support for Aria model (#10514) zhou fanandIsotr0py 2024-11-26 02:10:55 +08:00
• 452a4e80c3 [Docs] Add Snowflake Slides (#10641) Simon Mo 2024-11-25 09:34:46 -08:00
• c27df94e1f [Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850) Wallas HenriqueandMichael Goin 2024-11-25 14:23:32 -03:00
• d04b13a380 [Bug]: Authorization ignored when root_path is set (#10606) Chauncey 2024-11-26 00:21:41 +08:00
• 2b0879bfc2 Super tiny little typo fix (#10633) fzyzcjy 2024-11-25 21:08:30 +08:00
• ed46f14321 [Model] Support is_causal HF config field for Qwen2 model (#10621) Cyrus Leung 2024-11-25 17:51:20 +08:00
• 05d1f8c9c6 [misc] move functions to config.py (#10624) youkaichao 2024-11-25 01:27:30 -08:00
• 25d806e953 [misc] add torch.compile compatibility check (#10618) youkaichao 2024-11-24 23:40:08 -08:00
• 65813781a2 [torch.compile] add warning for unsupported models (#10622) youkaichao 2024-11-24 23:27:51 -08:00
• 7c2134beda [torch.compile] force inductor threads (#10620) Jee Jee Li 2024-11-25 15:04:21 +08:00
• a30a605d21 [Doc] Add encoder-based models to Supported Models page (#10616) Cyrus Leung 2024-11-25 14:34:07 +08:00
• 571841b7fc [torch.compile] support encoder based models (#10613) youkaichao 2024-11-24 21:24:33 -08:00
• 7ea3cd7c3e [Refactor][MISC] del redundant code in ParallelConfig.postinit (#10614) Mengqing Cao 2024-11-25 13:14:56 +08:00
• 214efc2c3c Support Cross encoder models (#10400) Maximilien de BayserandFlavia Beo 2024-11-24 23:56:20 -03:00
• 49628fe13e [Doc] Update README.md with Ray Summit talk links (#10610) Zhuohan Li 2024-11-24 16:45:09 -08:00
• e4fbb14414 [doc] update the code to add models (#10603) youkaichaoandCyrus Leung 2024-11-24 11:21:40 -08:00
• c055747867 [model][utils] add extract_layer_index utility function (#10599) youkaichao 2024-11-23 22:22:54 -08:00
• eda2b3589c Revert "Print running script to enhance CI log readability" (#10601) youkaichao 2024-11-23 21:31:47 -08:00
• 1c445dca51 [CI/Build] Print running script to enhance CI log readability (#10594) Jee Jee Li 2024-11-24 11:57:13 +08:00
• 1700c543a5 [Bugfix] Fix LoRA weight sharding (#10450) Jee Jee LiandCyrus Leung 2024-11-24 09:23:17 +08:00
• 17d8fc1806 [bugfix] Fix example/tensorize_vllm_model tests (#10595) Jee Jee Li 2024-11-24 09:22:33 +08:00
• 04668ebe7a [Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) Isotr0py 2024-11-24 02:12:20 +08:00
• 651f6c31ac For ppc64le, disabled tests for now and addressed space issues (#10538) Nishidha 2024-11-23 15:03:53 +05:30
• 86a44fb896 [Platforms] Refactor openvino code (#10573) JiHuazhong 2024-11-23 14:23:12 +08:00
• 4cfe5d2bca [Bugfix] multi_modal_kwargs broadcast for CPU tensor parallel (#10541) Isotr0py 2024-11-23 13:25:46 +08:00
• c8acd80548 [2/N] handling placeholders in merged multi-modal processor (#10485) Cyrus Leung 2024-11-23 13:25:09 +08:00
• 4634a89d18 Prefix Cache Aware Scheduling [1/n] (#10128) Ricky Xu 2024-11-22 21:15:55 -08:00
• 7c25fe45a6 [AMD] Add support for GGUF quantization on ROCm (#10254) kliuae 2024-11-23 13:14:49 +08:00
• 02a43f82a9 Update default max_num_batch_tokens for chunked prefill to 2048 (#10544) Michael Goin 2024-11-23 00:14:19 -05:00
• cfea9c04ef [Model] Fix Baichuan BNB online quantization (#10572) Chen Wu 2024-11-23 13:13:59 +08:00
• 7d8ffb344f [Bugfix] Internal Server Error when tool_choice is incorrect. (#10567) Varun Vinayak Shenoy 2024-11-22 21:13:29 -08:00
• 4aba6e3d1a [core] gemma2 full context length support (#10584) youkaichao 2024-11-22 20:13:54 -08:00
• 978b39744b [Misc] Add pynccl wrappers for all_gather and reduce_scatter (#9432) Tyler Michael Smith 2024-11-22 22:14:03 -05:00
• ebda51968b [Core] Fix broken log configuration (#10458) Russell Bryant 2024-11-22 21:23:51 -05:00
• 9195dbdbca [Bugfix][Frontend] Update Llama Chat Templates to also support Non-Tool use (#10164) Travis Johnson 2024-11-22 19:17:38 -07:00
• d559979c54 [bugfix] fix cpu tests (#10585) youkaichao 2024-11-22 17:34:03 -08:00
• d345f409b7 [V1] EngineCore supports profiling (#10564) Zhonghua Deng 2024-11-23 09:16:15 +08:00
• 28598f3939 [Core] remove temporary local variables in LLMEngine.__init__ (#10577) Russell Bryant 2024-11-22 19:22:53 -05:00
• 948c859571 support bitsandbytes quantization with qwen model (#10549) zixuanzhang226 2024-11-22 16:16:14 -08:00
• 97814fbf0f [v1] Refactor KVCacheManager for more hash input than token ids (#10507) Ricky XuandCody Yu 2024-11-22 15:27:25 -08:00
• eebad39f26 [torch.compile] support all attention backends (#10558) youkaichao 2024-11-22 14:04:42 -08:00
• db100c5cde [bugfix] fix full graph tests (#10581) youkaichao 2024-11-22 10:02:14 -08:00
• 11fcf0e066 Remove token-adding chat embedding params (#10551) Noam Gat 2024-11-22 09:59:47 +02:00
• b6374e09b0 [Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948) Isotr0py 2024-11-22 15:01:56 +08:00
• a111d0151f [platforms] absorb worker cls difference into platforms folder (#10555) youkaichaoandNick Hill 2024-11-21 21:00:32 -08:00
• 446c7806b2 [Minor] Fix line-too-long (#10563) Woosuk Kwon 2024-11-21 19:40:40 -08:00
• 33e0a2540a [9/N] torch.compile LLM usage (#10552) youkaichao 2024-11-21 19:13:31 -08:00
• aed074860a [Benchmark] Add new H100 machine (#10547) Simon Mo 2024-11-21 18:27:20 -08:00
• 9afa014552 Add small example to metrics.rst (#10550) Michael Goin 2024-11-21 18:43:43 -05:00
• 46fe9b46d8 [Minor] Revert change in offline inference example (#10545) Woosuk Kwon 2024-11-21 13:28:16 -08:00
• cf656f5a02 [misc] improve error message (#10553) youkaichao 2024-11-21 13:13:17 -08:00
• edec3385b6 [CI][Installation] Avoid uploading CUDA 11.8 wheel (#10535) Yunmengandsimon-mo 2024-11-22 05:03:58 +08:00
• f9310cbd0c [V1] Fix Compilation config & Enable CUDA graph by default (#10528) Woosuk Kwon 2024-11-21 12:53:39 -08:00
• 7560ae5caf [8/N] enable cli flag without a space (#10529) youkaichao 2024-11-21 12:30:42 -08:00
• e7a8341c7c [Bugfix] Allow token ID-only inputs in Qwen2-Audio (#10536) Cyrus Leung 2024-11-22 02:09:43 +08:00
• c51e397fe8 [Misc] Suppress duplicated logging regarding multimodal input pipeline (#10530) Roger Wang 2024-11-21 09:21:31 -08:00
• 2385b60d83 [Kernel] Register punica ops directly (#10522) Jee Jee Li 2024-11-22 01:18:11 +08:00
• da7e702c6f [Bug]: When apply continue_final_message for OpenAI server, the "echo":false is ignored (#10180) Chauncey 2024-11-22 00:24:32 +08:00
• 4d676f0852 [Bugfix] Embedding model pooling_type equals ALL and multi input's bug (#10494) Xiaoyu Zhang 2024-11-21 22:40:02 +08:00
• d5ec121f95 [Model] Expose dynamic_image_size as mm_processor_kwargs for InternVL2 models (#10518) Isotr0py 2024-11-21 22:20:08 +08:00
• 8a93a598d9 fix the issue that len(tokenizer(prompt)["input_ids"]) > prompt_len (#10524) Wang, Yi 2024-11-21 19:15:36 +08:00
• 1cfde82ffd [Model] Add Support for Multimodal Granite Models (#10291) Alex BrooksandCyrus Leung 2024-11-21 03:46:20 -07:00
• f0e0238016 [Doc] fix a small typo in docstring of llama_tool_parser (#10513) Zhong Qishuai 2024-11-21 17:05:23 +08:00
• aaddce5d26 [platforms] improve error message for unspecified platforms (#10520) youkaichao 2024-11-20 23:07:56 -08:00
• 3430857b64 [Misc] Increase default video fetch timeout (#10495) Cyrus Leung 2024-11-21 15:06:42 +08:00
• 8b0fe06c89 [torch.compile] Inductor code caching fix (#10273) Luka Govedič 2024-11-21 00:44:57 -05:00
• 9d827170a3 [Platforms] Add device_type in Platform (#10508) Mengqing Cao 2024-11-21 12:44:20 +08:00
• 6c1208d083 [Core] Add Sliding Window Support with Flashinfer (#10462) Pavani Majety 2024-11-20 19:56:47 -08:00
• 388ee3de66 [torch.compile] limit inductor threads and lazy import quant (#10482) youkaichao 2024-11-20 18:36:33 -08:00
• 2f77b6cfec [TPU] Implement prefix caching for TPUs (#10307) Woosuk Kwon 2024-11-20 13:54:15 -08:00
• c68f7ede6a [Bugfix]: allow extra fields in requests to openai compatible server (#10463) Guillaume Calmettes 2024-11-20 22:42:21 +01:00
• 0cd3d9717e [7/N] torch.compile, reduce compilation time (#10460) youkaichao 2024-11-20 11:20:38 -08:00
• 5f1d6af2b6 [perf bench] H200 development (#9768) Simon Mo 2024-11-20 11:06:56 -08:00
• 772a66732d [platforms] restore xpu check for parallel config (#10479) youkaichao 2024-11-20 09:13:28 -08:00
• 63f1fde277 [Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) Li, Jiang 2024-11-20 18:57:39 +08:00
• d5b28447e0 [Platforms] Refactor xpu code (#10468) Mengqing Cao 2024-11-20 14:52:13 +08:00
• 09dbf9ff16 [Bugfix] Handle conflicts between modern and legacy fields (#10471) Cyrus Leung 2024-11-20 14:45:08 +08:00
• 343041c4c4 [model] Reduce medusa weight (#10454) Sky Lee 2024-11-20 14:05:55 +08:00
• ed701ca963 [ci/build] Combine nightly and optional (#10465) Kevin H. Luu 2024-11-19 19:36:03 -10:00
• 7629a9c6e5 [CI/Build] Support compilation with local cutlass path (#10423) (#10424) wchen61 2024-11-20 13:35:50 +08:00
• 709c9f1f25 [CI/Build] Add sphinx/rst linter for docs (#10366) Rafael Vasquez 2024-11-20 00:35:31 -05:00
• b4be5a8adb [Bugfix] Enforce no chunked prefill for embedding models (#10470) Cyrus Leung 2024-11-20 13:12:51 +08:00
• ad44437ba3 [Bugfix] Fix Mamba model initialization and MLP Speculator weights loading (#10456) Isotr0py 2024-11-20 13:04:05 +08:00
• 9e05252b46 [Misc] Add __setitem__ for LazyDict (#10469) Yanyi Liu 2024-11-20 12:44:57 +08:00
• d200972e7f [Bugfix] Marlin 2:4 temp fix for large M dim (>256) (#10464) Lucas Wilkinson 2024-11-19 22:40:33 -05:00
• d5b68aba2f [CI/Build] Update Dockerfile.rocm (#10434) Alexei-V-Ivanov-AMD 2024-11-19 19:19:59 -06:00
• a324d3a1a7 Change granite chat template to keep json list formatting for tool calls (#10452) Maximilien de Bayser 2024-11-19 22:16:54 -03:00
• b00b33d77e [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) ElizaWszola 2024-11-19 22:31:12 +01:00
• efa9084628 [Core] Avoid metrics log noise when idle (#8868) Russell Bryant 2024-11-19 16:05:25 -05:00
• 803f37eaaa [6/N] torch.compile rollout to users (#10437) youkaichao 2024-11-19 10:09:03 -08:00
• fd9f124971 [Doc] fix link for page that was renamed (#10455) Russell Bryant 2024-11-19 12:48:30 -05:00
• 1ea291a417 Fix: Build error seen on Power Architecture (#10421) +717 people 2024-11-19 23:04:57 +05:30