Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• eeffde1ac0 [TPU] Upgrade PyTorch XLA nightly (#7967) Woosuk KwonandGitHub 2024-08-28 13:10:21 -07:00
• e5697d161c [Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and awq_gemm to support AWQ (#7386) rasmithandGitHub 2024-08-28 14:37:47 -05:00
• b98cc28f91 [Core][Kernels] Use FlashInfer backend for FP8 KV Cache when available. (#7798) 3 people 2024-08-28 10:01:22 -07:00
• ef9baee3c5 [Bugfix][VLM] Fix incompatibility between #7902 and #7230 (#7948) Cyrus LeungandGitHub 2024-08-28 23:11:18 +08:00
• 98c12cffe5 [Doc] fix the autoAWQ example (#7937) Stas BekmanandGitHub 2024-08-28 05:12:32 -07:00
• f52a43a8b9 [ci][test] fix pp test failure (#7945) youkaichaoandGitHub 2024-08-28 01:27:07 -07:00
• e3580537a4 [Performance] Enable chunked prefill and prefix caching together (#7753) Cody YuandGitHub 2024-08-28 00:36:31 -07:00
• f508e03e7f [Core] Async_output_proc: Add virtual engine support (towards pipeline parallel) (#7911) Alexander MatveevandGitHub 2024-08-28 03:02:30 -04:00
• 51f86bf487 [mypy][CI/Build] Fix mypy errors (#7929) Cyrus LeungandGitHub 2024-08-28 14:47:44 +08:00
• c166e7e43e [Bugfix] Allow ScalarType to be compiled with pytorch 2.3 and add checks for registering FakeScalarType and dynamo support. (#7886) bnellnmandGitHub 2024-08-27 23:13:45 -04:00
• bc6e42a9b1 [hardware][rocm] allow rocm to override default env var (#7926) youkaichaoandGitHub 2024-08-27 19:50:06 -07:00
• fab5f53e2d [Core][VLM] Stack multimodal tensors to represent multiple images within each prompt (#7902) Peter SalasandGitHub 2024-08-27 18:53:56 -07:00
• 9c71c97ae2 [mypy] Enable mypy type checking for vllm/core (#7229) Jonathan BerkhahnandGitHub 2024-08-27 16:11:14 -07:00
• 5340a2dccf [Model] Add multi-image input support for LLaVA-Next offline inference (#7230) zifeitongandGitHub 2024-08-27 16:09:02 -07:00
• 345be0e244 [benchmark] Update TGI version (#7917) Philipp SchmidandGitHub 2024-08-28 00:07:53 +02:00
• fc911880cc [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766) 3 people 2024-08-27 18:07:09 -04:00
• ed6f002d33 [cuda][misc] error on empty CUDA_VISIBLE_DEVICES (#7924) youkaichaoandGitHub 2024-08-27 12:06:11 -07:00
• b09c755be8 [Bugfix] Fix phi3v incorrect image_idx when using async engine (#7916) Isotr0pyandGitHub 2024-08-28 01:36:09 +08:00
• 42e932c7d4 [CI/Build][ROCm] Enabling tensorizer tests for ROCm (#7237) alexeykondratandGitHub 2024-08-27 13:09:13 -04:00
• 076169f603 [Hardware][Intel GPU] Add intel GPU pipeline parallel support. (#7810) Kunshang JiandGitHub 2024-08-28 01:07:02 +08:00
• 9db642138b [CI/Build][VLM] Cleanup multiple images inputs model test (#7897) Isotr0pyandGitHub 2024-08-27 23:28:30 +08:00
• 6fc4e6e07a [Model] Add Mistral Tokenization to improve robustness and chat encoding (#7739) Patrick von PlatenandGitHub 2024-08-27 14:40:02 +02:00
• 9606c7197d Revert #7509 (#7887) Cody YuandGitHub 2024-08-27 00:16:31 -07:00
• 64cc644425 [core][torch.compile] discard the compile for profiling (#7796) youkaichaoandGitHub 2024-08-26 21:33:58 -07:00
• 39178c7fbc [Tests] Disable retries and use context manager for openai client (#7565) Nick HillandGitHub 2024-08-26 21:33:17 -07:00
• 2eedede875 [Core] Asynchronous Output Processor (#7049) 3 people 2024-08-26 20:53:20 -07:00
• 015e6cc252 [Misc] Update compressed tensors lifecycle to remove prefix from create_weights (#7825) Dipika SikkaandGitHub 2024-08-26 20:09:34 -04:00
• 760e9f71a8 [Bugfix] neuron: enable tensor parallelism (#7562) omrishivandGitHub 2024-08-26 15:13:13 -07:00
• 05826c887b [misc] fix custom allreduce p2p cache file generation (#7853) youkaichaoandGitHub 2024-08-26 15:02:25 -07:00
• dd9857f5fa [Misc] Update gptq_marlin_24 to use vLLMParameters (#7762) 3 people 2024-08-26 17:44:54 -04:00
• 665304092d [Misc] Update qqq to use vLLMParameters (#7805) Dipika SikkaandGitHub 2024-08-26 15:16:15 -04:00
• 2deb029d11 [Performance][BlockManagerV2] Mark prefix cache block as computed after schedule (#7822) Cody YuandGitHub 2024-08-26 11:24:53 -07:00
• 029c71de11 [CI/Build] Avoid downloading all HF files in RemoteOpenAIServer (#7836) Cyrus LeungandGitHub 2024-08-26 13:31:10 +08:00
• 0b769992ec [Bugfix]: Use float32 for base64 embedding (#7855) ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟andGitHub 2024-08-26 06:16:38 +03:00
• 1856aff4d6 [Spec Decoding] Streamline batch expansion tensor manipulation (#7851) Nick HillandGitHub 2024-08-25 15:45:14 -07:00
• 70c094ade6 [misc][cuda] improve pynvml warning (#7852) youkaichaoandGitHub 2024-08-25 14:30:09 -07:00
• 2059b8d9ca [Misc] Remove snapshot_download usage in InternVL2 test (#7835) Isotr0pyandGitHub 2024-08-25 23:53:09 +08:00
• 8aaf3d5347 [Model][VLM] Support multi-images inputs for Phi-3-vision models (#7783) Isotr0pyandGitHub 2024-08-25 19:51:20 +08:00
• 80162c44b1 [Bugfix] Fix Phi-3v crash when input images are of certain sizes (#7840) zifeitongandGitHub 2024-08-24 18:16:24 -07:00
• aab0fcdb63 [ci][test] fix RemoteOpenAIServer (#7838) youkaichaoandGitHub 2024-08-24 10:31:28 -07:00
• ea9fa160e3 [ci][test] exclude model download time in server start time (#7834) youkaichaoandGitHub 2024-08-24 01:03:27 -07:00
• 7d9ffa2ae1 [misc][core] lazy import outlines (#7831) youkaichaoandGitHub 2024-08-24 00:51:38 -07:00
• d81abefd2e [Frontend] add json_schema support from OpenAI protocol (#7654) Tyler RockwoodandGitHub 2024-08-24 01:07:24 -05:00
• 8da48e4d95 [Frontend] Publish Prometheus metrics in run_batch API (#7641) Pooya DavoodiandGitHub 2024-08-23 23:04:22 -07:00
• 6885fde317 [Bugfix] Fix run_batch logger (#7640) Pooya DavoodiandGitHub 2024-08-23 13:58:26 -07:00
• 9db93de20c [Core] Add multi-step support to LLMEngine (#7789) Alexander MatveevandGitHub 2024-08-23 15:45:53 -04:00
• 09c7792610 Bump version to v0.5.5 (#7823) Simon MoandGitHub 2024-08-23 11:35:33 -07:00
• f1df5dbfd6 [Misc] Update marlin to use vLLMParameters (#7803) Dipika SikkaandGitHub 2024-08-23 14:30:52 -04:00
• 35ee2ad6b9 [github][misc] promote asking llm first (#7809) youkaichaoandGitHub 2024-08-23 09:38:50 -07:00
• e25fee57c2 [BugFix] Fix server crash on empty prompt (#7746) Maximilien de BayserandGitHub 2024-08-23 10:12:44 -03:00
• faeddb565d [misc] Add Torch profiler support for CPU-only devices (#7806) Jie Fu (傅杰)andGitHub 2024-08-23 13:46:25 +08:00
• fc5ebbd1d3 [Hardware][Intel GPU] refactor xpu_model_runner for tp (#7712) Kunshang JiandGitHub 2024-08-23 11:06:54 +08:00
• c01a6cb231 [Ray backend] Better error when pg topology is bad. (#7584) 3 people 2024-08-22 17:44:25 -07:00
• b903e1ba7f [Frontend] error suppression cleanup (#7786) Joe RundeandGitHub 2024-08-22 15:50:21 -06:00
• a152246428 [Misc] fix typo in triton import warning (#7794) Siyuan LiuandGitHub 2024-08-22 13:51:23 -07:00
• 666ad0aa16 [ci] Cleanup & refactor Dockerfile to pass different Python versions and sccache bucket via build args (#7705) Kevin H. LuuandGitHub 2024-08-22 13:10:55 -07:00
• 15310b5101 [Bugfix] Use LoadFormat values for vllm serve --load-format (#7784) Michael GoinandGitHub 2024-08-22 14:37:08 -04:00
• 57792ed469 [Doc] Fix incorrect docs from #7615 (#7788) Peter SalasandGitHub 2024-08-22 10:02:06 -07:00
• d3b5b98021 [Misc] Enhance prefix-caching benchmark tool (#6568) Jiaxin ShanandGitHub 2024-08-23 00:32:02 +08:00
• cc0eaf12b1 [Bugfix] spec decode handle None entries in topk args in create_sequence_group_output (#7232) Travis JohnsonandGitHub 2024-08-22 07:33:48 -06:00
• 955b5191c9 [Misc] update fp8 to use vLLMParameter (#7437) Dipika SikkaandGitHub 2024-08-22 08:36:18 -04:00
• 55d63b1211 [Bugfix] Don't build machete on cuda <12.0 (#7757) Lucas WilkinsonandGitHub 2024-08-22 08:28:52 -04:00
• 4f419c00a6 Fix ShardedStateLoader for vllm fp8 quantization (#7708) Flex WangandGitHub 2024-08-22 05:25:04 -07:00
• a3fce56b88 [Speculative Decoding] EAGLE Implementation with Top-1 proposer (#6830) Abhinav GoyalandGitHub 2024-08-22 15:12:24 +05:30
• b3856bef7d [Misc] Use torch.compile for GemmaRMSNorm (#7642) Woosuk KwonandGitHub 2024-08-22 01:14:13 -07:00
• 8c6f694a79 [ci] refine dependency for distributed tests (#7776) youkaichaoandGitHub 2024-08-22 00:54:15 -07:00
• eeee1c3b1a [TPU] Avoid initializing TPU runtime in is_tpu (#7763) Woosuk KwonandGitHub 2024-08-21 21:31:49 -07:00
• aae74ef95c Revert "[Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527)" (#7764) Michael GoinandGitHub 2024-08-21 23:42:14 -04:00
• cde9183b40 [Bug][Frontend] Improve ZMQ client robustness (#7443) Joe RundeandGitHub 2024-08-21 20:18:11 -06:00
• df1a21131d [Model] Fix Phi-3.5-vision-instruct 'num_crops' issue (#7710) zifeitongandGitHub 2024-08-21 18:36:24 -07:00
• 7937009a7e [Kernel] Replaced blockReduce[...] functions with cub::BlockReduce (#7233) 3 people 2024-08-21 20:18:00 -04:00
• 9984605412 [AMD][CI/Build] Disambiguation of the function call for ROCm 6.2 headers compatibility (#7477) 3 people 2024-08-21 19:47:36 -04:00
• 7eebe8ccaa [distributed][misc] error on same VLLM_HOST_IP setting (#7756) youkaichaoandGitHub 2024-08-21 16:25:34 -07:00
• 8678a69ab5 [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527) 3 people 2024-08-21 19:17:10 -04:00
• 5844017285 [ci] [multi-step] narrow multi-step test dependency paths (#7760) William LinandGitHub 2024-08-21 15:52:40 -07:00
• 1ca0d4f86b [Model] Add UltravoxModel and UltravoxConfig (#7615) Peter SalasandGitHub 2024-08-21 15:49:39 -07:00
• dd53c4b023 [misc] Add Torch profiler support (#7451) 3 people 2024-08-21 15:39:26 -07:00
• 970dfdc01d [Frontend] Improve Startup Failure UX (#7716) Robert ShawandGitHub 2024-08-21 15:53:01 -04:00
• 91f4522cbf [multi-step] Raise error if not using async engine (#7703) William LinandGitHub 2024-08-21 11:49:19 -07:00
• 1b32e02648 [Bugfix] Pass PYTHONPATH from setup.py to CMake (#7730) sasha0552andGitHub 2024-08-21 18:17:48 +00:00
• f7e3b0c5aa [Bugfix][Frontend] Fix Issues Under High Load With zeromq Frontend (#7394) 3 people 2024-08-21 13:34:14 -04:00
• d3c002eadc [Bugfix] chat method add_generation_prompt param (#7734) Brian LiandGitHub 2024-08-22 01:33:35 +08:00
• 9b73a2f498 [Spec Decoding] Use target model max length as default for draft model (#7706) Nick HillandGitHub 2024-08-21 12:23:22 -04:00
• 6925cdbeea [Bugfix][Hardware][CPU] Fix mm_limits initialization for CPU backend (#7735) Isotr0pyandGitHub 2024-08-22 00:23:03 +08:00
• 53328d7536 [BUG] fix crash on flashinfer backend with cudagraph disabled, when attention group_size not in [1,2,4,8] (#7509) LI MOUandGitHub 2024-08-21 23:54:31 +08:00
• c75363fbc0 [BugFix] Avoid premature async generator exit and raise all exception variations (#7698) Nick HillandGitHub 2024-08-21 11:45:55 -04:00
• dd3fa0e430 [Bugfix] Mirror jinja2 in pyproject.toml (#7723) sasha0552andGitHub 2024-08-21 13:41:17 +00:00
• baaedfdb2d [mypy] Enable following imports for entrypoints (#7248) 4 people 2024-08-21 14:28:21 +08:00
• 4506641212 [Doc] Section for Multimodal Language Models (#7719) Roger WangandGitHub 2024-08-20 23:24:01 -07:00
• 12e1c65bc9 [Model] Add AWQ quantization support for InternVL2 model (#7187) Isotr0pyandGitHub 2024-08-21 14:18:57 +08:00
• b74a125800 [ci] try to log process using the port to debug the port usage (#7711) youkaichaoandGitHub 2024-08-20 17:41:12 -07:00
• 66a9e713a7 [Core] Pipe worker_class_fn argument in Executor (#7707) Antoni BaumandGitHub 2024-08-20 17:37:39 -07:00
• 9e51b6a626 [ci][test] adjust max wait time for cpu offloading test (#7709) youkaichaoandGitHub 2024-08-20 17:12:44 -07:00
• 6e4658c7aa [Intel GPU] fix xpu not support punica kernel (which use torch.library.custom_op) (#7685) Kunshang JiandGitHub 2024-08-21 03:01:09 +08:00
• 3b682179dd [Core] Add AttentionState abstraction (#7663) Antoni BaumandGitHub 2024-08-20 11:50:45 -07:00
• c6af027a35 [Misc] Add jinja2 as an explicit build requirement (#7695) Lucas WilkinsonandGitHub 2024-08-20 13:17:47 -04:00
• 2aa00d59ad [CI/Build] Pin OpenTelemetry versions and make errors clearer (#7266) Ronen SchafferandGitHub 2024-08-20 20:02:21 +03:00
• c42590f97a [Hardware] [Intel GPU] refactor xpu worker/executor (#7686) Kunshang JiandGitHub 2024-08-21 00:54:10 +08:00
• aae6927be0 [VLM][Model] Add test for InternViT vision encoder (#7409) Isotr0pyandGitHub 2024-08-20 23:10:20 +08:00
• 398521ad19 [OpenVINO] Updated documentation (#7687) Ilya LavrenovandGitHub 2024-08-20 17:33:56 +04:00