Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• c7cb5c3335 [Misc] GPTQ Activation Ordering (#8135) Kyle Sayers 2024-09-09 16:27:26 -04:00
• f9b4a2d415 [Bugfix] Correct adapter usage for cohere and jamba (#8292) Vladislav Kruglikov 2024-09-09 21:20:46 +03:00
• 58fcc8545a [Frontend] Add progress reporting to run_batch.py (#8060) Adam LugowskiandAdam Lugowski 2024-09-09 11:16:37 -07:00
• 08287ef675 [Bugfix] Streamed tool calls now more strictly follow OpenAI's format; ensures Vercel AI SDK compatibility (#8272) Kyle Mistele 2024-09-09 09:45:11 -05:00
• 4ef41b8476 [Bugfix] Fix async postprocessor in case of preemption (#8267) Alexander Matveev 2024-09-08 00:01:51 -04:00
• cfe712bf1a [CI/Build] Use python 3.12 in cuda image (#8133) Joe Runde 2024-09-07 14:03:16 -06:00
• b962ee1470 ppc64le: Dockerfile fixed, and a script for buildkite (#8026) sumitd2 2024-09-07 23:48:40 +05:30
• 36bf8150cc [Model][VLM] Decouple weight loading logic for Paligemma (#8269) Isotr0py 2024-09-08 01:45:44 +08:00
• e807125936 [Model][VLM] Support multi-images inputs for InternVL2 models (#8201) Isotr0py 2024-09-07 16:38:23 +08:00
• 9f68e00d27 [Bugfix] Fix broken OpenAI tensorizer test (#8258) Cyrus Leung 2024-09-07 16:02:39 +08:00
• ce2702a923 [tpu][misc] fix typo (#8260) youkaichao 2024-09-06 22:40:46 -07:00
• 795b662cff Enable Random Prefix Caching in Serving Profiling Tool (benchmark_serving.py) (#8241) Wei-Sheng Chin 2024-09-06 20:18:16 -07:00
• 2f707fcb35 [Model] Multi-input support for LLaVA (#8238) Cyrus Leung 2024-09-07 10:57:24 +08:00
• 41e95c5247 [Bugfix] Fix Hermes tool call chat template bug (#8256) Kyle MisteleandKyle Mistele 2024-09-06 21:49:01 -05:00
• 12dd715807 [misc] [doc] [frontend] LLM torch profiler support (#7943) William Lin 2024-09-06 17:48:48 -07:00
• 29f49cd6e3 [Model] Allow loading from original Mistral format (#8168) Patrick von PlatenandMichael Goin 2024-09-07 01:02:05 +02:00
• 23f322297f [Misc] Remove SqueezeLLM (#8220) Dipika Sikka 2024-09-06 18:29:03 -04:00
• 9db52eab3d [Kernel] [Triton] Memory optimization for awq_gemm and awq_dequantize, 2x throughput (#8248) rasmith 2024-09-06 17:26:09 -05:00
• 1447c97e75 [CI/Build] Increasing timeout for multiproc worker tests (#8203) Alexey Kondratiev(AMD) 2024-09-06 14:51:03 -04:00
• de80783b69 [Misc] Use ray[adag] dependency instead of cuda (#7938) Rui Qiao 2024-09-06 09:18:35 -07:00
• e5cab71531 [Frontend] Add --logprobs argument to benchmark_serving.py (#8191) afeldman-nm 2024-09-06 12:01:14 -04:00
• baa5467547 [BugFix] Fix Granite model configuration (#8216) Nick Hill 2024-09-05 20:39:29 -07:00
• db3bf7c991 [Core] Support load and unload LoRA in api server (#6566) Jiaxin ShanandJee Jee Li 2024-09-05 18:10:33 -07:00
• 2febcf2777 [Documentation][Spec Decode] Add documentation about lossless guarantees in Speculative Decoding in vLLM (#7962) sroy745 2024-09-05 13:25:29 -07:00
• 2ee45281a5 Move verify_marlin_supported to GPTQMarlinLinearMethod (#8165) Michael Goin 2024-09-05 11:09:46 -04:00
• 9da25a88aa [MODEL] Qwen Multimodal Support (Qwen-VL / Qwen-VL-Chat) (#8029) Alex BrooksandDarkLight1337 2024-09-05 06:48:10 -06:00
• 8685ba1a1e Inclusion of InternVLChatModel In PP_SUPPORTED_MODELS(Pipeline Parallelism) (#7860) manikandan.tm@zucisystems.com 2024-09-05 17:03:37 +05:30
• 288a938872 [Doc] Indicate more information about supported modalities (#8181) Cyrus Leung 2024-09-05 18:51:53 +08:00
• e39ebf5cf5 [Core/Bugfix] Add query dtype as per FlashInfer API requirements. (#8173) Elfie Guo 2024-09-04 22:12:26 -07:00
• ba262c4e5a [ci] Mark LoRA test as soft-fail (#8160) Kevin H. Luu 2024-09-04 20:33:12 -07:00
• 4624d98dbd [Misc] Clean up RoPE forward_native (#8076) Woosuk Kwon 2024-09-04 20:31:48 -07:00
• 1afc931987 [bugfix] >1.43 constraint for openai (#8169) William LinandMichael Goin 2024-09-04 17:35:36 -07:00
• e01c2beb7d [Doc] [Misc] Create CODE_OF_CONDUCT.md (#8161) Maureen McElaney 2024-09-04 19:50:13 -04:00
• 32e7db2536 Bump version to v0.6.0 (#8166) Simon Mo 2024-09-04 16:34:27 -07:00
• 008cf886c9 [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) Harsha vardhan manoj BikkiandHarsha Bikki 2024-09-04 16:33:43 -07:00
• 77d9e514a2 [MISC] Replace input token throughput with total token throughput (#8164) Cody YuandMichael Goin 2024-09-04 13:23:22 -07:00
• e02ce498be [Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models (#5649) 3 people 2024-09-04 15:18:13 -05:00
• 561d6f8077 [CI] Change test input in Gemma LoRA test (#8163) Woosuk Kwon 2024-09-04 13:05:50 -07:00
• d1dec64243 [CI/Build][ROCm] Enabling LoRA tests on ROCm (#7369) alexeykondratandSimon Mo 2024-09-04 14:57:54 -04:00
• 2ad2e5608e [MISC] Consolidate FP8 kv-cache tests (#8131) Cody Yu 2024-09-04 11:53:25 -07:00
• d3311562fb [Bugfix] remove post_layernorm in siglip (#8106) wnma 2024-09-04 18:55:37 +08:00
• ccd7207191 chore: Update check-wheel-size.py to read MAX_SIZE_MB from env (#8103) TimWang 2024-09-04 14:17:05 +08:00
• 855c262a6b [Frontend] Multimodal support in offline chat (#8098) Cyrus Leung 2024-09-04 13:22:17 +08:00
• 2be8ec6e71 [Model] Add Ultravox support for multiple audio chunks (#7963) Peter Salas 2024-09-03 21:38:21 -07:00
• e16fa99a6a [Misc] Update fbgemmfp8 to use vLLMParameters (#7972) Dipika SikkaandMichael Goin 2024-09-03 22:12:41 -04:00
• 61f4a93d14 [TPU][Bugfix] Use XLA rank for persistent cache path (#8137) Woosuk Kwon 2024-09-03 18:35:33 -07:00
• d4db9f53c8 [Benchmark] Add --async-engine option to benchmark_throughput.py (#7964) Nick Hill 2024-09-03 17:57:41 -07:00
• 2188a60c7e [Misc] Update GPTQ to use vLLMParameters (#7976) Dipika Sikka 2024-09-03 17:21:44 -04:00
• dc0b6066ab [CI] Change PR remainder to avoid at-mentions (#8134) Simon Mo 2024-09-03 14:11:42 -07:00
• 0af3abe3d3 [TPU][Bugfix] Fix next_token_ids shape (#8128) Woosuk Kwon 2024-09-03 13:29:24 -07:00
• f1575dc99f [ci] Fix GHA workflow (#8129) Kevin H. Luu 2024-09-03 13:25:09 -07:00
• c02638efb3 [CI/Build] make pip install vllm work in macos (for import only) (#8118) tomeras91 2024-09-03 22:37:08 +03:00
• 652c83b697 [Misc] Raise a more informative exception in add/remove_logger (#7750) Antoni Baum 2024-09-03 12:28:25 -07:00
• 6d646d08a2 [Core] Optimize Async + Multi-step (#8050) Alexander Matveev 2024-09-03 14:50:29 -04:00
• 95a178f861 [CI] Only PR reviewers/committers can trigger CI on PR (#8124) Kevin H. Luu 2024-09-03 11:32:27 -07:00
• bd852f2a8b [Performance] Enable chunked prefill and prefix caching together (#8120) 3 people 2024-09-03 10:49:18 -07:00
• ec266536b7 [Bugfix][VLM] Add fallback to SDPA for ViT model running on CPU backend (#8061) Isotr0py 2024-09-03 21:37:52 +08:00
• 0fbc6696c2 [Bugfix] Fix single output condition in output processor (#7881) Woosuk Kwon 2024-09-02 20:35:42 -07:00
• 6e36f4fa6c improve chunked prefill performance wang.yuqi 2024-09-03 05:20:12 +08:00
• dd2a6a82e3 [Bugfix] Fix internlm2 tensor parallel inference (#8055) Isotr0py 2024-09-02 23:48:56 +08:00
• 4ca65a9763 [Core][Bugfix] Accept GGUF model without .gguf extension (#8056) Isotr0py 2024-09-02 20:43:26 +08:00
• e2b2aa5a0f [TPU] Align worker index with node boundary (#7932) Woosuk Kwon 2024-09-01 23:09:46 -07:00
• e6a26ed037 [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) Lily Liu 2024-09-01 21:23:29 -07:00
• f8d60145b4 [Model] Add Granite model (#7436) Shawn TanandNick Hill 2024-09-01 21:37:18 -04:00
• 5b86b19954 [Misc] Optional installation of audio related packages (#8063) Roger Wang 2024-09-01 14:46:57 -07:00
• 5231f0898e [Frontend][VLM] Add support for multiple multi-modal items (#8049) Roger Wang 2024-08-31 16:35:53 -07:00
• 8423aef4c8 [BugFix][Core] Multistep Fix Crash on Request Cancellation (#8059) Robert Shaw 2024-08-31 15:44:03 -04:00
• 4f5d8446ed [Bugfix] Fix ModelScope models in v0.5.5 (#8037) Nicolò Lucchesi 2024-08-31 09:27:58 +02:00
• d05f0a9db2 [Bugfix] Fix import error in Phi-3.5-MoE (#8052) Cyrus Leung 2024-08-31 13:26:55 +08:00
• 622f8abff8 [Bugfix] bugfix and add model test for flashinfer fp8 kv cache. (#8013) Pavani Majety 2024-08-30 22:18:50 -07:00
• 1248e8506a [Model] Adding support for MSFT Phi-3.5-MoE (#7729) 4 people 2024-08-31 03:42:57 +08:00
• 2684efc467 [TPU][Bugfix] Fix tpu type api (#8035) Woosuk Kwon 2024-08-30 09:01:26 -07:00
• 058344f89a [Frontend]-config-cli-args (#7737) 3 people 2024-08-30 08:21:02 -07:00
• 98cef6a227 [Core] Increase default max_num_batched_tokens for multimodal models (#8028) Cyrus Leung 2024-08-30 23:20:34 +08:00
• f97be32d1d [VLM][Model] TP support for ViTs (#7186) 3 people 2024-08-31 00:19:27 +09:00
• afd39a4511 [Bugfix] Fix import error in Exaone model (#8034) Cyrus Leung 2024-08-30 23:03:28 +08:00
• 2148441fd3 [TPU] Support single and multi-host TPUs on GKE (#7613) Richard Liu 2024-08-30 00:27:40 -07:00
• dc13e99348 [MODEL] add Exaone model support (#7819) Yohan Na 2024-08-30 15:34:20 +09:00
• 34a0e96d46 [Kernel] changing fused moe kernel chunk size default to 32k (#7995) Avshalom Manevich 2024-08-30 11:11:39 +07:00
• 80c7b089b1 [TPU] Async output processing for TPU (#8011) Woosuk Kwon 2024-08-29 19:35:29 -07:00
• 428dd1445e [Core] Logprobs support in Multi-step (#7652) afeldman-nm 2024-08-29 22:19:08 -04:00
• 4abed65c58 [VLM] Disallow overflowing max_model_len for multimodal models (#7998) Cyrus Leung 2024-08-30 08:49:04 +08:00
• 0c785d344d Add more percentiles and latencies (#7759) Wei-Sheng Chin 2024-08-29 16:48:11 -07:00
• 4664ceaad6 support bitsandbytes 8-bit and FP4 quantized models (#7445) chenqianfzh 2024-08-29 16:09:08 -07:00
• 257afc37c5 [Neuron] Adding support for context-lenght, token-gen buckets. (#7885) Harsha vardhan manoj BikkiandHarsha Bikki 2024-08-29 13:58:14 -07:00
• 86a677de42 [misc] update tpu int8 to use new vLLM Parameters (#7973) Dipika Sikka 2024-08-29 16:46:55 -04:00
• d78789ac16 [Bugfix] Fix incorrect vocal embedding shards for GGUF model in tensor parallelism (#7954) Isotr0py 2024-08-30 03:54:49 +08:00
• c334b1898b extend cuda graph size for H200 (#7894) kushanamandyoukaichao 2024-08-29 12:15:04 -07:00
• 6b3421567d [Core][Kernels] Enable FP8 KV Cache with Flashinfer backend. + BugFix for kv_cache_dtype=auto (#7985) 3 people 2024-08-29 11:53:11 -07:00
• 3f60f2244e [Core] Combine async postprocessor and multi-step (#7921) Alexander Matveev 2024-08-29 14:18:26 -04:00
• f205c09854 [Bugfix] Unify rank computation across regular decoding and speculative decoding (#7899) Jonas M. Kübler 2024-08-29 07:18:13 +02:00
• ef99a78760 Revert "[Core][Kernels] Use FlashInfer backend for FP8 KV Cache when available." (#7982) youkaichao 2024-08-28 21:27:06 -07:00
• 74d5543ec5 [VLM][Core] Fix exceptions on ragged NestedTensors (#7974) Peter Salas 2024-08-28 20:24:31 -07:00
• a7f65c2be9 [torch.compile] remove reset (#7975) youkaichao 2024-08-28 17:32:26 -07:00
• 4289cad37f [Frontend] Minor optimizations to zmq decoupled front-end (#7957) Nick HillandRobert Shaw 2024-08-28 17:22:43 -07:00
• af59df0a10 Remove faulty Meta-Llama-3-8B-Instruct-FP8.yaml lm-eval test (#7961) Michael Goin 2024-08-28 19:19:17 -04:00
• ce6bf3a2cf [torch.compile] avoid Dynamo guard evaluation overhead (#7898) youkaichaoandWoosuk Kwon 2024-08-28 16:10:12 -07:00
• 3cdfe1f38b [Bugfix] Make torch registration of punica ops optional (#7970) bnellnm 2024-08-28 18:11:49 -04:00
• fdd9daafa3 [Kernel/Model] Migrate mamba_ssm and causal_conv1d kernels to vLLM (#7651) Mor Zusman 2024-08-29 01:06:52 +03:00
• 8c56e57def [Doc] fix 404 link (#7966) Stas Bekman 2024-08-28 13:54:23 -07:00