Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• a046f86397 [Core/Bugfix] Add FP8 K/V Scale and dtype conversion for prefix/prefill Triton Kernel (#7208) 3 people 2024-08-12 15:47:41 -07:00
• 4ddc4743d7 [Core] Consolidate GB constant and enable float GB arguments (#7416) Cyrus LeungandGitHub 2024-08-13 05:14:14 +08:00
• 6aa33cb2dd [Misc] Use scalar type to dispatch to different gptq_marlin kernels (#7323) Lucas WilkinsonandGitHub 2024-08-12 14:40:13 -04:00
• 1137f343aa [ci] Cancel fastcheck when PR is ready (#7433) Kevin H. LuuandGitHub 2024-08-12 10:59:14 -07:00
• 9b3e2edd30 [ci] Cancel fastcheck run when PR is marked ready (#7427) Kevin H. LuuandGitHub 2024-08-12 10:56:52 -07:00
• 65950e8f58 [ci] Entrypoints run upon changes in vllm/ (#7423) Kevin H. LuuandGitHub 2024-08-12 10:18:03 -07:00
• cfba4def5d [Bugfix] Fix logit soft cap in flash-attn backend (#7425) Woosuk KwonandGitHub 2024-08-12 09:58:28 -07:00
• d2bc4510a4 [CI/Build] bump Dockerfile.neuron image base, use public ECR (#6832) DanieleandGitHub 2024-08-12 18:53:35 +02:00
• 24154f8618 [Frontend] Disallow passing model as both argument and option (#7347) Cyrus LeungandGitHub 2024-08-12 20:58:34 +08:00
• e6e42e4b17 [Core][VLM] Support image embeddings as input (#6613) Roger WangandGitHub 2024-08-12 01:16:06 -07:00
• ec2affa8ae [Kernel] Flashinfer correctness fix for v0.1.3 (#7319) Lily LiuandGitHub 2024-08-12 00:59:17 -07:00
• 86ab567bae [CI/Build] Minor refactoring for vLLM assets (#7407) Roger WangandGitHub 2024-08-11 19:41:52 -07:00
• f020a6297e [Docs] Update readme (#7316) Simon MoandGitHub 2024-08-11 17:13:37 -07:00
• 6c8e595710 [misc] add commit id in collect env (#7405) youkaichaoandGitHub 2024-08-11 15:40:48 -07:00
• 02b1988b9f [Doc] building vLLM with VLLM_TARGET_DEVICE=empty (#7403) tomeras91andGitHub 2024-08-12 00:38:17 +03:00
• 386087970a [CI/Build] build on empty device for better dev experience (#4773) tomeras91andGitHub 2024-08-11 23:09:44 +03:00
• c08e2b3086 [core] [2/N] refactor worker_base input preparation for multi-step (#7387) William LinandGitHub 2024-08-11 08:50:08 -07:00
• 4fb7b52a2c Updating LM Format Enforcer version to v0.10.6 (#7189) Noam GatandGitHub 2024-08-11 15:11:50 +03:00
• 90bab18f24 [TPU] Use mark_dynamic to reduce compilation time (#7340) Woosuk KwonandGitHub 2024-08-10 18:12:22 -07:00
• 4c5d8e8ea9 [Bugfix] Fix phi3v batch inference when images have different aspect ratio (#7392) Isotr0pyandGitHub 2024-08-11 00:19:33 +08:00
• baa240252e [Core] Fix edge case in chunked prefill + block manager v2 (#7380) Cade DanielandGitHub 2024-08-09 16:48:49 -07:00
• 999ef0b917 [Misc] Add numpy implementation of compute_slot_mapping (#7377) Antoni BaumandGitHub 2024-08-09 15:52:29 -07:00
• 5c6c54d67a [Bugfix] Fix PerTensorScaleParameter weight loading for fused models (#7376) Dipika SikkaandGitHub 2024-08-09 17:23:46 -04:00
• 933790c209 [Core] Add span metrics for model_forward, scheduler and sampler time (#7089) Mahesh KeralapuraandGitHub 2024-08-09 13:55:13 -07:00
• 70d268a399 [Bugfix] Fix ITL recording in serving benchmark (#7372) Roger WangandGitHub 2024-08-09 10:00:00 -07:00
• 249b88228d [Frontend] Support embeddings in the run_batch API (#7132) 3 people 2024-08-09 09:48:21 -07:00
• 74af2bbd90 [Bugfix] Fix reinit procedure in ModelInputForGPUBuilder (#7360) Alexander MatveevandGitHub 2024-08-09 12:35:49 -04:00
• fc7b8d1eef [Performance] e2e overheads reduction: Small followup diff (#7364) Alexander MatveevandGitHub 2024-08-09 11:49:36 -04:00
• 67abdbb42f [VLM][Doc] Add stop_token_ids to InternVL example (#7354) Isotr0pyandGitHub 2024-08-09 22:51:04 +08:00
• 07ab160741 [Model][Jamba] Mamba cache single buffer (#6739) 3 people 2024-08-09 17:07:06 +03:00
• b4e9528f95 [Core] Streamline stream termination in AsyncLLMEngine (#7336) Nick HillandGitHub 2024-08-09 00:06:36 -07:00
• 57b7be0e1c [Speculative decoding] [Multi-Step] decouple should_modify_greedy_probs_inplace (#6971) William LinandGitHub 2024-08-08 22:42:45 -07:00
• 99b4cf5f23 [Bugfix] Fix speculative decoding with MLPSpeculator with padded vocabulary (#7218) Travis JohnsonandGitHub 2024-08-08 23:08:46 -06:00
• e02ac55617 [Performance] Optimize e2e overheads: Reduce python allocations (#7162) Alexander MatveevandGitHub 2024-08-09 00:34:28 -04:00
• 73388c07a4 [TPU] Fix dockerfile.tpu (#7331) Woosuk KwonandGitHub 2024-08-08 20:24:58 -07:00
• 7eb4a51c5f [Core] Support serving encoder/decoder models (#7258) Cyrus LeungandGitHub 2024-08-09 10:39:41 +08:00
• 0fa14907da [TPU] Add Load-time W8A16 quantization for TPU Backend (#7005) Siyuan LiuandGitHub 2024-08-08 18:35:49 -07:00
• 5923532e15 Add Skywork AI as Sponsor (#7314) Simon MoandGitHub 2024-08-08 13:59:57 -07:00
• a049b107e2 [Misc] Temporarily resolve the error of BitAndBytes (#7308) Jee Jee LiandGitHub 2024-08-09 04:42:58 +08:00
• 8334c39f37 [Bugfix] Fix new Llama3.1 GGUF model loading (#7269) Isotr0pyandGitHub 2024-08-09 04:42:44 +08:00
• e904576743 [CI/Build] Dockerfile.cpu improvements (#7298) DanieleandGitHub 2024-08-08 21:24:52 +02:00
• e14fb22e59 [Doc] Put collect_env issue output in a <detail> block (#7310) Michael GoinandGitHub 2024-08-08 14:22:49 -04:00
• 782e53ab59 [Bugfix][fast] Fix the get_num_blocks_touched logic (#6849) Zach ZhengandGitHub 2024-08-08 10:43:30 -07:00
• 21b9c49aa3 [Frontend] Kill the server on engine death (#6594) Joe RundeandGitHub 2024-08-08 10:47:48 -06:00
• 5fb4a3f678 [Bugfix][Kernel] Increased atol to fix failing tests (#7305) Luka GovedičandGitHub 2024-08-08 12:16:13 -04:00
• 757ac70a64 [Model] Rename MiniCPMVQwen2 to MiniCPMV2.6 (#7273) Jee Jee LiandGitHub 2024-08-08 22:02:41 +08:00
• 6dffa4b0a6 [Bugfix] Fix LoRA with PP (#7292) Murali AndoorveeduandGitHub 2024-08-08 00:02:27 -07:00
• 48abee9e54 [Frontend] remove max_num_batched_tokens limit for lora (#7288) Cherilyn BurenandGitHub 2024-08-08 14:17:29 +08:00
• 746709642c [Misc] Fix typos in scheduler.py (#7285) Rui QiaoandGitHub 2024-08-07 17:06:01 -07:00
• e53dfd3eaf [Kernel] Fix Flashinfer Correctness (#7284) Lily LiuandGitHub 2024-08-07 16:26:52 -07:00
• 6d94420246 [Doc] Update supported_hardware.rst (#7276) Michael GoinandGitHub 2024-08-07 17:21:50 -04:00
• fc1493a01e [FrontEnd] Make merge_async_iterators is_cancelled arg optional (#7282) Nick HillandGitHub 2024-08-07 13:35:14 -07:00
• 311f743831 [Bugfix] Fix gptq failure on T4s (#7264) Lucas WilkinsonandGitHub 2024-08-07 16:05:37 -04:00
• 469b3bc538 [ci] Make building wheels per commit optional (#7278) Kevin H. LuuandGitHub 2024-08-07 11:34:25 -07:00
• 5223199e03 [Bugfix][FP8] Fix dynamic FP8 Marlin quantization (#7219) Michael GoinandGitHub 2024-08-07 14:23:12 -04:00
• fde47d3bc2 [BugFix] Fix frontend multiprocessing hang (#7217) 3 people 2024-08-07 15:09:36 -03:00
• 0e12cd67a8 [Doc] add online speculative decoding example (#7243) Stas BekmanandGitHub 2024-08-07 09:58:02 -07:00
• 80cbe10c59 [OpenVINO] migrate to latest dependencies versions (#7251) Ilya LavrenovandGitHub 2024-08-07 20:49:10 +04:00
• b764547616 [Bugfix] Fix input processor for InternVL2 model (#7164) 3 people 2024-08-08 00:32:07 +08:00
• ab0f5e2823 Fixes typo in function name (#7275) Rafael VasquezandGitHub 2024-08-07 12:29:27 -04:00
• 564985729a [ BugFix ] Move zmq frontend to IPC instead of TCP (#7222) Robert ShawandGitHub 2024-08-07 12:24:56 -04:00
• 0f7052bc7e [Misc] Refactor linear layer weight loading; introduce BasevLLMParameter and weight_loader_v2 (#5874) Dipika SikkaandGitHub 2024-08-07 12:17:58 -04:00
• 639159b2a6 [distributed][misc] add specialized method for cuda platform (#7249) youkaichaoandGitHub 2024-08-07 08:54:52 -07:00
• 66d617e343 [Frontend] Gracefully handle missing chat template and fix CI failure (#7238) 3 people 2024-08-07 17:12:05 +08:00
• 7b261092de [BUGFIX]: top_k is expected to be an integer. (#7227) Atilla AkkuşandGitHub 2024-08-07 10:32:16 +03:00
• 2385c8f374 [Doc] Mock new dependencies for documentation (#7245) Roger WangandGitHub 2024-08-06 23:43:03 -07:00
• 9a3f49ae07 [BugFix] Overhaul async request cancellation (#7111) Nick HillandGitHub 2024-08-06 22:21:41 -07:00
• f9a5600649 [Bugfix] Fix GPTQ and GPTQ Marlin CPU Offloading (#7225) Michael GoinandGitHub 2024-08-06 21:34:26 -04:00
• fd95e026e0 [Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942) 4 people 2024-08-06 16:51:47 -04:00
• 660470e5a3 [Core] Optimize evictor-v2 performance (#7193) xiaobochen123andGitHub 2024-08-07 03:34:25 +08:00
• 8d59dbb000 [Kernel] Add per-tensor and per-token AZP epilogues (#5941) 3 people 2024-08-06 14:17:08 -04:00
• 5c60c8c423 [SpecDecode] [Minor] Fix spec decode sampler tests (#7183) Lily LiuandGitHub 2024-08-06 10:40:32 -07:00
• 00afc78590 [Bugfix] add gguf dependency (#7198) 3 people 2024-08-06 19:08:35 +02:00
• 541c1852d3 [ BugFix ] Fix ZMQ when VLLM_PORT is set (#7205) Robert ShawandGitHub 2024-08-06 12:26:26 -04:00
• a3bbbfa1d8 [BugFix] Fix DeepSeek remote code (#7178) Dipika SikkaandGitHub 2024-08-06 11:16:53 -04:00
• 1f26efbb3a [Model] Support SigLIP encoder and alternative decoders for LLaVA models (#7153) 3 people 2024-08-06 16:55:31 +08:00
• 9118217f58 [LoRA] Relax LoRA condition (#7146) Jee Jee LiandGitHub 2024-08-06 09:57:25 +08:00
• e3c664bfcb [Build] Add initial conditional testing spec (#6841) Simon MoandGitHub 2024-08-05 17:39:22 -07:00
• 360bd67cf0 [Core] Support loading GGUF model (#5191) 3 people 2024-08-06 07:54:23 +08:00
• ef527be06c [MISC] Use non-blocking transfer in prepare_input (#7172) Cody YuandGitHub 2024-08-05 16:41:27 -07:00
• 89b8db6bb2 [Bugfix] Specify device when loading LoRA and embedding tensors (#7129) 3 people 2024-08-05 16:35:47 -07:00
• 789937af2e [Doc] [SpecDecode] Update MLPSpeculator documentation (#7100) Thomas ParnellandGitHub 2024-08-06 01:29:43 +02:00
• dfb1a15dcb [ci][frontend] deduplicate tests (#7101) youkaichaoandGitHub 2024-08-05 15:59:22 -07:00
• 4db5176d97 bump version to v0.5.4 (#7139) Simon MoandGitHub 2024-08-05 14:39:48 -07:00
• 4cf1dc39be [Bugfix][CI/Build] Fix CUTLASS FetchContent (#7171) Tyler Michael SmithandGitHub 2024-08-05 17:22:57 -04:00
• 6e4852ce28 [CI/Build] Suppress divide-by-zero and missing return statement warnings (#7001) Tyler Michael SmithandGitHub 2024-08-05 16:00:01 -04:00
• 8571ac4672 [Kernel] Update CUTLASS to 3.5.1 (#7085) Tyler Michael SmithandGitHub 2024-08-05 15:13:43 -04:00
• 997cf78308 [Misc] Fix typo in GroupCoordinator.recv() (#7167) Rui QiaoandGitHub 2024-08-05 11:10:16 -07:00
• 57f560aa23 [BugFix] Use args.trust_remote_code (#7121) Aditya PaliwalandGitHub 2024-08-05 09:26:14 -07:00
• 003f8ee128 [BugFix] Use IP4 localhost form for zmq bind (#7163) Nick HillandGitHub 2024-08-05 08:41:03 -07:00
• e9630458c7 [SpecDecode] Support FlashInfer in DraftModelRunner (#6926) Bongwon JangandGitHub 2024-08-06 00:05:05 +09:00
• 82a1b1a82b [Speculative decoding] Add periodic log with time spent in proposal/scoring/verification (#6963) Cade DanielandGitHub 2024-08-05 01:46:44 -07:00
• c0d8f1636c [Model] SiglipVisionModel ported from transformers (#6942) 3 people 2024-08-05 15:22:12 +09:00
• cc08fc7225 [Frontend] Reapply "Factor out code for running uvicorn" (#7095) Cyrus LeungandGitHub 2024-08-05 11:40:51 +08:00
• 7b86e7c9cd [Model] Add multi-image support for minicpmv (#7122) 4 people 2024-08-05 09:23:17 +08:00
• f80ab3521c Clean up remaining Punica C information (#7027) Jee Jee LiandGitHub 2024-08-05 06:37:08 +08:00
• 16a1cc9bb2 [misc][distributed] improve libcudart.so finding (#7127) youkaichaoandGitHub 2024-08-04 11:31:51 -07:00
• b1c9aa3daa [Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when using MLPSpeculator (#7105) Thomas ParnellandGitHub 2024-08-04 16:13:18 +02:00
• 179a6a36f2 [Model]Refactor MiniCPMV (#7020) 3 people 2024-08-04 16:12:41 +08:00
• 83c644fe7e [core][misc] simply output processing with shortcut code path (#7117) youkaichaoandGitHub 2024-08-04 00:22:19 -07:00