Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• 30efe41532 [Docs] Update docs for wheel location (#6580) Simon Mo 2024-07-19 12:14:11 -07:00
• 9ed82e7074 [Misc] Small perf improvements (#6520) Antoni Baum 2024-07-19 12:10:56 -07:00
• 51f8aa90ad [Bugfix][Frontend] remove duplicate init logger (#6581) Daniele 2024-07-19 19:16:27 +02:00
• a5314e8698 [Model] RowParallelLinear: pass bias to quant_method.apply (#6327) Thomas Parnell 2024-07-19 15:15:22 +02:00
• a921e86392 [BUGFIX] Raise an error for no draft token case when draft_tp>1 (#6369) Woo-Yeon Lee 2024-07-19 22:01:09 +09:00
• 6366efc67b [Bugfix][Frontend] Fix missing /metrics endpoint (#6463) Cyrus Leung 2024-07-19 11:55:13 +08:00
• dbe5588554 [ Misc ] non-uniform quantization via compressed-tensors for Llama (#6515) Robert Shaw 2024-07-18 22:39:18 -04:00
• d4201e06d5 [Bugfix] Make spec. decode respect per-request seed. (#6034) Thomas ParnellandNick Hill 2024-07-19 04:22:08 +02:00
• b5672a112c [Core] Multiprocessing Pipeline Parallel support (#6130) Nick HillandMurali Andoorveedu 2024-07-18 19:15:52 -07:00
• c5df56f88b Add support for a rope extension method (#6553) Simon Mo 2024-07-18 18:53:03 -07:00
• 1689219ebf [CI/Build] Build on Ubuntu 20.04 instead of 22.04 (#6517) Tyler Michael Smith 2024-07-18 20:29:25 -04:00
• 4ffffccb7e [Kernel] Implement fallback for FP8 channelwise using torch._scaled_mm (#6552) Tyler Michael Smith 2024-07-18 19:52:22 -04:00
• f53b8f0d05 [ci][test] add correctness test for cpu offloading (#6549) youkaichao 2024-07-18 16:41:06 -07:00
• 2d4733ba2d Fix PR comment bot (#6554) Kevin H. Luu 2024-07-18 14:48:29 -07:00
• 15c6a079b1 [Model] Support Mistral-Nemo (#6548) Michael Goin 2024-07-18 16:31:50 -04:00
• ecdb462c24 [ci] Reword Github bot comment (#6534) Kevin H. Luu 2024-07-18 08:01:45 -07:00
• 58ca663224 [ Misc ] Improve Min Capability Checking in compressed-tensors (#6522) Robert Shaw 2024-07-18 10:39:12 -04:00
• 4634c8728b [TPU] Refactor TPU worker & model runner (#6506) Woosuk Kwon 2024-07-18 01:34:16 -07:00
• c8a7d51c49 [Bugfix] Update flashinfer.py with PagedAttention forwards - Fixes Gemma2 OpenAI Server Crash (#6501) Noam Gat 2024-07-18 10:47:13 +03:00
• e2fbaee725 [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (#6227) Nick HillandCyrus Leung 2024-07-18 00:13:30 -07:00
• 8a74c68bd1 [Misc] Minor patch for draft model runner (#6523) Cody Yu 2024-07-17 23:06:21 -07:00
• 61e592747c [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) Rui QiaoandStephanie Wang 2024-07-17 22:27:09 -07:00
• d25877dd9b [BugFix] Avoid secondary error in ShmRingBuffer destructor (#6530) Nick Hill 2024-07-17 22:24:43 -07:00
• 1c27d25fb5 [core][model] yet another cpu offload implementation (#6496) youkaichaoandMichael Goin 2024-07-17 20:54:35 -07:00
• 18fecc3559 [ Kernel ] Fp8 Channelwise Weight Support (#6487) Robert Shaw 2024-07-17 23:18:13 -04:00
• b5af8c223c [Model] Pipeline parallel support for Mixtral (#6516) Cody Yu 2024-07-17 19:26:04 -07:00
• b5241e41d9 [ Kernel ] FP8 Dynamic-Per-Token Quant Kernel (#6511) Varun Sundar RabindranathandVarun Sundar Rabindranath 2024-07-17 21:38:35 -04:00
• e76466dde2 [Core] draft_model_runner: Implement prepare_inputs on GPU for advance_step (#6338) Alexander Matveev 2024-07-17 17:30:28 -04:00
• 5f0b9933e6 [Bugfix] Fix Ray Metrics API usage (#6354) Antoni Baum 2024-07-17 12:40:10 -07:00
• a38524f338 [DOC] - Add docker image to Cerebrium Integration (#6510) milo157 2024-07-17 13:22:53 -04:00
• 2fa4623d9e [Core] Refactor _prepare_model_input_tensors - take 2 (#6164) Cody Yu 2024-07-17 09:37:16 -07:00
• a9a2e74d21 [Misc] Use torch.Tensor for type annotation (#6505) Woosuk Kwon 2024-07-17 06:01:10 -07:00
• e09ce759aa [TPU] Remove multi-modal args in TPU backend (#6504) Woosuk Kwon 2024-07-17 04:02:53 -07:00
• 5fa6e9876e [Bugfix] Fix for multinode crash on 4 PP (#6495) Murali Andoorveedu 2024-07-17 04:25:10 -04:00
• 5bf35a91e4 [Doc][CI/Build] Update docs and tests to use vllm serve (#6431) Cyrus Leung 2024-07-17 15:43:21 +08:00
• a19e8d3726 [Misc][Speculative decoding] Typos and typing fixes (#6467) shangmingcandcaishangming.csm 2024-07-17 15:17:07 +08:00
• 10383887e0 [ROCm] Cleanup Dockerfile and remove outdated patch (#6482) Hongxia Yang 2024-07-17 01:47:02 -04:00
• 1d094fd7c0 [Distributed][PP] only create embedding & lm head when necessary (#6455) Wushi Dong 2024-07-16 19:20:26 -07:00
• ce37be7ba0 [misc][distributed] add seed to dummy weights (#6491) youkaichao 2024-07-16 19:16:34 -07:00
• 7f62077af5 [misc][distributed] improve tests (#6488) youkaichao 2024-07-16 17:35:52 -07:00
• 09c2eb85dd [ci][distributed] add pipeline parallel correctness test (#6410) youkaichao 2024-07-16 15:44:22 -07:00
• 978aed5300 [Kernel][Attention] Separate Attention.kv_scale into k_scale and v_scale (#6081) Michael Goin 2024-07-16 18:31:32 -04:00
• 160e1d8c99 [Misc] Log spec decode metrics (#6454) Cody Yu 2024-07-16 13:37:10 -07:00
• 94162beb9f [Doc] Fix the lora adapter path in server startup script (#6230) Jiaxin Shan 2024-07-16 10:11:04 -07:00
• c467dff24f [Hardware][TPU] Support MoE with Pallas GMM kernel (#6457) Woosuk Kwon 2024-07-16 09:56:28 -07:00
• 9f4ccec761 [doc][misc] remind to cancel debugging environment variables (#6481) youkaichao 2024-07-16 09:45:30 -07:00
• 38ef94888a [CI/Build] Remove "boardwalk" image asset (#6460) Cyrus Leung 2024-07-16 23:59:36 +08:00
• 2bb0489cb3 [Core] Use numpy to speed up padded token processing (#6442) Peng Guanwen 2024-07-16 23:13:25 +08:00
• 7508a3dc34 [Misc] Fix typos in spec. decode metrics logging. (#6470) Thomas Parnell 2024-07-16 15:55:15 +02:00
• 7a3d2a5b95 [Frontend] Support for chat completions input in the tokenize endpoint (#5923) sasha0552 2024-07-16 12:18:09 +00:00
• d97011512e [CI/Build] vLLM cache directory for images (#6444) Cyrus Leung 2024-07-16 14:12:25 +08:00
• 37d776606f [Docs] Announce 5th meetup (#6458) Woosuk Kwon 2024-07-15 21:04:58 -07:00
• d92b3c5cde [Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (#6419) Joe 2024-07-15 18:54:15 -07:00
• 9ad32dacd9 [BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (#6425) Mor ZusmanandMor Zusman 2024-07-16 04:32:55 +03:00
• d6f3b3d5c4 Pin sphinx-argparse version (#6453) Kevin H. Luu 2024-07-15 18:26:11 -07:00
• 4552e37b55 [CI/Build][TPU] Add TPU CI test (#6277) Woosuk Kwonandkevin 2024-07-15 14:31:16 -07:00
• ec9933f4a5 [Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (#6289) Woosuk Kwon 2024-07-15 12:02:14 -07:00
• 3dee97b05f [Docs] Add Google Cloud to sponsor list (#6450) Woosuk Kwon 2024-07-15 11:58:10 -07:00
• 4cf256ae7f [misc][distributed] fix pp missing layer condition (#6446) youkaichao 2024-07-15 10:32:35 -07:00
• 64fdc08c72 bump version to v0.5.2 (#6433) Simon Mo 2024-07-15 10:27:40 -07:00
• 4ef95b0f06 [Bugfix] use float32 precision in samplers/test_logprobs.py for comparing with HF (#6409) Thomas Parnell 2024-07-15 19:14:49 +02:00
• eaec4b9153 [Bugfix] Add custom Triton cache manager to resolve MoE MP issue (#6140) Thomas ParnellandChih-Chieh-Yang 2024-07-15 19:12:47 +02:00
• a63a4c6341 [Misc] Use 0.0.9 version for flashinfer (#6447) Pernekhan UtemuratovandPernekhan Utemuratov 2024-07-15 10:10:26 -07:00
• c8fd97f26d [Kernel] Use CUTLASS kernels for the FP8 layers with Bias (#6270) Tyler Michael Smith 2024-07-15 13:05:52 -04:00
• 94b82e8c18 [doc][distributed] add suggestion for distributed inference (#6418) youkaichao 2024-07-15 09:45:51 -07:00
• 6ae1597ddf [VLM] Minor space optimization for ClipVisionModel (#6436) Roger Wang 2024-07-15 02:29:51 -07:00
• 22e79ee8f3 [doc][misc] doc update (#6439) youkaichao 2024-07-14 23:33:25 -07:00
• de19916314 [Bugfix] Convert image to RGB by default (#6430) Cyrus Leung 2024-07-15 13:39:15 +08:00
• 69672f116c [core][distributed] simplify code to support pipeline parallel (#6406) youkaichao 2024-07-14 21:20:51 -07:00
• 44874a0bf9 [Doc] add env docs for flashinfer backend (#6437) DefTruth 2024-07-15 12:16:51 +08:00
• b47008b4d2 [BugFix] BatchResponseData body should be optional (#6345) zifeitongandCyrus Leung 2024-07-14 21:06:09 -07:00
• 9bfece89fd Add FUNDING.yml (#6435) Simon Mo 2024-07-14 20:36:16 -07:00
• 32c9d7f765 Report usage for beam search (#6404) Simon Mo 2024-07-14 19:37:35 -07:00
• ccb20db8bd [Bugfix] Benchmark serving script used global parameter 'args' in function 'sample_random_requests' (#6428) Fish 2024-07-15 10:27:01 +08:00
• a754dc2cb9 [CI/Build] Cross python wheel (#6394) Robert Shaw 2024-07-14 21:54:46 -04:00
• 61e85dbad8 [Doc] xpu backend requires running setvars.sh (#6393) Robert Cohn 2024-07-14 20:10:11 -04:00
• dbfe254eda [Feature] vLLM CLI (#5090) Ethan Xuandsimon-mo 2024-07-14 15:36:43 -07:00
• 73030b7dae [ Misc ] Enable Quantizing All Layers of DeekSeekv2 (#6423) Robert Shaw 2024-07-14 17:38:42 -04:00
• ccd3c04571 [ci][build] fix commit id (#6420) youkaichaoandCyrus Leung 2024-07-14 07:16:21 -07:00
• 9dad5cc859 [Kernel] Turn off CUTLASS scaled_mm for Ada Lovelace (#6384) Tyler Michael Smith 2024-07-14 09:37:19 -04:00
• 6ef3bf912c Remove unnecessary trailing period in spec_decode.rst (#6405) Yuan Tang 2024-07-14 02:58:09 -05:00
• 540c0368b1 [Model] Initialize Fuyu-8B support (#3924) Isotr0pyandRoger Wang 2024-07-14 13:27:14 +08:00
• fb6af8bc08 [ Misc ] Apply MoE Refactor to Deepseekv2 To Support Fp8 (#6417) Robert Shaw 2024-07-13 23:03:58 -04:00
• eeceadaecc [Misc] Add deprecation warning for beam search (#6402) Woosuk Kwon 2024-07-13 11:52:22 -07:00
• babf52dade [ Misc ] More Cleanup of Marlin (#6359) Robert ShawandRobert Shaw 2024-07-13 06:21:37 -04:00
• 9da4aad44b Updating LM Format Enforcer version to v10.3 (#6411) Noam Gat 2024-07-13 13:09:12 +03:00
• 41708e5034 [ci] try to add multi-node tests (#6280) youkaichaoandMuralidhar Andoorveedu 2024-07-12 21:51:48 -07:00
• d80aef3776 [Docs] Clean up latest news (#6401) Woosuk Kwon 2024-07-12 19:36:53 -07:00
• e1684a766a [Bugfix] Fix hard-coded value of x in context_attention_fwd (#6373) Thomas Parnell 2024-07-13 03:30:54 +02:00
• a27f87da34 [Doc] Fix Typo in Doc (#6392) Saliya EkanayakeandSaliya Ekanayake 2024-07-12 17:48:23 -07:00
• 16ff6bd58c [ci] Fix wording for GH bot (#6398) Kevin H. Luu 2024-07-12 16:34:37 -07:00
• f8f9ff57ee [Bugfix][TPU] Fix megacore setting for v5e-litepod (#6397) Woosuk Kwon 2024-07-12 15:59:47 -07:00
• 6bc9710f6e Fix release pipeline's dir permission (#6391) Simon Mo 2024-07-12 15:52:43 -07:00
• 111fc6e7ec [Misc] Add generated git commit hash as vllm.__commit__ (#6386) Michael Goin 2024-07-12 18:52:15 -04:00
• 75f64d8b94 [Bugfix] Fix illegal memory access in FP8 MoE kernel (#6382) Cody Yu 2024-07-12 14:33:33 -07:00
• 21b2dcedab Fix release pipeline's -e flag (#6390) Simon Mo 2024-07-12 14:08:04 -07:00
• 07b35af86d Fix interpolation in release pipeline (#6389) Simon Mo 2024-07-12 14:03:39 -07:00
• bb1a784b05 Fix release-pipeline.yaml (#6388) Simon Mo 2024-07-12 14:00:57 -07:00
• d719ba24c5 Build some nightly wheels by default (#6380) Simon Mo 2024-07-12 13:56:59 -07:00
• aa48e502fb [MISC] Upgrade dependency to PyTorch 2.3.1 (#5327) Cody Yu 2024-07-12 12:04:26 -07:00