Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• 30efe41532 [Docs] Update docs for wheel location (#6580) Simon MoandGitHub 2024-07-19 12:14:11 -07:00
• 9ed82e7074 [Misc] Small perf improvements (#6520) Antoni BaumandGitHub 2024-07-19 12:10:56 -07:00
• 51f8aa90ad [Bugfix][Frontend] remove duplicate init logger (#6581) DanieleandGitHub 2024-07-19 19:16:27 +02:00
• a5314e8698 [Model] RowParallelLinear: pass bias to quant_method.apply (#6327) Thomas ParnellandGitHub 2024-07-19 15:15:22 +02:00
• a921e86392 [BUGFIX] Raise an error for no draft token case when draft_tp>1 (#6369) Woo-Yeon LeeandGitHub 2024-07-19 22:01:09 +09:00
• 6366efc67b [Bugfix][Frontend] Fix missing /metrics endpoint (#6463) Cyrus LeungandGitHub 2024-07-19 11:55:13 +08:00
• dbe5588554 [ Misc ] non-uniform quantization via compressed-tensors for Llama (#6515) Robert ShawandGitHub 2024-07-18 22:39:18 -04:00
• d4201e06d5 [Bugfix] Make spec. decode respect per-request seed. (#6034) 3 people 2024-07-19 04:22:08 +02:00
• b5672a112c [Core] Multiprocessing Pipeline Parallel support (#6130) 3 people 2024-07-18 19:15:52 -07:00
• c5df56f88b Add support for a rope extension method (#6553) Simon MoandGitHub 2024-07-18 18:53:03 -07:00
• 1689219ebf [CI/Build] Build on Ubuntu 20.04 instead of 22.04 (#6517) Tyler Michael SmithandGitHub 2024-07-18 20:29:25 -04:00
• 4ffffccb7e [Kernel] Implement fallback for FP8 channelwise using torch._scaled_mm (#6552) Tyler Michael SmithandGitHub 2024-07-18 19:52:22 -04:00
• f53b8f0d05 [ci][test] add correctness test for cpu offloading (#6549) youkaichaoandGitHub 2024-07-18 16:41:06 -07:00
• 2d4733ba2d Fix PR comment bot (#6554) Kevin H. LuuandGitHub 2024-07-18 14:48:29 -07:00
• 15c6a079b1 [Model] Support Mistral-Nemo (#6548) Michael GoinandGitHub 2024-07-18 16:31:50 -04:00
• ecdb462c24 [ci] Reword Github bot comment (#6534) Kevin H. LuuandGitHub 2024-07-18 08:01:45 -07:00
• 58ca663224 [ Misc ] Improve Min Capability Checking in compressed-tensors (#6522) Robert ShawandGitHub 2024-07-18 10:39:12 -04:00
• 4634c8728b [TPU] Refactor TPU worker & model runner (#6506) Woosuk KwonandGitHub 2024-07-18 01:34:16 -07:00
• c8a7d51c49 [Bugfix] Update flashinfer.py with PagedAttention forwards - Fixes Gemma2 OpenAI Server Crash (#6501) Noam GatandGitHub 2024-07-18 10:47:13 +03:00
• e2fbaee725 [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (#6227) 3 people 2024-07-18 00:13:30 -07:00
• 8a74c68bd1 [Misc] Minor patch for draft model runner (#6523) Cody YuandGitHub 2024-07-17 23:06:21 -07:00
• 61e592747c [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) 3 people 2024-07-17 22:27:09 -07:00
• d25877dd9b [BugFix] Avoid secondary error in ShmRingBuffer destructor (#6530) Nick HillandGitHub 2024-07-17 22:24:43 -07:00
• 1c27d25fb5 [core][model] yet another cpu offload implementation (#6496) 3 people 2024-07-17 20:54:35 -07:00
• 18fecc3559 [ Kernel ] Fp8 Channelwise Weight Support (#6487) Robert ShawandGitHub 2024-07-17 23:18:13 -04:00
• b5af8c223c [Model] Pipeline parallel support for Mixtral (#6516) Cody YuandGitHub 2024-07-17 19:26:04 -07:00
• b5241e41d9 [ Kernel ] FP8 Dynamic-Per-Token Quant Kernel (#6511) 3 people 2024-07-17 21:38:35 -04:00
• e76466dde2 [Core] draft_model_runner: Implement prepare_inputs on GPU for advance_step (#6338) Alexander MatveevandGitHub 2024-07-17 17:30:28 -04:00
• 5f0b9933e6 [Bugfix] Fix Ray Metrics API usage (#6354) Antoni BaumandGitHub 2024-07-17 12:40:10 -07:00
• a38524f338 [DOC] - Add docker image to Cerebrium Integration (#6510) milo157andGitHub 2024-07-17 13:22:53 -04:00
• 2fa4623d9e [Core] Refactor _prepare_model_input_tensors - take 2 (#6164) Cody YuandGitHub 2024-07-17 09:37:16 -07:00
• a9a2e74d21 [Misc] Use torch.Tensor for type annotation (#6505) Woosuk KwonandGitHub 2024-07-17 06:01:10 -07:00
• e09ce759aa [TPU] Remove multi-modal args in TPU backend (#6504) Woosuk KwonandGitHub 2024-07-17 04:02:53 -07:00
• 5fa6e9876e [Bugfix] Fix for multinode crash on 4 PP (#6495) Murali AndoorveeduandGitHub 2024-07-17 04:25:10 -04:00
• 5bf35a91e4 [Doc][CI/Build] Update docs and tests to use vllm serve (#6431) Cyrus LeungandGitHub 2024-07-17 15:43:21 +08:00
• a19e8d3726 [Misc][Speculative decoding] Typos and typing fixes (#6467) 3 people 2024-07-17 15:17:07 +08:00
• 10383887e0 [ROCm] Cleanup Dockerfile and remove outdated patch (#6482) Hongxia YangandGitHub 2024-07-17 01:47:02 -04:00
• 1d094fd7c0 [Distributed][PP] only create embedding & lm head when necessary (#6455) Wushi DongandGitHub 2024-07-16 19:20:26 -07:00
• ce37be7ba0 [misc][distributed] add seed to dummy weights (#6491) youkaichaoandGitHub 2024-07-16 19:16:34 -07:00
• 7f62077af5 [misc][distributed] improve tests (#6488) youkaichaoandGitHub 2024-07-16 17:35:52 -07:00
• 09c2eb85dd [ci][distributed] add pipeline parallel correctness test (#6410) youkaichaoandGitHub 2024-07-16 15:44:22 -07:00
• 978aed5300 [Kernel][Attention] Separate Attention.kv_scale into k_scale and v_scale (#6081) Michael GoinandGitHub 2024-07-16 18:31:32 -04:00
• 160e1d8c99 [Misc] Log spec decode metrics (#6454) Cody YuandGitHub 2024-07-16 13:37:10 -07:00
• 94162beb9f [Doc] Fix the lora adapter path in server startup script (#6230) Jiaxin ShanandGitHub 2024-07-16 10:11:04 -07:00
• c467dff24f [Hardware][TPU] Support MoE with Pallas GMM kernel (#6457) Woosuk KwonandGitHub 2024-07-16 09:56:28 -07:00
• 9f4ccec761 [doc][misc] remind to cancel debugging environment variables (#6481) youkaichaoandGitHub 2024-07-16 09:45:30 -07:00
• 38ef94888a [CI/Build] Remove "boardwalk" image asset (#6460) Cyrus LeungandGitHub 2024-07-16 23:59:36 +08:00
• 2bb0489cb3 [Core] Use numpy to speed up padded token processing (#6442) Peng GuanwenandGitHub 2024-07-16 23:13:25 +08:00
• 7508a3dc34 [Misc] Fix typos in spec. decode metrics logging. (#6470) Thomas ParnellandGitHub 2024-07-16 15:55:15 +02:00
• 7a3d2a5b95 [Frontend] Support for chat completions input in the tokenize endpoint (#5923) sasha0552andGitHub 2024-07-16 12:18:09 +00:00
• d97011512e [CI/Build] vLLM cache directory for images (#6444) Cyrus LeungandGitHub 2024-07-16 14:12:25 +08:00
• 37d776606f [Docs] Announce 5th meetup (#6458) Woosuk KwonandGitHub 2024-07-15 21:04:58 -07:00
• d92b3c5cde [Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (#6419) JoeandGitHub 2024-07-15 18:54:15 -07:00
• 9ad32dacd9 [BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (#6425) 3 people 2024-07-16 04:32:55 +03:00
• d6f3b3d5c4 Pin sphinx-argparse version (#6453) Kevin H. LuuandGitHub 2024-07-15 18:26:11 -07:00
• 4552e37b55 [CI/Build][TPU] Add TPU CI test (#6277) 3 people 2024-07-15 14:31:16 -07:00
• ec9933f4a5 [Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (#6289) Woosuk KwonandGitHub 2024-07-15 12:02:14 -07:00
• 3dee97b05f [Docs] Add Google Cloud to sponsor list (#6450) Woosuk KwonandGitHub 2024-07-15 11:58:10 -07:00
• 4cf256ae7f [misc][distributed] fix pp missing layer condition (#6446) youkaichaoandGitHub 2024-07-15 10:32:35 -07:00
• 64fdc08c72 bump version to v0.5.2 (#6433) Simon MoandGitHub 2024-07-15 10:27:40 -07:00
• 4ef95b0f06 [Bugfix] use float32 precision in samplers/test_logprobs.py for comparing with HF (#6409) Thomas ParnellandGitHub 2024-07-15 19:14:49 +02:00
• eaec4b9153 [Bugfix] Add custom Triton cache manager to resolve MoE MP issue (#6140) 3 people 2024-07-15 19:12:47 +02:00
• a63a4c6341 [Misc] Use 0.0.9 version for flashinfer (#6447) 3 people 2024-07-15 10:10:26 -07:00
• c8fd97f26d [Kernel] Use CUTLASS kernels for the FP8 layers with Bias (#6270) Tyler Michael SmithandGitHub 2024-07-15 13:05:52 -04:00
• 94b82e8c18 [doc][distributed] add suggestion for distributed inference (#6418) youkaichaoandGitHub 2024-07-15 09:45:51 -07:00
• 6ae1597ddf [VLM] Minor space optimization for ClipVisionModel (#6436) Roger WangandGitHub 2024-07-15 02:29:51 -07:00
• 22e79ee8f3 [doc][misc] doc update (#6439) youkaichaoandGitHub 2024-07-14 23:33:25 -07:00
• de19916314 [Bugfix] Convert image to RGB by default (#6430) Cyrus LeungandGitHub 2024-07-15 13:39:15 +08:00
• 69672f116c [core][distributed] simplify code to support pipeline parallel (#6406) youkaichaoandGitHub 2024-07-14 21:20:51 -07:00
• 44874a0bf9 [Doc] add env docs for flashinfer backend (#6437) DefTruthandGitHub 2024-07-15 12:16:51 +08:00
• b47008b4d2 [BugFix] BatchResponseData body should be optional (#6345) 3 people 2024-07-14 21:06:09 -07:00
• 9bfece89fd Add FUNDING.yml (#6435) Simon MoandGitHub 2024-07-14 20:36:16 -07:00
• 32c9d7f765 Report usage for beam search (#6404) Simon MoandGitHub 2024-07-14 19:37:35 -07:00
• ccb20db8bd [Bugfix] Benchmark serving script used global parameter 'args' in function 'sample_random_requests' (#6428) FishandGitHub 2024-07-15 10:27:01 +08:00
• a754dc2cb9 [CI/Build] Cross python wheel (#6394) Robert ShawandGitHub 2024-07-14 21:54:46 -04:00
• 61e85dbad8 [Doc] xpu backend requires running setvars.sh (#6393) Robert CohnandGitHub 2024-07-14 20:10:11 -04:00
• dbfe254eda [Feature] vLLM CLI (#5090) 3 people 2024-07-14 15:36:43 -07:00
• 73030b7dae [ Misc ] Enable Quantizing All Layers of DeekSeekv2 (#6423) Robert ShawandGitHub 2024-07-14 17:38:42 -04:00
• ccd3c04571 [ci][build] fix commit id (#6420) 3 people 2024-07-14 07:16:21 -07:00
• 9dad5cc859 [Kernel] Turn off CUTLASS scaled_mm for Ada Lovelace (#6384) Tyler Michael SmithandGitHub 2024-07-14 09:37:19 -04:00
• 6ef3bf912c Remove unnecessary trailing period in spec_decode.rst (#6405) Yuan TangandGitHub 2024-07-14 02:58:09 -05:00
• 540c0368b1 [Model] Initialize Fuyu-8B support (#3924) 3 people 2024-07-14 13:27:14 +08:00
• fb6af8bc08 [ Misc ] Apply MoE Refactor to Deepseekv2 To Support Fp8 (#6417) Robert ShawandGitHub 2024-07-13 23:03:58 -04:00
• eeceadaecc [Misc] Add deprecation warning for beam search (#6402) Woosuk KwonandGitHub 2024-07-13 11:52:22 -07:00
• babf52dade [ Misc ] More Cleanup of Marlin (#6359) 3 people 2024-07-13 06:21:37 -04:00
• 9da4aad44b Updating LM Format Enforcer version to v10.3 (#6411) Noam GatandGitHub 2024-07-13 13:09:12 +03:00
• 41708e5034 [ci] try to add multi-node tests (#6280) 3 people 2024-07-12 21:51:48 -07:00
• d80aef3776 [Docs] Clean up latest news (#6401) Woosuk KwonandGitHub 2024-07-12 19:36:53 -07:00
• e1684a766a [Bugfix] Fix hard-coded value of x in context_attention_fwd (#6373) Thomas ParnellandGitHub 2024-07-13 03:30:54 +02:00
• a27f87da34 [Doc] Fix Typo in Doc (#6392) 3 people 2024-07-12 17:48:23 -07:00
• 16ff6bd58c [ci] Fix wording for GH bot (#6398) Kevin H. LuuandGitHub 2024-07-12 16:34:37 -07:00
• f8f9ff57ee [Bugfix][TPU] Fix megacore setting for v5e-litepod (#6397) Woosuk KwonandGitHub 2024-07-12 15:59:47 -07:00
• 6bc9710f6e Fix release pipeline's dir permission (#6391) Simon MoandGitHub 2024-07-12 15:52:43 -07:00
• 111fc6e7ec [Misc] Add generated git commit hash as vllm.__commit__ (#6386) Michael GoinandGitHub 2024-07-12 18:52:15 -04:00
• 75f64d8b94 [Bugfix] Fix illegal memory access in FP8 MoE kernel (#6382) Cody YuandGitHub 2024-07-12 14:33:33 -07:00
• 21b2dcedab Fix release pipeline's -e flag (#6390) Simon MoandGitHub 2024-07-12 14:08:04 -07:00
• 07b35af86d Fix interpolation in release pipeline (#6389) Simon MoandGitHub 2024-07-12 14:03:39 -07:00
• bb1a784b05 Fix release-pipeline.yaml (#6388) Simon MoandGitHub 2024-07-12 14:00:57 -07:00
• d719ba24c5 Build some nightly wheels by default (#6380) Simon MoandGitHub 2024-07-12 13:56:59 -07:00
• aa48e502fb [MISC] Upgrade dependency to PyTorch 2.3.1 (#5327) Cody YuandGitHub 2024-07-12 12:04:26 -07:00