Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• d7740ea4dc [Core] Optimize sampler get_logprobs (#4594) SangBin ChoandGitHub 2024-05-09 00:42:28 +09:00
• cc466a3290 [Core][Distributed] support cpu&device in broadcast tensor dict (#4660) youkaichaoandGitHub 2024-05-07 19:34:47 -07:00
• 8344f7742b [Bug fix][Core] fixup ngram not setup correctly (#4551) 5 people 2024-05-08 02:40:18 +08:00
• 469f85c782 [Core][Optimization] change copy-on-write from dict[int, list] to list (#4648) youkaichaoandGitHub 2024-05-07 11:06:32 -07:00
• 10760da800 [Bugfix] Fixed error in slice_lora_b for MergedQKVParallelLinearWithLora (#4609) Austin VeselkaandGitHub 2024-05-07 12:59:07 -05:00
• 478aed5827 [Build/CI] Fixing 'docker run' to re-enable AMD CI tests. (#4642) Alexei-V-Ivanov-AMDandGitHub 2024-05-07 11:23:17 -05:00
• 63575bc2e1 [Core][Optimization] change python dict to pytorch tensor (#4607) youkaichaoandGitHub 2024-05-06 21:30:27 -07:00
• a98187cf72 [Kernel] Make static FP8 scaling more robust (#4570) Philipp MoritzandGitHub 2024-05-06 17:39:28 -07:00
• bd99d22629 Update lm-format-enforcer to 0.10.1 (#4631) Noam GatandGitHub 2024-05-07 02:51:59 +03:00
• 19cb4716ee [CI] Add retry for agent lost (#4633) Cade DanielandGitHub 2024-05-06 16:18:57 -07:00
• e186d37cb1 [CI] use ccache actions properly in release workflow (#4629) Simon MoandGitHub 2024-05-06 15:23:36 -07:00
• 323f27b904 [Bugfix] Fix asyncio.Task not being subscriptable (#4623) Cyrus LeungandGitHub 2024-05-07 00:31:05 +08:00
• 0650e5935b Disable cuda version check in vllm-openai image (#4530) zhaoyang-starandGitHub 2024-05-06 07:58:55 +08:00
• c7f2cf2b7f [CI] Reduce wheel size by not shipping debug symbols (#4602) Simon MoandGitHub 2024-05-04 21:28:58 -07:00
• 8d8357c8ed bump version to v0.4.2 (#4600) Simon MoandGitHub 2024-05-04 17:09:49 -07:00
• 4302987069 [Bugfix] Fix inappropriate content of model_name tag in Prometheus metrics (#3937) DearPlanetandGitHub 2024-05-05 06:39:34 +08:00
• 021b1a2ab7 [CI] check size of the wheels (#4319) Simon MoandGitHub 2024-05-04 13:44:36 -07:00
• 2a052011ca [Kernel] Support MoE Fp8 Checkpoints for Mixtral (Static Weights with Dynamic/Static Activations) (#4527) Michael GoinandGitHub 2024-05-04 14:45:16 -04:00
• 36fb68f947 [Doc] Chunked Prefill Documentation (#4580) SangBin ChoandGitHub 2024-05-04 16:18:00 +09:00
• bc8ad68455 [Misc][Refactor] Introduce ExecuteModelData (#4540) Cody YuandGitHub 2024-05-03 17:47:07 -07:00
• 344bf7cd2d [Misc] add installation time env vars (#4574) youkaichaoandGitHub 2024-05-03 15:55:56 -07:00
• ab50275111 [Speculative decoding] Support target-model logprobs (#4378) Cade DanielandGitHub 2024-05-03 15:52:01 -07:00
• 43c413ec57 [Kernel] Use flashinfer for decoding (#4353) 3 people 2024-05-03 15:51:27 -07:00
• f8e7adda21 Fix/async chat serving (#2727) Sebastian SchoennenbeckandGitHub 2024-05-03 20:04:14 +02:00
• 7e65477e5e [Bugfix] Allow "None" or "" to be passed to CLI for string args that default to None (#4586) Michael GoinandGitHub 2024-05-03 13:32:21 -04:00
• 3521ba4f25 [Core][Model runner refactoring 1/N] Refactor attn metadata term (#4518) SangBin ChoandGitHub 2024-05-04 02:20:12 +09:00
• 2d7bce9cd5 [Doc] add env vars to the doc (#4572) youkaichaoandGitHub 2024-05-02 22:13:49 -07:00
• ce3f1eedf8 [Misc] remove chunk detected debug logs (#4571) DefTruthandGitHub 2024-05-03 12:48:08 +08:00
• 808632d3b4 [BugFix] Prevent the task of _force_log from being garbage collected (#4567) Yang, BoandGitHub 2024-05-02 18:35:18 -07:00
• 344a5d0c33 [Core][Distributed] enable allreduce for multiple tp groups (#4566) youkaichaoandGitHub 2024-05-02 17:32:33 -07:00
• 0f8a91401c [Core] Ignore infeasible swap requests. (#4557) SangBin ChoandGitHub 2024-05-03 06:31:20 +09:00
• 9b5c9f9484 [CI/Build] AMD CI pipeline with extended set of tests. (#4267) 3 people 2024-05-02 14:29:07 -05:00
• 32881f3f31 [kernel] fix sliding window in prefix prefill Triton kernel (#4405) 3 people 2024-05-02 11:23:37 -07:00
• 5b8a7c1cb0 [Misc] centralize all usage of environment variables (#4548) youkaichaoandGitHub 2024-05-02 11:13:25 -07:00
• 1ff0c73a79 [BugFix] Include target-device specific requirements.txt in sdist (#4559) Mark McLoughlinandGitHub 2024-05-02 18:52:51 +01:00
• 5ad60b0cbd [Misc] Exclude the tests directory from being packaged (#4552) Hu DongandGitHub 2024-05-03 01:50:25 +08:00
• fb087af52e [mypy][7/N] Cover all directories (#4555) SangBin ChoandGitHub 2024-05-03 02:47:41 +09:00
• 7038e8b803 [Kernel] Support running GPTQ 8-bit models in Marlin (#4533) alexm-nmandGitHub 2024-05-02 12:56:22 -04:00
• 2a85f93007 [Core][Distributed] enable multiple tp group (#4512) 3 people 2024-05-01 21:28:21 -07:00
• cf8cac8c70 [mypy][6/N] Fix all the core subdirectory typing (#4450) 3 people 2024-05-02 12:01:00 +09:00
• 5e401bce17 [CI]Add regression tests to ensure the async engine generates metrics (#4524) Ronen SchafferandGitHub 2024-05-02 05:57:12 +03:00
• 0d62fe58db [Bug fix][Core] assert num_new_tokens == 1 fails when SamplingParams.n is not 1 and max_tokens is large & Add tests for preemption (#4451) SangBin ChoandGitHub 2024-05-02 11:24:13 +09:00
• b8afa8b95a [MISC] Rework logger to enable pythonic custom logging configuration to be provided (#4273) Danny GuintherandGitHub 2024-05-01 20:34:40 -04:00
• 826b82a260 [Misc] Fix expert_ids shape in MoE (#4517) Woosuk KwonandGitHub 2024-05-01 16:47:59 -07:00
• c9d852d601 [Misc] Remove Mixtral device="cuda" declarations (#4543) Philipp MoritzandGitHub 2024-05-01 16:30:52 -07:00
• 6ef09b08f8 [Core][Distributed] fix pynccl del error (#4508) youkaichaoandGitHub 2024-05-01 15:23:06 -07:00
• 3a922c1e7e [Bugfix][Core] Fix and refactor logging stats (#4336) RoyandGitHub 2024-05-02 04:08:14 +08:00
• c47ba4aaa9 [Bugfix] Add validation for seed (#4529) sasha0552andGitHub 2024-05-01 19:31:22 +00:00
• 24bb4fe432 [Kernel] Update fused_moe tuning script for FP8 (#4457) Philipp MoritzandGitHub 2024-05-01 11:47:38 -07:00
• a657bfc48a [Core] Add multiproc_worker_utils for multiprocessing-based workers (#4357) Nick HillandGitHub 2024-05-01 11:41:59 -07:00
• 24750f4cad [Core] Enable prefix caching with block manager v2 enabled (#4142) 4 people 2024-05-02 02:20:32 +08:00
• b38e42fbca [Speculative decoding] Add ngram prompt lookup decoding (#4237) 3 people 2024-05-02 02:13:03 +08:00
• 8b798eec75 [CI/Build][Bugfix] VLLM_USE_PRECOMPILED should skip compilation (#4534) Travis JohnsonandGitHub 2024-05-01 12:01:50 -06:00
• 69909126a7 [Bugfix] Use random seed if seed is -1 (#4531) sasha0552andGitHub 2024-05-01 17:41:17 +00:00
• e491c7e053 [Doc] update(example model): for OpenAI compatible serving (#4503) FrαnçoisandGitHub 2024-05-01 19:14:16 +02:00
• 4dc8026d86 [Bugfix] Fix 307 Redirect for /metrics (#4523) Robert ShawandGitHub 2024-05-01 12:14:13 -04:00
• a88bb9b032 [Bugfix] Fix the fp8 kv_cache check error that occurs when failing to obtain the CUDA version. (#4173) AnyISalInandGitHub 2024-05-02 00:11:03 +08:00
• 6f1df80436 [Test] Add ignore_eos test (#4519) SangBin ChoandGitHub 2024-05-01 21:45:42 +09:00
• d6f4bd7cdd [Misc]Add customized information for models (#4132) Jee LiandGitHub 2024-05-01 12:18:14 +08:00
• c3845d82dc Allow user to define whitespace pattern for outlines (#4305) Robert CaulkandGitHub 2024-05-01 05:48:39 +02:00
• a822eb3413 [Misc] fix typo in block manager (#4453) Pastel！andGitHub 2024-05-01 11:41:32 +08:00
• f458112e8a [Misc][Typo] type annotation fix (#4495) harrywuandGitHub 2024-05-01 11:21:39 +08:00
• 2e240c69a9 [Core] Centralize GPU Worker construction (#4419) Nick HillandGitHub 2024-04-30 18:06:34 -07:00
• ee37328da0 Unable to find Punica extension issue during source code installation (#4494) 3 people 2024-05-01 08:42:09 +08:00
• 6ad58f42c5 fix_tokenizer_snapshot_download_bug (#4493) fuchen.ljlandGitHub 2024-05-01 07:38:50 +08:00
• dd1a50a8bc [Bugfix][Minor] Make ignore_eos effective (#4468) Li, JiangandGitHub 2024-05-01 07:33:33 +08:00
• 715c2d854d [Frontend] [Core] Tensorizer: support dynamic num_readers, update version (#4467) Alpay AriyakandGitHub 2024-04-30 19:32:13 -04:00
• a494140433 [Frontend] Support complex message content for chat completions endpoint (#3467) 4 people 2024-05-01 01:28:46 +02:00
• 111815d482 [Kernel] Support Fp8 Checkpoints (Dynamic + Static) (#4332) 7 people 2024-04-30 17:46:12 -04:00
• b31a1fb63c [Doc] add visualization for multi-stage dockerfile (#4456) 3 people 2024-04-30 10:41:59 -07:00
• 4bb53e2dde [BugFix] fix num_lookahead_slots missing in async executor (#4165) 3 people 2024-05-01 01:12:59 +08:00
• 26f2fb5113 [Core]Refactor gptq_marlin ops (#4466) Kunshang JiandGitHub 2024-04-30 12:14:47 +00:00
• fa32207842 [Bugfix][Kernel] Fix compute_type for MoE kernel (#4463) Woosuk KwonandGitHub 2024-04-29 22:05:40 -07:00
• d627a3d837 [Misc] Upgrade to torch==2.3.0 (#4454) Michael GoinandGitHub 2024-04-29 20:05:47 -04:00
• f4f921b7f1 [Core][Distributed] use cpu group to broadcast metadata in cpu (#4444) youkaichaoandGitHub 2024-04-29 13:52:22 -07:00
• ac5ccf0156 [CI] hotfix: soft fail neuron test (#4458) Simon MoandGitHub 2024-04-29 12:50:01 -07:00
• 73c8d677e5 [Kernel] Marlin Expansion: Support AutoGPTQ Models with Marlin (#3922) 4 people 2024-04-29 12:35:34 -04:00
• df29793dc7 [mypy][5/N] Support all typing on model executor (#4427) SangBin ChoandGitHub 2024-04-29 11:01:26 +09:00
• 03dd7d52bf [CI] clean docker cache for neuron (#4441) Simon MoandGitHub 2024-04-28 16:32:07 -07:00
• bf480c5302 Add more Prometheus metrics (#2764) 4 people 2024-04-29 01:59:33 +03:00
• 9c7306ac11 [Misc] fix typo in llm_engine init logging (#4428) DefTruthandGitHub 2024-04-28 18:58:30 +08:00
• 4ea1f9678d [BugFix] Resolved Issues For LinearMethod --> QuantConfig (#4418) Robert ShawandGitHub 2024-04-27 14:35:33 -04:00
• ba4be44c32 [BugFix] Fix return type of executor execute_model methods (#4402) Nick HillandGitHub 2024-04-27 11:17:45 -07:00
• d6e520e170 [Core] Support offline use of local cache for models (#4374) 3 people 2024-04-27 09:59:55 -07:00
• 81661da7b2 [BugFix] Fix min_tokens when eos_token_id is None (#4389) 3 people 2024-04-27 09:52:46 -07:00
• dfea173148 [Bugfix] Abort requests when the connection to /v1/completions is interrupted (#4363) Ruoyu QinandGitHub 2024-04-28 00:48:37 +08:00
• 7134303cbb [Bugfix][Core] Fix get decoding config from ray (#4335) RoyandGitHub 2024-04-27 19:30:08 +08:00
• 3da24c2df7 [Model] Phi-3 4k sliding window temp. fix (#4380) Caio MendesandGitHub 2024-04-27 07:08:15 -03:00
• eefeb16464 [Kernel] Full Tensor Parallelism for LoRA Layers (#3524) 3 people 2024-04-27 02:03:48 -05:00
• 18d23f642a [ROCm][Hardware][AMD] Enable group query attention for triton FA (#4406) Hongxia YangandGitHub 2024-04-27 02:37:40 -04:00
• 87f545ba6f [Misc] Fix logger format typo (#4396) RoyandGitHub 2024-04-27 13:45:02 +08:00
• 8947bc3c15 [Frontend][Bugfix] Disallow extra fields in OpenAI API (#4355) Cyrus LeungandGitHub 2024-04-27 13:08:24 +08:00
• 12628d3c78 [Kernel] Optimize FP8 support for MoE kernel / Mixtral via static scales (#4343) 3 people 2024-04-26 21:49:59 -07:00
• 258a2c58d0 [Core] Introduce DistributedGPUExecutor abstract class (#4348) Nick HillandGitHub 2024-04-26 21:14:26 -07:00
• aba47be3fe [Misc] add RFC issue template (#4401) 3 people 2024-04-26 15:47:45 -07:00
• a62aaf1df5 [Misc][Refactor] Generalize linear_method to be quant_method (#4373) Cody YuandGitHub 2024-04-26 13:41:14 -07:00
• 603ad84815 [Core] Refactoring sampler and support prompt logprob for chunked prefill (#4309) SangBin ChoandGitHub 2024-04-26 22:02:02 +09:00
• a88081bf76 [CI] Disable non-lazy string operation on logging (#4326) 3 people 2024-04-26 16:16:58 +09:00
• 2f30e7c72f [Frontend] Add --log-level option to api server (#4377) Norman MuandGitHub 2024-04-25 22:36:01 -07:00
• a74dee9b62 [Bugfix] Fix parameter name in get_tokenizer (#4107) Cyrus LeungandGitHub 2024-04-26 10:10:48 +08:00