Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• f342153b48 Revert "bump version to v0.4.0" (#3708) youkaichaoandGitHub 2024-03-28 18:49:42 -07:00
• 27a57cad52 bump version to v0.4.0 (#3705) Simon MoandGitHub 2024-03-28 18:26:51 -07:00
• 98a42e7078 [Benchmark] Change mii to use persistent deployment and support tensor parallel (#3628) Yile (Michael) GuandGitHub 2024-03-28 17:33:52 -07:00
• 0267fef52a [Core] fix del of communicator (#3702) youkaichaoandGitHub 2024-03-28 17:24:58 -07:00
• 4716a32dd4 fix logging msg for block manager (#3701) Simon MoandGitHub 2024-03-28 16:29:55 -07:00
• c0935c96d3 [Bugfix] Set enable_prefix_caching=True in prefix caching example (#3703) Woosuk KwonandGitHub 2024-03-28 16:26:30 -07:00
• cb40b3ab6b [Kernel] Add MoE Triton kernel configs for A100 40GB (#3700) Woosuk KwonandGitHub 2024-03-28 15:26:24 -07:00
• 515386ef3c [Core] Support multi-node inference(eager and cuda graph) (#3686) RoyandGitHub 2024-03-29 06:01:55 +08:00
• a4075cba4d [CI] Add test case to run examples scripts (#3638) Simon MoandGitHub 2024-03-28 14:36:10 -07:00
• 96aa014d1e fix benchmark format reporting in buildkite (#3693) Simon MoandGitHub 2024-03-28 14:35:16 -07:00
• 1715056fef [Bugfix] Update neuron_executor.py to add optional vision_language_config (#3695) Adam BoeglinandGitHub 2024-03-28 10:43:34 -07:00
• b51c1cc9d2 [2/N] Chunked prefill data update (#3538) SangBin ChoandGitHub 2024-03-29 02:06:01 +09:00
• ce567a2926 [Kernel] DBRX Triton MoE kernel H100 (#3692) Roger WangandGitHub 2024-03-28 10:05:34 -07:00
• d6ea427f04 [Model] Add support for Qwen2MoeModel (#3346) wenyujin333andGitHub 2024-03-28 23:19:59 +08:00
• 14ccd94c89 [Core][Bugfix]Refactor block manager for better testability (#3492) Cade DanielandGitHub 2024-03-27 23:59:28 -07:00
• 8267b06c30 [Kernel] Add Triton MoE kernel configs for DBRX on A100 (#3679) Woosuk KwonandGitHub 2024-03-27 22:22:25 -07:00
• 3492859b68 [CI/Build] update default number of jobs and nvcc threads to avoid overloading the system (#3675) youkaichaoandGitHub 2024-03-27 21:18:54 -07:00
• 098e1776ba [Model] Add support for xverse (#3610) 4 people 2024-03-28 09:12:54 +08:00
• 10e6322283 [Model] Fix and clean commandr (#3671) RoyandGitHub 2024-03-28 08:20:00 +08:00
• 6d9aa00fc4 [Docs] Add Command-R to supported models (#3669) Woosuk KwonandGitHub 2024-03-27 15:20:00 -07:00
• 1182607e18 Add support for Cohere's Command-R model (#3433) 4 people 2024-03-27 21:19:32 +00:00
• 45b6ef6513 feat(benchmarks): Add Prefix Caching Benchmark to Serving Benchmark (#3277) Roger WangandGitHub 2024-03-27 13:39:26 -07:00
• 1956931436 [Misc] add the "download-dir" option to the latency/throughput benchmarks (#3621) AmadeusChanandGitHub 2024-03-27 16:39:05 -04:00
• e24336b5a7 [Model] Add support for DBRX (#3660) Megha AgarwalandGitHub 2024-03-27 13:01:46 -07:00
• d18f4e73f3 [Bugfix] [Hotfix] fix nccl library name (#3661) youkaichaoandGitHub 2024-03-27 10:23:54 -07:00
• 82c540bebf [Bugfix] More faithful implementation of Gemma (#3653) Woosuk KwonandGitHub 2024-03-27 09:37:18 -07:00
• 8f44facddd [Core] remove cupy dependency (#3625) youkaichaoandGitHub 2024-03-27 00:33:26 -07:00
• e66b629c04 [Misc] Minor fix in KVCache type (#3652) Woosuk KwonandGitHub 2024-03-26 23:14:06 -07:00
• 76879342a3 [Doc]add lora support (#3649) Jee LiandGitHub 2024-03-27 10:06:46 +08:00
• 566b57c5c4 [Kernel] support non-zero cuda devices in punica kernels (#3636) Jee LiandGitHub 2024-03-27 08:37:42 +08:00
• 0dc72273b8 [BugFix] Fix ipv4 address parsing regression (#3645) Nick HillandGitHub 2024-03-26 14:39:44 -07:00
• a979d9771e [Bugfix] Fix ipv6 address parsing bug (#3641) liiliiliilandGitHub 2024-03-27 02:58:20 +08:00
• 8af890a865 Enable more models to inference based on LoRA (#3382) 3 people 2024-03-26 09:09:31 +08:00
• dfeb2ecc3a [Misc] Include matched stop string/token in responses (#2976) 3 people 2024-03-25 17:31:32 -07:00
• 3a243095e5 Optimize _get_ranks in Sampler (#3623) Antoni BaumandGitHub 2024-03-25 16:03:02 -07:00
• 64172a976c [Feature] Add vision language model support. (#3042) xwjiang2010andGitHub 2024-03-25 14:16:30 -07:00
• f408d05c52 hotfix isort on logprobs ranks pr (#3622) Simon MoandGitHub 2024-03-25 11:55:46 -07:00
• 0b4997e05c [Bugfix] API stream returning two stops (#3450) 3 people 2024-03-25 10:14:34 -07:00
• c13ad1b7bd feat: implement the min_tokens sampling parameter (#3124) 3 people 2024-03-25 11:14:26 -06:00
• 819924e749 [Core] Adding token ranks along with logprobs (#3516) 3 people 2024-03-25 13:13:10 -04:00
• 01bfb22b41 [CI] Try introducing isort. (#3495) SangBin ChoandGitHub 2024-03-25 23:59:47 +09:00
• e67c295b0c [Bugfix] fix automatic prefix args and add log info (#3608) TianYu GUOandGitHub 2024-03-25 20:35:22 +08:00
• 925f3332ca [Core] Refactor Attention Take 2 (#3462) Woosuk KwonandGitHub 2024-03-24 21:39:33 -07:00
• b0dfa91dd7 [Model] Add starcoder2 awq support (#3569) 少年andGitHub 2024-03-25 12:07:36 +08:00
• 56a8652f33 [Bugfix] store lock file in tmp directory (#3578)" (#3599) 3 people 2024-03-24 20:06:50 -07:00
• 6d93d35308 [BugFix] tensor.get_device() -> tensor.device (#3604) Kunshang JiandGitHub 2024-03-25 10:01:13 +08:00
• 837e185142 [CI/Build] fix flaky test (#3602) youkaichaoandGitHub 2024-03-24 17:43:05 -07:00
• 42bc386129 [CI/Build] respect the common environment variable MAX_JOBS (#3600) youkaichaoandGitHub 2024-03-24 17:04:00 -07:00
• 8b268a46a7 [CI] typo fix: is_hip --> is_hip() (#3595) youkaichaoandGitHub 2024-03-24 16:03:06 -07:00
• 41deac4a3d [BugFix] 1D query fix for MoE models (#3597) Nick HillandGitHub 2024-03-24 16:00:16 -07:00
• af9e53496f [BugFix] Fix Falcon tied embeddings (#3590) 3 people 2024-03-24 06:34:01 -07:00
• f8a12ecc7f [Misc] Bump transformers version (#3592) Roger WangandGitHub 2024-03-24 06:32:45 -07:00
• 3c5ab9b811 [Misc] Fix BLOOM copyright notice (#3591) Woosuk KwonandGitHub 2024-03-23 23:30:56 -07:00
• 743a0b7402 [Bugfix] use SoftLockFile instead of LockFile (#3578) kota-iizukaandGitHub 2024-03-24 03:43:11 +09:00
• bfdb1ba5c3 [Core] Improve detokenization performance for prefill (#3469) 3 people 2024-03-22 13:44:12 -07:00
• cf2f084d56 Dynamic scheduler delay to improve ITL performance (#3279) 3 people 2024-03-22 20:28:14 +01:00
• f721096d48 [BugFix] Some fixes for custom allreduce kernels (#2760) Hanzhi ZhouandGitHub 2024-03-21 23:02:58 -07:00
• e90fc21f2e [Hardware][Neuron] Refactor neuron support (#3471) Zhuohan LiandGitHub 2024-03-21 18:22:17 -07:00
• ea5f14e6ff [Bugfix][Model] Fix Qwen2 (#3554) RoyandGitHub 2024-03-22 08:18:58 +08:00
• b7050ca7df [BugFix] gemma loading after quantization or LoRA. (#3553) Taemin LeeandGitHub 2024-03-22 05:16:57 +09:00
• c188ecb080 [Misc] Bump up transformers to v4.39.0 & Remove StarCoder2Config (#3551) 4 people 2024-03-21 07:58:12 -07:00
• 865732342b [Misc][Log] Add log for tokenizer length not equal to vocabulary size (#3500) RoyandGitHub 2024-03-21 18:07:48 +08:00
• 4c07dd28c0 [🚀 Ready to be merged] Added support for Jais models (#3183) Lalit PradhanandGitHub 2024-03-21 13:45:24 +04:00
• 3bbff9e5ab Fix 1D query issue from _prune_hidden_states (#3539) SangBin ChoandGitHub 2024-03-21 17:49:06 +09:00
• 6ebd02bdef [PREFIX CACHING FOLLOW UP] OrderedDict-based evictor (#3431) 4 people 2024-03-21 07:20:04 +01:00
• 523e30ea0c [BugFix] Hot fix in setup.py for neuron build (#3537) Zhuohan LiandGitHub 2024-03-20 17:59:52 -07:00
• f1c0fc3919 Migrate logits computation and gather to model_runner (#3233) RoyandGitHub 2024-03-21 07:25:01 +08:00
• 6e435de766 [1/n][Chunked Prefill] Refactor input query shapes (#3236) SangBin ChoandGitHub 2024-03-21 06:46:05 +09:00
• 426ec4ec67 [1/n] Triton sampling kernel (#3186) 3 people 2024-03-20 14:45:08 -07:00
• 80e254834d [Bugfix] Fix ROCm support in CMakeLists.txt (#3534) James WhedbeeandGitHub 2024-03-20 16:05:03 -05:00
• ba8ae1d84f Check for _is_cuda() in compute_num_jobs (#3481) bnellnmandGitHub 2024-03-20 13:06:56 -04:00
• 84eaa68425 Abort when nvcc command is not found in the PATH (#3527) Allen.DouandGitHub 2024-03-21 00:28:29 +08:00
• 5ee14494e4 [Misc] Remove cache stream and cache events (#3461) Woosuk KwonandGitHub 2024-03-20 00:38:53 -07:00
• 4ad521d8b5 [Core] Add generic typing to LRUCache (#3511) Nick HillandGitHub 2024-03-20 00:36:09 -07:00
• 9474e89ba4 [PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled (#3357) 3 people 2024-03-20 08:11:11 +01:00
• 20478c4d3a Use lru_cache for some environment detection utils (#3508) Simon MoandGitHub 2024-03-19 14:34:15 -07:00
• 63e8b28a99 [Doc] minor fix of spelling in amd-installation.rst (#3506) Jim BurtoftandGitHub 2024-03-19 16:32:30 -04:00
• cc63d03fbb Revert "[Core] Cache some utils" (#3507) Simon MoandGitHub 2024-03-19 13:22:58 -07:00
• 2a60c9bd17 [Doc] minor fix to neuron-installation.rst (#3505) Jim BurtoftandGitHub 2024-03-19 16:21:35 -04:00
• c614cfee58 Update dockerfile with ModelScope support (#3429) ifsheldonandGitHub 2024-03-20 01:54:59 +08:00
• 7341c77d69 [BugFix] Avoid initializing CUDA too early (#3487) Nick HillandGitHub 2024-03-18 23:05:20 -07:00
• ef65dcfa6f [Doc] Add docs about OpenAI compatible server (#3288) Simon MoandGitHub 2024-03-18 22:05:34 -07:00
• 6a9c583e73 [Core] print error before deadlock (#3459) youkaichaoandGitHub 2024-03-18 21:06:23 -07:00
• b37cdce2b1 [Core] Cache some utils (#3474) Antoni BaumandGitHub 2024-03-18 17:14:26 -07:00
• b30880a762 [Misc] Update README for the Third vLLM Meetup (#3479) Zhuohan LiandGitHub 2024-03-18 15:58:38 -07:00
• 49eedea373 [Core] Zero-copy asdict for InputMetadata (#3475) Antoni BaumandGitHub 2024-03-18 15:56:40 -07:00
• 9fdf3de346 Cmake based build system (#2830) bnellnmandGitHub 2024-03-18 18:38:33 -04:00
• c0c17d4896 [Misc] Fix PR Template (#3478) Zhuohan LiandGitHub 2024-03-18 15:00:31 -07:00
• 097aa0ea22 [CI/Build] Fix Bad Import In Test (#3473) Robert ShawandGitHub 2024-03-18 15:28:00 -05:00
• 482b0adf1b [Testing] Add test_config.py to CI (#3437) Cade DanielandGitHub 2024-03-18 12:48:45 -07:00
• 8c654c045f CI: Add ROCm Docker Build (#2886) Simon MoandGitHub 2024-03-18 12:33:47 -07:00
• 9101d832e6 [Bugfix] Make moe_align_block_size AMD-compatible (#3470) Woosuk KwonandGitHub 2024-03-18 11:26:24 -07:00
• 93348d9458 [CI] Shard tests for LoRA and Kernels to speed up (#3445) Simon MoandGitHub 2024-03-17 14:56:30 -07:00
• abfc4f3387 [Misc] Use dataclass for InputMetadata (#3452) 3 people 2024-03-17 03:02:46 -07:00
• 6b78837b29 Fix setup.py neuron-ls issue (#2671) Simon MoandGitHub 2024-03-16 16:00:25 -07:00
• 120157fd2a Support arbitrary json_object in OpenAI and Context Free Grammar (#3211) Simon MoandGitHub 2024-03-16 13:35:27 -07:00
• 8e67598aa6 [Misc] fix line length for entire codebase (#3444) Simon MoandGitHub 2024-03-16 00:36:29 -07:00
• ad50bf4b25 fix lint simon-mo 2024-03-15 22:23:38 -07:00
• cf6ff18246 Fix Baichuan chat template (#3340) Dinghow YangandGitHub 2024-03-16 12:02:12 +08:00
• 14e3f9a1b2 Replace lstrip() with removeprefix() to fix Ruff linter warning (#2958) Ronen SchafferandGitHub 2024-03-16 06:01:30 +02:00