Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• 4dbebd03cc [ci] Add GHA workflows to enable full CI run (#6381) Kevin H. LuuandGitHub 2024-07-12 11:36:26 -07:00
• b75bce1008 [ci] Add grouped tests & mark tests to run by default for fastcheck pipeline (#6365) Kevin H. LuuandGitHub 2024-07-12 09:58:38 -07:00
• b039cbbce3 [Misc] add fixture to guided processor tests (#6341) Yihuan BuandGitHub 2024-07-12 12:55:39 -04:00
• f9d25c2519 [Build/CI] Checking/Waiting for the GPU's clean state (#6379) Alexei-V-Ivanov-AMDandGitHub 2024-07-12 11:42:24 -05:00
• 024ad87cdc [Bugfix] Fix dtype mismatch in PaliGemma (#6367) Cyrus LeungandGitHub 2024-07-12 23:22:18 +08:00
• aea19f0989 [ Misc ] Support Models With Bias in compressed-tensors integration (#6356) Robert ShawandGitHub 2024-07-12 11:11:29 -04:00
• f7160d946a [Misc][Bugfix] Update transformers for tokenizer issue (#6364) Roger WangandGitHub 2024-07-12 01:40:07 -07:00
• 6047187cd8 [ Misc ] Remove separate bias add (#6353) Robert ShawandGitHub 2024-07-12 01:06:09 -04:00
• b6c16cf8ff [ROCm][AMD] unify CUDA_VISIBLE_DEVICES usage in cuda/rocm (#6352) Hongxia YangandGitHub 2024-07-12 00:30:46 -04:00
• d26a8b3f1f [CI/Build] (2/2) Switching AMD CI to store images in Docker Hub (#6350) adityagoel14andGitHub 2024-07-12 00:26:26 -04:00
• d59eb98489 [Model][Phi3-Small] Remove scipy from blocksparse_attention (#6343) Michael GoinandGitHub 2024-07-11 22:47:17 -04:00
• adf32e0a0f [Bugfix] Fix usage stats logging exception warning with OpenVINO (#6349) Helena KloostermanandGitHub 2024-07-12 04:47:00 +02:00
• 2b0fb53481 [distributed][misc] be consistent with pytorch for libcudart.so (#6346) youkaichaoandGitHub 2024-07-11 19:35:17 -07:00
• d6ab528997 [Misc] Remove flashinfer warning, add flashinfer tests to CI (#6351) Lily LiuandGitHub 2024-07-11 18:32:06 -07:00
• 7ed6a4f0e1 [ BugFix ] Prompt Logprobs Detokenization (#6223) 3 people 2024-07-11 18:02:29 -04:00
• a4feba929b [CI/Build] Add nightly benchmarking for tgi, tensorrt-llm and lmdeploy (#5362) Kuntai DuandGitHub 2024-07-11 13:28:38 -07:00
• 2d23b42d92 [doc] update pipeline parallel in readme (#6347) youkaichaoandGitHub 2024-07-11 11:38:40 -07:00
• 1df43de9bb [bug fix] Fix llava next feature size calculation. (#6339) xwjiang2010andGitHub 2024-07-11 10:21:10 -07:00
• 52b7fcb35a Benchmark: add H100 suite (#6047) Simon MoandGitHub 2024-07-11 09:17:07 -07:00
• b675069d74 [ Misc ] Refactor Marlin Python Utilities (#6082) 3 people 2024-07-11 11:40:11 -04:00
• 55f692b46e [BugFix] get_and_reset only when scheduler outputs are not empty (#6266) Mor ZusmanandGitHub 2024-07-11 17:40:20 +03:00
• 8a1415cf77 [Bugfix] GPTBigCodeForCausalLM: Remove lm_head from supported_lora_modules. (#6326) 3 people 2024-07-11 16:05:59 +02:00
• 546b101fa0 [BugFix]: fix engine timeout due to request abort (#6255) 3 people 2024-07-11 21:46:31 +08:00
• 3963a5335b [Misc] refactor(config): clean up unused code (#6320) aniaanandGitHub 2024-07-11 17:39:07 +08:00
• c4774eb841 [Bugfix] Fix snapshot download in serving benchmark (#6318) Roger WangandGitHub 2024-07-11 00:04:05 -07:00
• fc17110bbe [BugFix]: set outlines pkg version (#6262) Lim Xiang YangandGitHub 2024-07-11 12:37:11 +08:00
• 439c84581a [Doc] Update description of vLLM support for CPUs (#6003) Jie Fu (傅杰)andGitHub 2024-07-11 12:15:29 +08:00
• 99ded1e1c4 [Doc] Remove comments incorrectly copied from another project (#6286) daquexianandGitHub 2024-07-11 01:05:26 +01:00
• 997df46a32 [Bugfix][Neuron] Fix soft prompt method error in NeuronExecutor (#6313) Woosuk KwonandGitHub 2024-07-10 16:39:02 -07:00
• ae151d73be [Speculative Decoding] Enabling bonus token in speculative decoding for KV cache based models (#5765) sroy745andGitHub 2024-07-10 16:02:47 -07:00
• 44cc76610d [Bugfix] Fix OpenVINOExecutor abstractmethod error (#6296) sangjune.parkandGitHub 2024-07-11 02:03:32 +09:00
• b422d4961a [CI/Build] Enable mypy typing for remaining folders (#6268) Benjamin MuskallaandGitHub 2024-07-10 16:15:55 +02:00
• c38eba3046 [Bugfix] MLPSpeculator: Use ParallelLMHead in tie_weights=False case. (#6303) Thomas ParnellandGitHub 2024-07-10 15:04:07 +02:00
• e72ae80b06 [Bugfix] Support 2D input shape in MoE layer (#6287) Woosuk KwonandGitHub 2024-07-10 06:03:16 -07:00
• 8a924d2248 [Doc] Guide for adding multi-modal plugins (#6205) Cyrus LeungandGitHub 2024-07-10 14:55:34 +08:00
• 5ed3505d82 [Bugfix][TPU] Add prompt adapter methods to TPUExecutor (#6279) Woosuk KwonandGitHub 2024-07-09 19:30:56 -07:00
• da78caecfa [core][distributed] zmq fallback for broadcasting large objects (#6183) youkaichaoandGitHub 2024-07-09 18:49:11 -07:00
• 2416b26e11 [Speculative Decoding] Medusa Implementation with Top-1 proposer (#4978) Abhinav GoyalandGitHub 2024-07-10 07:04:02 +05:30
• d3a245138a [Bugfix]fix and needs_scalar_to_array logic check (#6238) 3 people 2024-07-10 07:43:24 +08:00
• 673dd4cae9 [Docs] Docs update for Pipeline Parallel (#6222) 3 people 2024-07-09 16:24:58 -07:00
• 4d6ada947c [CORE] Adding support for insertion of soft-tuned prompts (#4645) 5 people 2024-07-09 16:26:36 -04:00
• a0550cbc80 Add support for multi-node on CI (#5955) Kevin H. LuuandGitHub 2024-07-09 12:56:56 -07:00
• 08c5bdecae [Bugfix][TPU] Fix outlines installation in TPU Dockerfile (#6256) Woosuk KwonandGitHub 2024-07-09 02:56:06 -07:00
• 5d5b4c5fe5 [Bugfix][TPU] Add missing None to model input (#6245) Woosuk KwonandGitHub 2024-07-09 00:21:37 -07:00
• 70c232f85a [core][distributed] fix ray worker rank assignment (#6235) youkaichaoandGitHub 2024-07-08 21:31:44 -07:00
• a3c9435d93 [hardware][cuda] use device id under CUDA_VISIBLE_DEVICES for get_device_capability (#6216) youkaichaoandGitHub 2024-07-08 20:02:15 -07:00
• 4f0e0ea131 Add FlashInfer to default Dockerfile (#6172) Simon MoandGitHub 2024-07-08 13:38:03 -07:00
• ddc369fba1 [Bugfix] Mamba cache Cuda Graph padding (#6214) tomeras91andGitHub 2024-07-08 21:25:51 +03:00
• 185ad31f37 [Bugfix] use diskcache in outlines _get_guide #5436 (#6203) EricandGitHub 2024-07-09 02:23:24 +08:00
• 543aa48573 [Kernel] Correctly invoke prefill & decode kernels for cross-attention (towards eventual encoder/decoder model support) (#4888) 3 people 2024-07-08 13:12:15 -04:00
• f7a8fa39d8 [Kernel] reloading fused_moe config on the last chunk (#6210) Avshalom ManevichandGitHub 2024-07-08 18:00:38 +03:00
• 717f4bcea0 Feature/add benchmark testing (#5947) 3 people 2024-07-08 15:52:06 +08:00
• 16620f439d do not exclude object field in CompletionStreamResponse (#6196) kczimmandGitHub 2024-07-07 21:32:57 -05:00
• 3b08fe2b13 [misc][frontend] log all available endpoints (#6195) 3 people 2024-07-07 15:11:12 -07:00
• abfe705a02 [ Misc ] Support Fp8 via llm-compressor (#6110) 3 people 2024-07-07 16:42:11 -04:00
• 333306a252 add benchmark for fix length input and output (#5857) 3 people 2024-07-07 15:42:13 +08:00
• 6206dcb29e [Model] Add PaliGemma (#5189) 3 people 2024-07-06 18:25:50 -07:00
• 9389380015 [Doc] Move guide for multimodal model and other improvements (#6168) Cyrus LeungandGitHub 2024-07-06 17:18:59 +08:00
• 175c43eca4 [Doc] Reorganize Supported Models by Type (#6167) Roger WangandGitHub 2024-07-05 22:59:36 -07:00
• bc96d5c330 Move release wheel env var to Dockerfile instead (#6163) Simon MoandGitHub 2024-07-05 17:19:53 -07:00
• f0250620dd Fix release wheel build env var (#6162) Simon MoandGitHub 2024-07-05 16:24:31 -07:00
• 2de490d60f Update wheel builds to strip debug (#6161) Simon MoandGitHub 2024-07-05 14:51:25 -07:00
• 79d406e918 [Docs] Fix readthedocs for tag build (#6158) Simon MoandGitHub 2024-07-05 12:44:40 -07:00
• abad5746a7 bump version to v0.5.1 (#6157) Simon MoandGitHub 2024-07-05 12:04:51 -07:00
• e58294ddf2 [Bugfix] Add verbose error if scipy is missing for blocksparse attention (#5695) JGSweetsandGitHub 2024-07-05 12:41:01 -05:00
• f1e15da6fe [Frontend] Continuous usage stats in OpenAI completion API (#5742) jvlunterenandGitHub 2024-07-05 19:37:09 +02:00
• 0097bb1829 [Bugfix] Use templated datasource in grafana.json to allow automatic imports (#6136) Christian RohmannandGitHub 2024-07-05 18:49:47 +02:00
• ea4b570483 [VLM] Cleanup validation and update docs (#6149) Cyrus LeungandGitHub 2024-07-05 13:49:38 +08:00
• a41357e941 [VLM] Improve consistency between feature size calculation and dummy data for profiling (#6146) Roger WangandGitHub 2024-07-04 18:29:47 -07:00
• ae96ef8fbd [VLM] Calculate maximum number of multi-modal tokens by model (#6121) Cyrus LeungandGitHub 2024-07-05 07:37:23 +08:00
• 69ec3ca14c [Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051) 3 people 2024-07-04 16:35:51 -07:00
• 81d7a50f24 [Hardware][Intel CPU] Adding intel openmp tunings in Docker file (#6008) YuanandGitHub 2024-07-05 06:22:12 +08:00
• 27902d42be [misc][doc] try to add warning for latest html (#5979) youkaichaoandGitHub 2024-07-04 09:57:09 -07:00
• 56b325e977 [ROCm][AMD][Model]Adding alibi slopes support in ROCm triton flash attention and naive flash attention (#6043) 3 people 2024-07-04 01:19:38 -04:00
• 3dd507083f [CI/Build] Cleanup VLM tests (#6107) Cyrus LeungandGitHub 2024-07-04 09:58:18 +08:00
• 0ed646b7aa [Distributed][Core] Support Py39 and Py38 for PP (#6120) Murali AndoorveeduandGitHub 2024-07-03 17:52:29 -07:00
• 1dab9bc8a9 [Bugfix] set OMP_NUM_THREADS to 1 by default for multiprocessing (#6109) 3 people 2024-07-03 17:56:59 -06:00
• 3de6e6a30e [core][distributed] support n layers % pp size != 0 (#6115) youkaichaoandGitHub 2024-07-03 16:40:31 -07:00
• 966fe72141 [doc][misc] bump up py version in installation doc (#6119) youkaichaoandGitHub 2024-07-03 15:52:04 -07:00
• 62963d129e [ Misc ] Clean Up CompressedTensorsW8A8 (#6113) Robert ShawandGitHub 2024-07-03 18:50:08 -04:00
• d9e98f42e4 [vlm] Remove vision language config. (#6089) 3 people 2024-07-03 15:14:16 -07:00
• 3c6325f0fc [core][distributed] custom allreduce when pp size > 1 (#6117) youkaichaoandGitHub 2024-07-03 14:41:32 -07:00
• 47f0954af0 [Kernel] Expand FP8 support to Ampere GPUs using FP8 Marlin (#5975) Michael GoinandGitHub 2024-07-03 13:38:00 -04:00
• 7cd2ebb025 [Bugfix] Fix compute_logits in Jamba (#6093) Roger WangandGitHub 2024-07-03 00:32:35 -07:00
• f1c78138aa [Doc] Fix Mock Import (#6094) Roger WangandGitHub 2024-07-03 00:13:56 -07:00
• 3a86b54fb0 [VLM][Frontend] Proper Image Prompt Formatting from OpenAI API (#6091) 3 people 2024-07-02 23:41:23 -07:00
• f666207161 [misc][distributed] error on invalid state (#6092) youkaichaoandGitHub 2024-07-02 23:37:29 -07:00
• d830656a97 [BugFix] Avoid unnecessary Ray import warnings (#6079) Nick HillandGitHub 2024-07-02 23:09:40 -07:00
• d18bab3587 [CI] Fix base url doesn't strip "/" (#6087) SangBin ChoandGitHub 2024-07-03 13:31:25 +09:00
• 9831aec49f [Core] Dynamic image size support for VLMs (#5276) 6 people 2024-07-03 11:34:00 +08:00
• 482045ee77 [hardware][misc] introduce platform abstraction (#6080) youkaichaoandGitHub 2024-07-02 20:12:22 -07:00
• 9d6a8daa87 [Model] Jamba support (#4115) 8 people 2024-07-03 02:11:29 +03:00
• ee93f4f92a [CORE] Quantized lm-head Framework (#4442) 4 people 2024-07-03 06:25:17 +08:00
• 7c008c51a9 [ Misc ] Refactor MoE to isolate Fp8 From Mixtral (#5970) 4 people 2024-07-02 17:54:35 -04:00
• 4d26d806e1 Update conftest.py (#6076) Robert ShawandGitHub 2024-07-02 16:14:22 -04:00
• c5832d2ae9 [Core] Pipeline Parallel Support (#4412) Murali AndoorveeduandGitHub 2024-07-02 10:58:08 -07:00
• 15aba081f3 [Speculative Decoding] MLPSpeculator Tensor Parallel support (1/2) (#6050) 4 people 2024-07-02 07:20:29 -07:00
• 31354e563f [Doc] Reinstate doc dependencies (#6061) Cyrus LeungandGitHub 2024-07-02 18:53:16 +08:00
• 98d6682cd1 [VLM] Remove image_input_type from VLM config (#5852) 4 people 2024-07-02 00:57:09 -07:00
• 2c37540aa6 [Frontend] Add template related params to request (#5709) danieljannai21andGitHub 2024-07-02 09:01:57 +03:00