Commit Graph - vllm - Gitea: Git with a cup of tea

wassname/vllm

Fork 0

mirror of https://github.com/wassname/vllm.git synced 2026-09-17 13:00:06 +08:00

87bd7e0515 [CI/Build] change conflict PR comment from mergify (#10080) Russell Bryant 2024-11-06 13:15:42 -05:00
098f94de42 [CI/Build] Drop Python 3.8 support (#10038) Russell BryantandDarkLight1337 2024-11-06 09:31:01 -05:00
399c798608 Remove ScaledActivation for AWQ (#10057) Michael Goin 2024-11-06 09:27:06 -05:00
406d4cc480 [Model][LoRA]LoRA support added for Qwen2VLForConditionalGeneration (#10022) Eric 2024-11-06 22:13:15 +08:00
a5bba7d234 [Model] Add Idefics3 support (#9767) Jee Jee LiandB-201 2024-11-06 19:41:17 +08:00
2003cc3513 [Model][LoRA]LoRA support added for LlamaEmbeddingModel (#10071) Jee Jee Li 2024-11-06 17:49:19 +08:00
6a585a23d2 [Hotfix] Fix ruff errors (#10073) Woosuk Kwon 2024-11-06 01:24:28 -08:00
a02a50e6e5 [Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) +26
Konrad Zawora Kunshang Ji Sanju C Sudhakaran Michal Adamczyk Marceli Fylcek Himangshu Lahkar Vivek Goel yuwenzho Dominika Olszewska barak goldberg Michal Szutenberg Jan Kaniecki Agata Dobrzyniewicz Krzysztof Wisniewski Dudi Lester Ilia Taraban Chendi.Xue Michał Kuligowski Jakub Maksymczuk Tomasz Zielinski Sun Choi Iryna Boiko Bob Zhu hlin99 Zehao Huang Andrzej Kotłowski Yan Tomsinsky Nir David Yu-Zhou Ruheena Suhani Shaik Karol Damaszke Marcin Swiniarski Woosuk Kwon Jacek Czaja Jacek Czaja Yuan
2024-11-06 10:09:10 +01:00
a5fda50a10 [CI/Build] Fix large_gpu_mark reason (#10070) Isotr0py 2024-11-06 16:50:37 +08:00
21063c11c7 [CI/Build] drop support for Python 3.8 EOL (#8464) Aaron Pham 2024-11-06 02:11:55 -05:00
4be3a45158 [distributed] add function to create ipc buffers directly (#10064) youkaichao 2024-11-05 22:35:03 -08:00
4089985552 [V1] Integrate Piecewise CUDA graphs (#10058) Woosuk Kwon 2024-11-05 22:16:04 -08:00
9d59b75593 [Bugfix] Remove CustomChatCompletionContentPartParam multimodal input type (#10054) zifeitong 2024-11-05 21:13:09 -08:00
ea928f608c [Bugfix] Gpt-j-6B patch kv_scale to k_scale path (#10063) arakowsk-amd 2024-11-05 21:10:40 -08:00
2bcbae704c [Bugfix] Fix edge-case crash when using chat with the Mistral Tekken Tokenizer (#10051) Travis Johnson 2024-11-05 21:28:29 -07:00
ffc0f2b47a [Model][OpenVINO] Fix regressions from #8346 (#10045) Peter Salas 2024-11-05 20:19:15 -08:00
82bfc38d07 [Misc] Sort the list of embedding models (#10037) Cyrus Leung 2024-11-06 12:05:05 +08:00
c4cacbaa7f [v1] reduce graph capture time for piecewise cudagraph (#10059) youkaichao 2024-11-05 18:19:50 -08:00
0c63c34f72 [Bugfix][SpecDecode] kv corruption with bonus tokens in spec decode (#9730) Sungjae LeeandLiuXiaoxuanPKU 2024-11-06 10:45:45 +09:00
966e31697b [Bugfix] Fix pickle of input when async output processing is on (#9931) Wallas Henrique 2024-11-05 21:39:26 -03:00
43300bd98a [Bugfix] Properly propagate trust_remote_code settings (#10047) zifeitong 2024-11-05 16:34:40 -08:00
ca9844b340 [bugfix] fix weak ref in piecewise cudagraph and tractable test (#10048) youkaichao 2024-11-05 14:49:20 -08:00
235366fe2e [CI] Prune back the number of tests in tests/kernels/* (#9932) Michael Goin 2024-11-05 16:02:32 -05:00
02462465ea [CI] Prune tests/models/decoder_only/language/* tests (#9940) Michael Goin 2024-11-05 16:02:23 -05:00
b9c64c0ca7 [Misc] Modify BNB parameter name (#9997) Jee Jee Li 2024-11-06 03:40:08 +08:00
d2e80332a7 [Feature] Update benchmark_throughput.py to support image input (#9851) lkchenandLinkun Chen 2024-11-05 11:30:02 -08:00
a53046b16f [Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921) Michael Goin 2024-11-05 13:42:20 -05:00
731aec5be7 [CI/Build] Limit github CI jobs based on files changed (#9928) Russell Bryant 2024-11-05 13:30:42 -05:00
09d3550372 [Misc] Add logging for CUDA memory (#10027)
Chenghao (Alan) Yang Chenghao Yang youkaichao
2024-11-05 11:50:50 -06:00
cd34029e91 Refactor TPU requirements file and pin build dependencies (#10010) Richard Liu 2024-11-05 08:48:44 -08:00
5952d81139 [Frontend] Fix tcp port reservation for api server (#10012) Russell Bryant 2024-11-05 10:50:57 -05:00
93dee88f6b [Misc] vllm CLI flags should be ordered for better user readability (#10017) Chauncey 2024-11-05 18:59:56 +08:00
7a83b1aec0 [BugFix] Lazy import ray (#10021) Gene Der Su 2024-11-05 02:04:10 -08:00
ad23318928 [Bugfix] Fixup Mamba (#10004) Tyler Michael Smith 2024-11-04 22:46:38 -05:00
bbc3619dc8 [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) Cyrus Leung 2024-11-05 10:07:31 +08:00
04bbf38e05 [Core] Use os.sched_yield in ShmRingBuffer instead of time.sleep (#9994) Tyler Michael Smith 2024-11-04 20:08:21 -05:00
8f0a9ca890 [Bugfix] Respect modules_to_not_convert within awq_marlin (#9895) Michael Goin 2024-11-04 18:57:44 -05:00
2094062b4e [4.5/N] bugfix for quant config in speculative decode (#10007) youkaichao 2024-11-04 15:11:59 -08:00
d93478b399 [Bugfix] Upgrade to pytorch 2.5.1 (#10001) bnellnm 2024-11-04 18:11:28 -05:00
ac04a97a9f [Frontend] Add max_tokens prometheus metric (#9881) tomeras91 2024-11-05 00:53:24 +02:00
9a5664d4a4 [Misc] Refactor benchmark_throughput.py (#9779)
lkchen Linkun Chen Linkun Chen
2024-11-04 14:32:16 -08:00
04cef2c6ab [Bugfix] Fix MQLLMEngine hanging (#9973) Robert Shaw 2024-11-04 16:01:43 -05:00
6e056bcf04 [Doc] Update VLM doc about loading from local files (#9999) Roger Wang 2024-11-04 11:47:11 -08:00
5208dc7a20 [Bugfix][CI/Build][Hardware][AMD] Shard ID parameters in AMD tests running parallel jobs (#9279) hissu-hyvarinen 2024-11-04 21:37:46 +02:00
1c45f4c385 [CI] Basic Integration Test For TPU (#9968) Robert Shaw 2024-11-04 14:34:26 -05:00
603a661ae8 [Model] factoring out MambaMixer out of Jamba (#8993) Mor Zusman 2024-11-04 20:00:00 +02:00
fb2716d641 [Misc]Reduce BNB static variable (#9987) Jee Jee Li 2024-11-05 01:04:40 +08:00
8d72bb20fa [4/N] make quant config first-class citizen (#9978) youkaichao 2024-11-04 08:51:31 -08:00
ac6b8f19b9 [Frontend] Multi-Modality Support for Loading Local Image Files (#9915) Chauncey 2024-11-04 23:34:57 +08:00
ccb5376a9a [Bugfix][OpenVINO] Fix circular reference #9939 (#9974) Mengqing Cao 2024-11-04 18:14:13 +08:00
ea4adeddc1 [Bugfix] Fix E2EL mean and median stats (#9984) Tran Quang Dai 2024-11-04 16:37:58 +07:00
4dbcbbeb09 [Misc] Compute query_start_loc/seq_start_loc on CPU (#9447) Yang ZhengandYang Zheng 2024-11-04 16:54:37 +08:00
b67feb1274 [Bugfix]Using the correct type hints (#9885) Gregory Shtrasberg 2024-11-04 01:19:51 -05:00
c49f0407ba [Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917) Jee Jee Li 2024-11-04 11:36:41 +08:00
91c9ebbb1b [V1] Fix Configs (#9971) Robert Shaw 2024-11-03 19:24:40 -05:00
54597724f4 [Model] Add support for H2OVL-Mississippi models (#9747) shanshan wangandRoger Wang 2024-11-03 18:15:36 -06:00
1f1b6d6eda [V1] Support per-request seed (#9945) Nick Hill 2024-11-03 17:14:17 +00:00
3bb4befea7 [bugfix] fix tsts (#9959) youkaichao 2024-11-02 15:54:05 -07:00
ae5279a163 [torch.compile] Adding torch compile to vision-language models (#9946) Yongzao 2024-11-03 03:56:05 +08:00
1b73ab2a1f [CI/Build] Quoting around > (#9956) Nikita Furin 2024-11-02 22:50:28 +03:00
cea808f325 [3/N] model runner pass the whole config to model (#9958) youkaichao 2024-11-02 12:08:49 -07:00
74b529ceee [bugfix] fix chatglm dummy_data_for_glmv (#9955) youkaichao 2024-11-02 08:03:33 -07:00
d6459b4516 [V1] Fix EngineArgs refactor on V1 (#9954) Robert Shaw 2024-11-02 10:44:38 -04:00
e893795443 [2/N] executor pass the complete config to worker/modelrunner (#9938) youkaichaoandNick Hill 2024-11-02 07:35:05 -07:00
1d4cfe2be1 [Doc] Updated tpu-installation.rst with more details (#9926) Michael Green 2024-11-02 14:06:45 +00:00
eed92f12fc [Docs] Update Granite 3.0 models in supported models table (#9930) Nick HillandCyrus Leung 2024-11-02 09:02:18 +00:00
af7380d83b [torch.compile] fix cpu broken code (#9947) youkaichao 2024-11-01 23:35:47 -07:00
a78dd3303e [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) sroy745 2024-11-01 23:22:49 -07:00
d522034c85 [ci/build] Have dependabot ignore pinned dependencies (#9935) Kevin H. Luu 2024-11-01 13:56:13 -10:00
6c0b7f548d [Core][VLM] Add precise multi-modal placeholder tracking (#8346) Peter Salas 2024-11-01 16:21:10 -07:00
d151fde834 [ci/build] Bump the patch-update group with 10 updates (#9897) dependabot[bot]andKevin H. Luu 2024-11-01 23:04:42 +00:00
27cd36e6e2 [Bugfix] PicklingError on RayTaskError (#9934) Gene Der Su 2024-11-01 15:08:23 -07:00
18bd7587b7 [1/N] pass the complete config from engine to executor (#9933) youkaichao 2024-11-01 13:51:57 -07:00
598b6d7b07 [Bugfix/Core] Flashinfer k_scale and v_scale (#9861) Pavani Majety 2024-11-01 12:15:05 -07:00
aff1fd8188 [torch.compile] use interpreter with stable api from pytorch (#9889) youkaichao 2024-11-01 11:50:37 -07:00
4581d2cc02 [Core] Refactor: Clean up unused argument in Scheduler._preempt (#9696) André Jonasson 2024-11-01 19:41:38 +01:00
1dd4cb2935 [Bugfix] Fix edge cases for MistralTokenizer (#9625)
Travis Johnson Prashant Gupta Patrick von Platen
2024-11-01 11:33:15 -06:00
ba0d892074 [Frontend] Use a proper chat template for VLM2Vec (#9912) Cyrus Leung 2024-11-01 22:09:07 +08:00
30a2e80742 [CI/Build] Add Model Tests for PixtralHF (#9813) Michael Goin 2024-11-01 09:55:29 -04:00
06386a64dd [Frontend] Chat-based Embeddings API (#9759) Cyrus Leung 2024-11-01 16:13:35 +08:00
d3aa2a8b2f [Doc] Update multi-input support (#9906) Cyrus Leung 2024-11-01 15:34:49 +08:00
2b5bf20988 [torch.compile] Adding torch compile annotations to some models (#9876) Yongzaoandyoukaichao 2024-11-01 15:25:47 +08:00
93a76dd21d [Model] Support bitsandbytes for MiniCPMV (#9891) Michael Goin 2024-11-01 01:31:56 -04:00
566cd27797 [torch.compile] rework test plans (#9866) youkaichao 2024-10-31 22:20:17 -07:00
37a4947dcd [Bugfix] Fix layer skip logic with bitsandbytes (#9887) Michael Goin 2024-11-01 01:12:44 -04:00
96e0c9cbbd [torch.compile] directly register custom op (#9896) youkaichao 2024-10-31 21:56:09 -07:00
031a7995f3 [Bugfix][Frontend] Reject guided decoding in multistep mode (#9892) Joe Runde 2024-10-31 19:09:46 -06:00
b63c64d95b [ci/build] Configure dependabot to update pip dependencies (#9811) Kevin H. Luu 2024-10-31 12:55:38 -10:00
9fb12f7848 [BugFix][Kernel] Fix Illegal memory access in causal_conv1d in H100 (#9838) Mor Zusman 2024-10-31 22:06:25 +02:00
55650c83a0 [Bugfix] Fix illegal memory access error with chunked prefill, prefix caching, block manager v2 and xformers enabled together (#9532) sasha0552 2024-10-31 18:46:36 +00:00
77f7ef2908 [CI/Build] Adding a forced docker system prune to clean up space (#9849) Alexei-V-Ivanov-AMD 2024-10-31 12:02:58 -05:00
16b8f7a86f [CI/Build] Add Model Tests for Qwen2-VL (#9846)
Alex Brooks Cyrus Leung DarkLight1337
2024-10-31 10:10:52 -06:00
5608e611c2 [Doc] Update Qwen documentation (#9869) Jee Jee Li 2024-10-31 16:54:18 +08:00
3ea2dc2ec4 [Misc] Remove deprecated arg for cuda graph capture (#9864) Roger Wang 2024-10-31 00:22:07 -07:00
d087bf863e [Model] Support quantization of Qwen2VisionTransformer (#9817) Michael Goin 2024-10-31 01:41:20 -04:00
890ca36072 Revert "[Bugfix] Use host argument to bind to interface (#9798)" (#9852) Kevin H. Luu 2024-10-30 15:44:51 -10:00
abbfb6134d [Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837) Guillaume Calmettes 2024-10-31 02:15:56 +01:00
64384bbcdf [torch.compile] upgrade tests (#9858) youkaichao 2024-10-30 16:34:22 -07:00
00d91c8a2c [CI/Build] Simplify exception trace in api server tests (#9787) Yongzaoandyoukaichao 2024-10-31 05:52:05 +08:00
c2cd1a2142 [doc] update pp support (#9853) youkaichao 2024-10-30 13:36:51 -07:00

... 16 17 18 19 20 ...

Commit Graph Select branches Hide Pull Requests main Mono Color

Commit Graph

Select branches

Hide Pull Requests

main