Commit Graph - vllm - Gitea: Git with a cup of tea

wassname/vllm

Fork 0

mirror of https://github.com/wassname/vllm.git synced 2026-07-18 12:50:17 +08:00

61af633256 [BUGFIX] Fix UnspecifiedPlatform package name (#11916) Kunshang JiandGitHub 2025-01-10 16:20:46 +08:00
ac2f3f7fee [Bugfix] Validate lora adapters to avoid crashing server (#11727)
Joe Runde GitHub Jee Jee Li
2025-01-10 00:56:36 -07:00
cf5f000d21 [torch.compile] Hide KV cache behind torch.compile boundary (#11677) Chen ZhangandGitHub 2025-01-10 13:14:42 +08:00
3de2b1eafb [Doc] Show default pooling method in a table (#11904) Cyrus LeungandGitHub 2025-01-10 11:25:20 +08:00
b844b99ad3 [VLM] Enable tokenized inputs for merged multi-modal processor (#11900) Cyrus LeungandGitHub 2025-01-10 11:24:00 +08:00
c3cf54dda4 [Doc][5/N] Move Community and API Reference to the bottom (#11896)
Cyrus Leung GitHub Simon Mo
2025-01-10 11:10:12 +08:00
36f5303578 [Docs] Add Modal to deployment frameworks (#11907) Charles FryeandGitHub 2025-01-09 15:26:37 -08:00
9a228348d2 [Misc] Provide correct Pixtral-HF chat template (#11891) Cyrus LeungandGitHub 2025-01-10 01:19:37 +08:00
bd82872211 [ci]try to fix flaky multi-step tests (#11894) youkaichaoandGitHub 2025-01-09 22:47:29 +08:00
405eb8e396 [platform] Allow platform specify attention backend (#11609)
wangxiyuan GitHub Mengqing Cao
2025-01-09 21:46:50 +08:00
65097ca0af [Doc] Add model development API Reference (#11884) Cyrus LeungandGitHub 2025-01-09 17:43:40 +08:00
1d967acb45 [Bugfix] fix beam search input errors and latency benchmark script (#11875)
Ye (Charlotte) Qi GitHub yeq
2025-01-09 01:36:39 -08:00
0bd1ff4346 [Bugfix] Override dunder methods of placeholder modules (#11882) Cyrus LeungandGitHub 2025-01-09 17:02:53 +08:00
310aca88c9 [perf]fix current stream (#11870) youkaichaoandGitHub 2025-01-09 15:18:21 +08:00
a732900efc [Doc] Intended links Python multiprocessing library (#11878) Guspan TanadiandGitHub 2025-01-09 12:39:39 +07:00
d848800e88 [Misc] Move print_*_once from utils to logger (#11298)
Cyrus Leung GitHub Maxime Fournioux
2025-01-09 12:48:12 +08:00
730e9592e9 [Doc] Recommend uv and python 3.12 for quickstart guide (#11849) Michael GoinandGitHub 2025-01-08 22:37:48 -05:00
1fe554bac3 treat do_lower_case in the same way as the sentence-transformers library (#11815) Maximilien de BayserandGitHub 2025-01-09 00:05:43 -03:00
615e4a5401 [CI] Turn on basic correctness tests for V1 (#10864) Tyler Michael SmithandGitHub 2025-01-08 21:20:44 -05:00
3db0cafdf1 [Docs] Add Google Cloud Meetup (#11864) Simon MoandGitHub 2025-01-08 12:38:28 -08:00
526de822d5 [Kernel][Triton][AMD] Use block size heuristic for avg 2.8x speedup for int8 models (#11698) rasmithandGitHub 2025-01-08 14:23:15 -06:00
56fe4c297c [TPU][Quantization] TPU W8A8 (#11785)
Robert Shaw GitHub Woosuk Kwon
2025-01-08 14:33:29 -05:00
47de8821d3 [Misc]add some explanations for BlockHashType (#11847) WangErXiaoandGitHub 2025-01-09 02:21:30 +08:00
5984499e47 [Doc] Expand Multimodal API Reference (#11852) Cyrus LeungandGitHub 2025-01-09 01:14:14 +08:00
ca47e176af [Misc] Move some model utils into vision file (#11848) Cyrus LeungandGitHub 2025-01-09 01:04:46 +08:00
78f4590b60 [Bugfix][XPU] fix silu_and_mul (#11823) Yan MaandGitHub 2025-01-09 00:11:50 +08:00
2f7024987e [CI/Build][Bugfix] Fix CPU CI image clean up (#11836) Li, JiangandGitHub 2025-01-08 23:18:28 +08:00
6cd40a5bfe [Doc][4/N] Reorganize API Reference (#11843) Cyrus LeungandGitHub 2025-01-08 21:34:44 +08:00
aba8d6ee00 [Doc] Move examples into categories (#11840) Harry MellorandGitHub 2025-01-08 13:09:53 +00:00
2a0596bc48 [VLM] Reorganize profiling/processing-related code (#11812) Cyrus LeungandGitHub 2025-01-08 18:59:58 +08:00
f12141170a [torch.compile] consider relevant code in compilation cache (#11614) youkaichaoandGitHub 2025-01-08 18:46:43 +08:00
cfd3219f58 [Hardware][Apple] Native support for macOS Apple Silicon (#11696)
Wallas Henrique GitHub Michael Goin
2025-01-08 05:35:49 -03:00
a1b2b8606e [Docs] Update sponsor name: 'Novita' to 'Novita AI' (#11833) Simon MoandGitHub 2025-01-07 23:05:46 -08:00
ad9f1aa679 [doc] update wheels url (#11830) youkaichaoandGitHub 2025-01-08 14:36:49 +08:00
889e662eae [misc] improve memory profiling (#11809)
youkaichao GitHub Cyrus Leung
2025-01-08 14:36:03 +08:00
ef68eb28d8 [Bug] Fix pickling of ModelConfig when RunAI Model Streamer is used (#11825) Cyrus LeungandGitHub 2025-01-08 13:40:09 +08:00
259abd8953 [Docs] reorganize sponsorship page (#11639) Simon MoandGitHub 2025-01-07 21:16:08 -08:00
f645eb6954 [Bugfix] Add checks for LoRA and CPU offload (#11810) Jee Jee LiandGitHub 2025-01-08 13:08:48 +08:00
f4923cb8bc [OpenVINO] Fixed Docker.openvino build (#11732) Ilya LavrenovandGitHub 2025-01-08 09:08:30 +04:00
b640b19cc0 Fixed docker build for ppc64le (#11518) NishidhaandGitHub 2025-01-08 10:35:37 +05:30
dc71af0a71 Remove the duplicate imports of MultiModalKwargs and PlaceholderRange… (#11824) WangErXiaoandGitHub 2025-01-08 12:09:25 +08:00
4d29e91be8 [Misc] sort torch profiler table by kernel timing (#11813) Divakar VermaandGitHub 2025-01-07 20:57:04 -06:00
91445c7bc8 [Bugfix] Fix image input for Pixtral-HF (#11741) Cyrus LeungandGitHub 2025-01-08 10:17:16 +08:00
5950f555a1 [Doc] Group examples into categories (#11782) Harry MellorandGitHub 2025-01-08 01:20:12 +00:00
a4e2b26856 [Bugfix] Significant performance drop on CPUs with --num-scheduler-steps > 1 (#11794) Jie Fu (傅杰)andGitHub 2025-01-08 08:15:50 +08:00
973f5dc581 [Doc]Add documentation for using EAGLE in vLLM (#11417) sroy745andGitHub 2025-01-07 11:19:12 -08:00
c994223d56 [Bugfix] update the prefix for qwen2 (#11795)
jiangjiadi GitHub jiadi.jjd
2025-01-08 02:36:34 +08:00
869579a702 [optimization] remove python function call for custom op (#11750) youkaichaoandGitHub 2025-01-08 01:04:28 +08:00
c0efe92d8b [Doc] Add note to gte-Qwen2 models (#11808) Cyrus LeungandGitHub 2025-01-07 21:50:58 +08:00
d9fa1c05ad [doc] update how pip can install nightly wheels (#11806) youkaichaoandGitHub 2025-01-07 21:42:58 +08:00
2de197bdd4 [V1] Support audio language models on V1 (#11733) Roger WangandGitHub 2025-01-07 03:47:36 -08:00
869e829b85 [doc] add doc to explain how to use uv (#11773)
youkaichao GitHub Cyrus Leung
2025-01-07 18:41:17 +08:00
8f37be38eb [Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calculation (#11800) Cyrus LeungandGitHub 2025-01-07 18:25:02 +08:00
8082ad7950 [V1][Doc] Update V1 support for LLaVa-NeXT-Video (#11798) Roger WangandGitHub 2025-01-07 01:55:39 -08:00
1e4ce295ae [CI][CPU] adding build number to docker image name (#11788) YuanandGitHub 2025-01-07 15:28:01 +08:00
ce1917fcf2 [Doc] Create a vulnerability management team (#9925) Russell BryantandGitHub 2025-01-07 01:57:32 -05:00
e512f76a89 fix init error for MessageQueue when n_local_reader is zero (#11768) XiaobingZhangandGitHub 2025-01-07 14:12:48 +08:00
898cdf033e [CI] Fix neuron CI and run offline tests (#11779) Liangfu ChenandGitHub 2025-01-06 21:36:10 -08:00
0f3f3c86ec [Bugfix] Update attention interface in Whisper (#11784) Roger WangandGitHub 2025-01-06 20:36:24 -08:00
b278557935 [Kernel][LoRA]Punica prefill kernels fusion (#11234)
Jee Jee Li GitHub Zhonghua Deng
2025-01-07 12:01:39 +08:00
8ceffbf315 [Doc][3/N] Reorganize Serving section (#11766) Cyrus LeungandGitHub 2025-01-07 11:20:01 +08:00
d93d2d74fd [XPU] Make pp group initilized for pipeline-parallelism (#11648) YiSheng5andGitHub 2025-01-07 11:09:58 +08:00
d0169e1b0f [Model] Future-proof Qwen2-Audio multi-modal processor (#11776) Cyrus LeungandGitHub 2025-01-07 11:05:17 +08:00
08fb75c72e [Bugfix] Fix LLaVA-NeXT feature size precision error (for real) (#11772) Cyrus LeungandGitHub 2025-01-07 09:10:54 +08:00
91b361ae89 [V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (#11685)
Roger Wang GitHub DarkLight1337
2025-01-06 11:58:16 -08:00
e20c92bb61 [Kernel] Move attn_type to Attention.__init__() (#11690) Chen ZhangandGitHub 2025-01-07 00:11:28 +08:00
32c9eff2ff [Bugfix][V1] Fix molmo text-only inputs (#11676) Jee Jee LiandGitHub 2025-01-06 23:22:25 +08:00
4ca5d40adc [doc] explain how to add interleaving sliding window support (#11771) youkaichaoandGitHub 2025-01-06 21:57:44 +08:00
9279b9f83d [Bugfix] Fix max image size for LLaVA-Onevision (#11769) Roger WangandGitHub 2025-01-06 05:48:53 -08:00
ee77fdb5de [Doc][2/N] Reorganize Models and Usage sections (#11755) Cyrus LeungandGitHub 2025-01-06 21:40:31 +08:00
996357e480 [VLM] Separate out profiling-related logic (#11746) Cyrus LeungandGitHub 2025-01-06 16:02:21 +08:00
2a622d704a k8s-config: Update the secret to use stringData (#11679) Suraj DeshmukhandGitHub 2025-01-06 00:01:22 -08:00
9c749713f6 [mypy] Forward pass function type hints in lora (#11740)
Lucas Tucker GitHub lucast2021
2025-01-06 01:59:36 -06:00
022c5c6944 [V1] Refactor get_executor_cls (#11754) Rui QiaoandGitHub 2025-01-05 23:59:16 -08:00
f8fcca100b [Misc] Fix typo for valid_tool_parses (#11753) Rui QiaoandGitHub 2025-01-05 23:12:38 -08:00
06bfb51963 [V1] Add BlockTable class (#11693) Woosuk KwonandGitHub 2025-01-06 14:24:42 +09:00
408e560015 [Bugfix] Remove block size constraint (#11723) Cody YuandGitHub 2025-01-05 20:49:55 -08:00
402d378360 [Doc] [1/N] Reorganize Getting Started section (#11645) Cyrus LeungandGitHub 2025-01-06 10:18:33 +08:00
9e764e7b10 [distributed] remove pynccl's redundant change_state (#11749) cennnandGitHub 2025-01-06 09:05:48 +08:00
33fc1e2e86 [Frontend] Improve StreamingResponse Exception Handling (#11752) Robert ShawandGitHub 2025-01-05 16:35:01 -05:00
eba17173d3 fix: [doc] fix typo (#11751)
Lancer GitHub Lancer
2025-01-06 00:48:16 +08:00
635b897246 [distributed] remove pynccl's redundant stream (#11744) cennnandGitHub 2025-01-05 23:09:11 +08:00
4068f4b5b5 [MISC] Replace c10::optional with std::optional (#11730) Lu FangandGitHub 2025-01-04 17:20:34 -08:00
47831430cc [Bugfix][V1] Fix test_kv_cache_utils.py (#11738) Jee Jee LiandGitHub 2025-01-05 00:07:59 +08:00
65c08928c2 [Model] Remove unnecessary weight initialization logic (#11736)
Cyrus Leung GitHub Isotr0py
2025-01-04 23:46:21 +08:00
ba214dffbe [Bugfix] Fix precision error in LLaVA-NeXT (#11735) Cyrus LeungandGitHub 2025-01-04 23:45:57 +08:00
eed11ebee9 [VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-OneVision (#11717) Cyrus LeungandGitHub 2025-01-04 19:40:53 +08:00
300acb8347 [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233) Yan BurmanandGitHub 2025-01-04 08:50:16 +02:00
d91457d529 [V1] Add kv cache utils tests. (#11513) xcnickandGitHub 2025-01-04 14:49:46 +08:00
fbf2564554 [V1] Add RayExecutor support for AsyncLLM (api server) (#11712) Kunshang JiandGitHub 2025-01-04 14:41:31 +08:00
d1d49397e7 Update bnb.md with example for OpenAI (#11718) Alberto FerrerandGitHub 2025-01-04 00:29:02 -06:00
9c93636d84 Update tool_calling.md (#11701) Hust_YangXianandGitHub 2025-01-04 14:16:30 +08:00
e5d7ed0c53 [V1] log GPU blocks num for MultiprocExecutor (#11656) WangErXiaoandGitHub 2025-01-04 08:13:12 +08:00
ad0d567e1c [V1] Chore: cruft removal (#11724) Robert ShawandGitHub 2025-01-03 18:25:02 -05:00
bf0d97d786 Update requirements-tpu.txt to support python 3.9 and 3.11 (#11695) Michael GoinandGitHub 2025-01-03 17:36:46 -05:00
a655eb3025 [Misc]Add BNB quantization for Qwen2VL (#11719)
Jee Jee Li GitHub Isotr0py
2025-01-04 06:19:02 +08:00
1543914c04 [V1] Improve TP>1 Error Handling + Stack Trace (#11721)
Robert Shaw GitHub Tyler Michael Smith
2025-01-03 16:29:11 -05:00
61fed92c7e [Bugfix] Fix ColumnParallelLinearWithLoRA slice (#11708) ZincCatandGitHub 2025-01-03 13:02:34 -08:00
80c751e7f6 [V1] Simplify Shutdown (#11659) Robert ShawandGitHub 2025-01-03 12:25:38 -05:00
e1a5c2f0a1 [Model] Whisper model implementation (#11280)
Aurick Qiao GitHub Aurick Qiao
2025-01-03 03:39:19 -05:00

... 8 9 10 11 12 ...

Commit Graph Select branches Hide Pull Requests main Mono Color

Commit Graph

Select branches

Hide Pull Requests

main