wassname/vllm
1
0
Fork 0
mirror of https://github.com/wassname/vllm.git synced 2026-07-21 12:50:05 +08:00
Code Issues Packages Projects Releases Wiki Activity
83 Commits 1 Branch 0 Tags
cfae35b861c5fc0c9f3689f99c7aba2e4501beb8
Commit Graph
7 Commits
Author SHA1 Message Date
Woosuk KwonandGitHub cfae35b861 Add miscellaneous updates (#8) 2023-03-13 13:48:38 -07:00
Woosuk KwonandGitHub e9d3f2ff77 Add memory analyzer & utomatically configure KV cache size (#6) 2023-03-11 23:23:14 -08:00
Woosuk KwonandGitHub 1a7eb7da61 Support beam search & parallel generation (#7) 2023-03-10 09:58:21 -08:00
Woosuk KwonandGitHub 04e5acc08e Fix a bug in 1D input shape (#5) 2023-03-06 10:05:27 -08:00
Woosuk KwonandGitHub 3e9f991d6a Use FlashAttention for multi_query_kv_attention (#4) 2023-03-01 21:13:08 -08:00
Woosuk Kwon fa16389a2e Clean up the server script 2023-02-24 11:56:21 +00:00
Woosuk Kwon afdbe5d373 [WIP] Add server script 2023-02-24 01:33:37 +00:00