Commit Graph - vllm - Gitea: Git with a cup of tea

wassname/vllm

Fork 0

mirror of https://github.com/wassname/vllm.git synced 2026-07-15 11:28:07 +08:00

f8c688d746 [Minor] Add Phi 2 to supported models (#2159) Woosuk KwonandGitHub 2023-12-17 02:54:57 -08:00
c9fadda543 [Minor] Fix xformers version (#2158) Woosuk KwonandGitHub 2023-12-17 02:28:02 -08:00
30fb0956df [Minor] Add more detailed explanation on quantization argument (#2145) Woosuk KwonandGitHub 2023-12-17 01:56:16 -08:00
3a765bd5e1 Temporarily enforce eager mode for GPTQ models (#2154) Woosuk KwonandGitHub 2023-12-17 01:51:12 -08:00
26c52a5ea6 [Docs] Add CUDA graph support to docs (#2148) Woosuk KwonandGitHub 2023-12-17 01:49:20 -08:00
c3372e87be Remove dependency on CuPy (#2152) Woosuk KwonandGitHub 2023-12-17 01:49:07 -08:00
b0a1d667b0 Pin PyTorch & xformers versions (#2155) Woosuk KwonandGitHub 2023-12-17 01:46:54 -08:00
e1d5402238 Fix all-reduce memory usage (#2151) Woosuk KwonandGitHub 2023-12-17 01:44:45 -08:00
3d1cfbfc74 [Minor] Delete Llama tokenizer warnings (#2146) Woosuk KwonandGitHub 2023-12-16 22:05:18 -08:00
37ca558103 Optimize model execution with CUDA graph (#1926)
Woosuk Kwon GitHub Chen Shen Antoni Baum
2023-12-16 21:12:08 -08:00
eed74a558f Simplify weight loading logic (#2133) RoyandGitHub 2023-12-17 04:41:23 +08:00
2acd76f346 [ROCm] Temporarily remove GPTQ ROCm support (#2138) Woosuk KwonandGitHub 2023-12-15 17:13:58 -08:00
b81a6a6bb3 [Docs] Add supported quantization methods to docs (#2135) Woosuk KwonandGitHub 2023-12-15 13:29:22 -08:00
0fbfc4b81b Add GPTQ support (#916) CHU TianxiangandGitHub 2023-12-15 19:04:22 +08:00
c06170cc8e Add a flag to include stop string in output text (#1976) Yunfeng BaiandGitHub 2023-12-15 00:45:58 -08:00
614856da25 Avoid multiple redefinition (#1817) Mingcan XiangandGitHub 2023-12-14 12:35:58 -05:00
05bdf4eaf3 Fix Dockerfile.rocm (#2101)
TJian GitHub miloice
2023-12-14 16:45:58 +08:00
6774bd50b0 Fix typing in AsyncLLMEngine & add toml to requirements-dev (#2100) mezuzzaandGitHub 2023-12-14 03:19:41 -05:00
31c1f3255e Bump up to v0.2.5 (#2095) Woosuk KwonandGitHub 2023-12-13 23:56:15 -08:00
21d93c140d Optimize Mixtral with expert parallelism (#2090) Antoni BaumandGitHub 2023-12-13 23:55:07 -08:00
f1c8520146 [BugFix] Fix input positions for long context with sliding window (#2088) Woosuk KwonandGitHub 2023-12-13 12:28:13 -08:00
096827c284 [Docs] Add notes on ROCm-supported models (#2087) Woosuk KwonandGitHub 2023-12-13 09:45:34 -08:00
6565d9e33e Update installation instruction for vLLM + CUDA 11.8 (#2086) Woosuk KwonandGitHub 2023-12-13 09:25:59 -08:00
f375ec8440 [ROCm] Upgrade xformers version for ROCm & update doc (#2079)
TJian GitHub miloice
2023-12-13 16:56:05 +08:00
518369d78c Implement lazy model loader (#2044) Woosuk KwonandGitHub 2023-12-12 22:21:45 -08:00
30bad5c492 Fix peak memory profiling (#2031) Woosuk KwonandGitHub 2023-12-12 22:01:53 -08:00
3fefe271ec Update Dockerfile to build Megablocks (#2042) Simon MoandGitHub 2023-12-12 17:34:17 -08:00
6428f1d051 Support MPT with GQA (#1938)
Megha Agarwal GitHub Woosuk Kwon
2023-12-12 10:16:05 -08:00
7e1b21daac Remove einops from requirements (#2049) Woosuk KwonandGitHub 2023-12-12 09:34:09 -08:00
cb3f30c600 Upgrade transformers version to 4.36.0 (#2046) Woosuk KwonandGitHub 2023-12-11 18:39:14 -08:00
f3e024bece [CI/CD] Upgrade PyTorch version to v2.1.1 (#2045) Woosuk KwonandGitHub 2023-12-11 17:48:11 -08:00
31d2ab4aff Remove python 3.10 requirement (#2040) Woosuk KwonandGitHub 2023-12-11 12:26:42 -08:00
eb17212858 Update Dockerfile to support Mixtral (#2027) Simon MoandGitHub 2023-12-11 11:59:08 -08:00
4dd4b5c538 Bump up to v0.2.4 (#2034) Woosuk KwonandGitHub 2023-12-11 11:49:39 -08:00
6120e5aaea Fix import error msg for megablocks (#2038) Woosuk KwonandGitHub 2023-12-11 11:40:56 -08:00
2eaa81b236 Update README.md to add megablocks requirement for mixtral (#2033) RamandGitHub 2023-12-12 01:07:34 +05:30
81ce2a4b26 [Minor] Fix type annotation in Mixtral (#2036) Woosuk KwonandGitHub 2023-12-11 11:32:39 -08:00
5dd80d3777 Fix latency benchmark script (#2035) Woosuk KwonandGitHub 2023-12-11 11:19:08 -08:00
beeee69bc9 Revert adding Megablocks (#2030) Woosuk KwonandGitHub 2023-12-11 10:49:00 -08:00
9bf28d0b69 Update requirements.txt for mixtral (#2029) RamandGitHub 2023-12-12 00:09:29 +05:30
c0ce15dfb2 Update run_on_sky.rst (#2025) Ikko Eltociear AshimineandGitHub 2023-12-12 03:32:58 +09:00
b9bcdc7158 Change the load format to pt for Mixtral (#2028) Woosuk KwonandGitHub 2023-12-11 10:32:17 -08:00
4ff0203987 Minor fixes for Mixtral (#2015) Woosuk KwonandGitHub 2023-12-11 09:16:15 -08:00
b5f882cc98 Mixtral 8x7B support (#2011)
Pierre Stock GitHub Pierre Stock Zhuohan Li
2023-12-11 10:09:15 +01:00
2e8fc0d4c3 Fix completion API echo and logprob combo (#1992) Simon MoandGitHub 2023-12-10 13:20:30 -08:00
dacaf5a400 Replace head_mapping params with num_kv_heads to attention kernel. (#1997)
wbn GitHub wangguoya Yang Zhao
2023-12-11 02:12:53 +08:00
24cde76a15 [Minor] Add comment on skipping rope caches (#2004) Woosuk KwonandGitHub 2023-12-10 10:04:12 -08:00
1aa1361510 Fix OpenAI server completion_tokens referenced before assignment (#1996) Jin ShangandGitHub 2023-12-10 13:01:21 +08:00
fe470ae5ad [Minor] Fix code style for baichuan (#2003) Woosuk KwonandGitHub 2023-12-09 19:24:29 -08:00
3a8c2381f7 Fix for KeyError on Loading LLaMA (#1978) Jun GaoandGitHub 2023-12-10 07:59:57 +08:00
c85b80c2b6 [Docker] Add cuda arch list as build option (#1950) Simon MoandGitHub 2023-12-08 09:53:47 -08:00
2b981012a6 Fix Baichuan2-7B-Chat (#1987) firebookandGitHub 2023-12-09 01:38:36 +08:00
6ccc0bfffb Merge EmbeddedLLM/vllm-rocm into vLLM main (#1836)
TJian GitHub Philipp Moritz Amir Balwel root tjtanaa kuanfu miloice
2023-12-08 15:16:52 +08:00
c8e7eb1eb3 fix typo in getenv call (#1972) Daya KhudiaandGitHub 2023-12-07 16:04:41 -08:00
24f60a54f4 [Docker] Adding number of nvcc_threads during build as envar (#1893) AguirreNicolasandGitHub 2023-12-07 16:00:32 -03:00
42c02f5892 Fix quickstart.rst typo jinja (#1964) gottlikeandGitHub 2023-12-07 17:34:44 +01:00
ebede26ebf Make InternLM follow rope_scaling in config.json (#1956)
Jie Li GitHub lijie8
2023-12-08 00:32:08 +08:00
d940ce497e Fix typo in adding_model.rst (#1947) Peter GötzandGitHub 2023-12-06 19:04:26 +01:00
05ff90b692 Save pytorch profiler output for latency benchmark (#1871) Antoni BaumandGitHub 2023-12-05 20:55:55 -08:00
1d9b737e05 Support ChatGLMForConditionalGeneration (#1932)
dancingpipi GitHub shujunhua1
2023-12-06 02:52:48 +08:00
60dc62dc9e add custom server params (#1868) RoyandGitHub 2023-12-04 04:59:18 +08:00
0f90effc66 Bump up to v0.2.3 (#1903) Woosuk KwonandGitHub 2023-12-03 12:27:47 -08:00
464dd985e3 Fix num_gpus when TP > 1 (#1852) Woosuk KwonandGitHub 2023-12-03 12:24:30 -08:00
c07a442854 chore(examples-docs): upgrade to OpenAI V1 (#1785) Massimiliano PronestiandGitHub 2023-12-03 10:11:22 +01:00
cd3aa153a4 Fix broken worker test (#1900) Woosuk KwonandGitHub 2023-12-02 22:17:33 -08:00
9b294976a2 Add PyTorch-native implementation of custom layers (#1898) Woosuk KwonandGitHub 2023-12-02 21:18:40 -08:00
5313c2cb8b Add Production Metrics in Prometheus format (#1890) Simon MoandGitHub 2023-12-02 16:37:44 -08:00
5f09cbdb63 Fix broken sampler tests (#1896)
Woosuk Kwon GitHub Antoni Baum
2023-12-02 16:06:17 -08:00
4cefa9b49b [Docs] Update the AWQ documentation to highlight performance issue (#1883) Simon MoandGitHub 2023-12-02 15:52:47 -08:00
f86bd6190a Fix the typo in SamplingParams' docstring (#1886) JerryandGitHub 2023-12-01 18:06:36 +08:00
e5452ddfd6 Normalize head weights for Baichuan 2 (#1876) Woosuk KwonandGitHub 2023-11-30 20:03:58 -08:00
d06980dfa7 Fix Baichuan tokenizer error (#1874) Woosuk KwonandGitHub 2023-11-30 18:35:50 -08:00
66785cc05c Support chat template and echo for chat API (#1756) Adam BrusselbackandGitHub 2023-11-30 19:43:13 -05:00
05a38612b0 docs: add instruction for langchain (#1162) Massimiliano PronestiandGitHub 2023-11-30 19:57:44 +01:00
d27f4bae39 Fix rope cache key error (#1867) RoyandGitHub 2023-12-01 00:29:28 +08:00
8d8c2f6ffe Support max-model-len argument for throughput benchmark (#1858) aisensiyandGitHub 2023-12-01 00:10:24 +08:00
51d3cb951d Remove max_num_seqs in latency benchmark script (#1855) Woosuk KwonandGitHub 2023-11-30 00:00:32 -08:00
e74b1736a1 Add profile option to latency benchmark script (#1839) Woosuk KwonandGitHub 2023-11-29 23:42:52 -08:00
f07c1ceaa5 [FIX] Fix docker build error (#1831) (#1832)
Allen GitHub Antoni Baum
2023-11-30 15:06:50 +08:00
63b2206ad0 Avoid multiple instantiations of the RoPE class (#1828) Jee LiandGitHub 2023-11-30 15:06:27 +08:00
27feead2f8 Refactor Worker & InputMetadata (#1843) Woosuk KwonandGitHub 2023-11-29 22:16:37 -08:00
c782195662 Disable Logs Requests should Disable Logging of requests. (#1779)
Michael McCulloch GitHub Michael McCulloch
2023-11-29 22:50:02 -07:00
0f621c2c7d [Docs] Add information about using shared memory in docker (#1845) Simon MoandGitHub 2023-11-29 18:33:56 -08:00
a9e4574261 Refactor Attention (#1840) Woosuk KwonandGitHub 2023-11-29 15:37:31 -08:00
0229c386c5 Better integration with Ray Serve (#1821)
FlorianJoncour GitHub FlorianJoncour
2023-11-29 21:25:43 +00:00
a7b3e33078 [Fix] Fix RoPE in ChatGLM-32K (#1841) Woosuk KwonandGitHub 2023-11-29 13:01:19 -08:00
e19a64c7ef [FIX] Fix formatting error in main branch (#1822) Zhuohan LiandGitHub 2023-11-28 16:56:43 -08:00
1cb4ad8de9 [FIX] Fix formatting error Zhuohan Li 2023-11-29 00:40:19 +00:00
6ed068a71a Use the type BlockTable (#1791) explainerauthorsandGitHub 2023-11-28 16:34:05 -08:00
708e6c18b0 [FIX] Fix class naming (#1803) Zhuohan LiandGitHub 2023-11-28 14:08:01 -08:00
b943890484 Fix OPT param names (#1819) Woosuk KwonandGitHub 2023-11-28 11:22:44 -08:00
a1125ad4df Correct comments in parallel_state.py (#1818) explainerauthorsandGitHub 2023-11-28 10:19:35 -08:00
a8b150c595 Init model on GPU to reduce CPU memory footprint (#1796) ljssandGitHub 2023-11-28 03:18:26 +08:00
665cbcec4b Added echo function to OpenAI API server. (#1504) Yunmo ChenandGitHub 2023-11-27 13:29:17 +08:00
7c600440f7 Fix model docstrings (#1764) Woosuk KwonandGitHub 2023-11-23 23:04:44 -08:00
e0c6f556e8 [Build] Avoid building too many extensions (#1624) Yanming WandGitHub 2023-11-23 16:31:19 -08:00
de23687d16 Fix repetition penalty aligned with huggingface (#1577) ljssandGitHub 2023-11-23 06:41:44 +08:00
4cea74c73b Set top_p=0 and top_k=-1 in greedy sampling (#1748) ljssandGitHub 2023-11-23 04:51:09 +08:00
a921d8be9d [DOCS] Add engine args documentation (#1741) CasperandGitHub 2023-11-22 21:31:27 +01:00
094f716bf2 Add stop_token_ids in SamplingParams.__repr__ (#1745) 陈序andGitHub 2023-11-22 12:13:53 +08:00

... 43 44 45 46 47 ...

Commit Graph Select branches Hide Pull Requests main Mono Color

Commit Graph

Select branches

Hide Pull Requests

main