 wchen61andGitHub
|
3d158cdc8d
|
Add default value to avoid Falcon crash (#5363) (#10347)
Signed-off-by: wchen61 <wchen61@foxmail.com>
|
2024-11-15 08:52:20 +00:00 |
|
|
Cyrus LeungandGitHub
|
2ac6d0e75b
|
[Misc] Consolidate pooler config overrides (#10351)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-15 06:59:00 +00:00 |
|
|
Sky LeeandGitHub
|
2ec8827288
|
[Bugfix] Qwen-vl output is inconsistent in speculative decoding (#10350)
|
2024-11-15 05:40:10 +00:00 |
|
|
Cyrus LeungandGitHub
|
b40cf6402e
|
[Model] Support Qwen2 embeddings and use tags to select model tests (#10184)
|
2024-11-14 20:23:09 -08:00 |
|
|
Tyler Michael SmithandGitHub
|
2885ba0e24
|
[Misc] Change RedundantReshapesPass and FusionPass logging from info to debug (#10308)
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2024-11-15 02:44:26 +00:00 |
|
|
|
bf2ddc6610
|
[bugfix] Fix static asymmetric quantization case (#10334)
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
|
2024-11-15 09:35:11 +08:00 |
|
|
Cyrus LeungandGitHub
|
972112d82f
|
[Bugfix] Fix unable to load some models (#10312)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-14 16:55:54 -08:00 |
|
|
Patrick von PlatenandGitHub
|
11cd1ae6ad
|
[Tool parsing] Improve / correct mistral tool parsing (#10333)
|
2024-11-15 00:42:49 +00:00 |
|
|
Zijin XiaoandGitHub
|
554af9228d
|
[Bugfix] use AF_INET6 for OpenAI Compatible Server with ipv6 (#9583)
Signed-off-by: xiaozijin <xiaozijin@bytedance.com>
|
2024-11-14 16:38:53 -08:00 |
|
|
Murali AndoorveeduandGitHub
|
b2e0ad3b59
|
[Perf] Reduce peak memory usage of llama (#10339)
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
|
2024-11-15 00:38:20 +00:00 |
|
|
|
4a18fd14ba
|
Support Roberta embedding models (#9387)
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
|
2024-11-14 21:23:29 +00:00 |
|
|
|
294bf467ba
|
[Model] Add BNB quantization support for Idefics3 (#10310)
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-11-14 06:31:44 +00:00 |
|
|
Guillaume CalmettesandGitHub
|
52b48c1ead
|
[BugFix]: properly deserialize tool_calls iterator before processing by mistral-common when MistralTokenizer is used (#9951)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
|
2024-11-14 04:48:16 +00:00 |
|
|
Mike DepinetandGitHub
|
f67ce05d0b
|
[Frontend] Pythonic tool parser (#9859)
Signed-off-by: Mike Depinet <mike@fixie.ai>
|
2024-11-14 04:14:34 +00:00 |
|
|
youkaichaoandGitHub
|
504ac53d18
|
[misc] error early for old-style class (#10304)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-13 18:55:39 -08:00 |
|
|
Isotr0pyandGitHub
|
15bb8330aa
|
[Bugfix] Fix tensor parallel for qwen2 classification model (#10297)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2024-11-14 10:54:59 +08:00 |
|
|
HoangCongDucandGitHub
|
ac49b59d8b
|
[Bugfix] bitsandbytes models fail to run pipeline parallel (#10200)
Signed-off-by: Hoang Cong Duc <hoangcongducltt@gmail.com>
|
2024-11-13 09:56:39 -07:00 |
|
|
Cyrus LeungandGitHub
|
0b8bb86bf1
|
[1/N] Initial prototype for multi-modal processor (#10044)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-13 12:39:03 +00:00 |
|
|
Roger WangandGitHub
|
bb7991aa29
|
[V1] Add missing tokenizer options for Detokenizer (#10288)
Signed-off-by: Roger Wang <ywang@roblox.com>
|
2024-11-13 11:02:56 +00:00 |
|
|
B-201andGitHub
|
d909acf9fe
|
[Model][LoRA]LoRA support added for idefics3 (#10281)
Signed-off-by: B-201 <Joy25810@foxmail.com>
|
2024-11-13 17:25:59 +08:00 |
|
|
|
1b886aa104
|
[Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944)
Signed-off-by: FurtherAI <austin.veselka@lighton.ai>
Co-authored-by: FurtherAI <austin.veselka@lighton.ai>
|
2024-11-13 08:28:13 +00:00 |
|
|
电脑星人andGitHub
|
3945c82346
|
[Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (#10221)
Signed-off-by: imkero <kerorek@outlook.com>
|
2024-11-13 07:07:22 +00:00 |
|
|
Xin YangandGitHub
|
032fcf16ae
|
[Doc] Fix typo in arg_utils.py (#10264)
Signed-off-by: Xin Yang <xyang19@gmail.com>
|
2024-11-12 21:54:52 -08:00 |
|
|
|
bbd3e86926
|
[V1] Support VLMs with fine-grained scheduling (#9871)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
|
2024-11-13 04:53:13 +00:00 |
|
|
youkaichaoandGitHub
|
0d4ea3fb5c
|
[core][distributed] use tcp store directly (#10275)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-12 17:36:08 -08:00 |
|
|
Woosuk KwonandGitHub
|
112fa0bbe5
|
[V1] Fix CI tests on V1 engine (#10272)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-12 16:17:20 -08:00 |
|
|
Woosuk KwonandGitHub
|
1f55e05713
|
[V1] Enable Inductor when using piecewise CUDA graphs (#10268)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-12 13:39:56 -08:00 |
|
|
|
8a06428c70
|
[LoRA] Adds support for bias in LoRA (#5733)
Signed-off-by: Umesh Deshpande <udeshpa@us.ibm.com>
Co-authored-by: Umesh Deshpande <udeshpa@us.ibm.com>
|
2024-11-12 11:08:40 -08:00 |
|
|
sroy745andGitHub
|
b41fb9d3b1
|
[Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982)
Signed-off-by: Sourashis Roy <sroy@roblox.com>
|
2024-11-12 10:53:57 -08:00 |
|
|
Woosuk KwonandGitHub
|
7c65527918
|
[V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to EngineCoreRequest (#10245)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-12 08:57:14 -08:00 |
|
|
zifeitongandGitHub
|
47db6ec831
|
[Frontend] Add per-request number of cached token stats (#10174)
|
2024-11-12 16:42:28 +00:00 |
|
|
Jie Fu (傅杰)andGitHub
|
176fcb1c71
|
[Bugfix] Fix QwenModel argument (#10262)
Signed-off-by: Jie Fu <jiefu@tencent.com>
|
2024-11-12 16:36:51 +00:00 |
|
|
Jee Jee LiandGitHub
|
a838ba7254
|
[Misc]Fix Idefics3Model argument (#10255)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-11-12 13:07:11 +00:00 |
|
|
Guillaume CalmettesandGitHub
|
36c513a076
|
[BugFix] Do not raise a ValueError when tool_choice is set to the supported none option and tools are not defined. (#10000)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
|
2024-11-12 11:13:46 +00:00 |
|
|
Jee Jee LiandGitHub
|
7f5edb5900
|
[Misc][LoRA] Replace hardcoded cuda device with configurable argument (#10223)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-11-12 11:10:15 +08:00 |
|
|
youkaichaoandGitHub
|
eea55cca5b
|
[1/N] torch.compile user interface design (#10237)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-11 18:01:06 -08:00 |
|
|
Russell BryantandGitHub
|
9cdba9669c
|
[Doc] Update help text for --distributed-executor-backend (#10231)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
2024-11-12 09:55:09 +08:00 |
|
|
|
6ace6fba2c
|
[V1] AsyncLLM Implementation (#9826)
Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2024-11-11 23:05:38 +00:00 |
|
|
Woosuk KwonandGitHub
|
9d5b4e4dea
|
[V1] Enable custom ops with piecewise CUDA graphs (#10228)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-11 11:58:07 -08:00 |
|
|
Woosuk KwonandGitHub
|
fe15729a2b
|
[V1] Use custom ops for piecewise CUDA graphs (#10227)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-11 11:26:48 -08:00 |
|
|
|
330e82d34a
|
[v1][torch.compile] support managing cudagraph buffer (#10203)
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-11 11:10:27 -08:00 |
|
|
Woosuk KwonandGitHub
|
d7a4f2207b
|
[V1] Do not use inductor for piecewise CUDA graphs (#10225)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-11 11:05:57 -08:00 |
|
|
Woosuk KwonandGitHub
|
f9dadfbee3
|
[V1] Fix detokenizer ports (#10224)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-11-11 10:42:07 -08:00 |
|
|
youkaichaoandGitHub
|
e6de9784d2
|
[core][distributed] add stateless process group (#10216)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-11-11 09:02:14 -08:00 |
|
|
Yangcheng LiandGitHub
|
36fc439de0
|
[Doc] fix doc string typo in block_manager swap_out function (#10212)
|
2024-11-11 08:53:07 -08:00 |
|
|
|
874f551b36
|
[Metrics] add more metrics (#4464)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-11-12 00:17:38 +08:00 |
|
|
Isotr0pyandGitHub
|
2cebda42bb
|
[Bugfix][Hardware][CPU] Fix broken encoder-decoder CPU runner (#10218)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2024-11-11 12:37:58 +00:00 |
|
|
Roger WangandGitHub
|
5fb1f935b0
|
[V1] Allow tokenizer_mode and trust_remote_code for Detokenizer (#10211)
Signed-off-by: Roger Wang <ywang@roblox.com>
|
2024-11-11 18:01:18 +08:00 |
|
|
Jee Jee LiandGitHub
|
36e4acd02a
|
[LoRA][Kernel] Remove the unused libentry module (#10214)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-11-11 09:43:23 +00:00 |
|
|
Isotr0pyandGitHub
|
58170d6503
|
[Hardware][CPU] Add embedding models support for CPU backend (#10193)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2024-11-11 08:54:28 +00:00 |
|