|
7379b3d4b2
|
[V1] Fix multimodal profiling for Molmo (#11325)
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
|
2024-12-19 16:27:22 +00:00 |
|
|
|
6c7f881541
|
[Model] Add JambaForSequenceClassification model (#10860)
Signed-off-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-19 22:48:06 +08:00 |
|
|
Cyrus LeungandGitHub
|
a0f7d53beb
|
[Bugfix] Cleanup Pixtral HF code (#11333)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-19 13:22:00 +00:00 |
|
|
|
5aef49806d
|
[Feature] Add load generation config from model (#11164)
Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2024-12-19 10:50:38 +00:00 |
|
|
|
98356735ac
|
[misc] benchmark_throughput : Add LoRA (#11267)
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
|
2024-12-19 15:43:16 +08:00 |
|
|
|
f26c4aeecb
|
[Misc] Optimize ray worker initialization time (#11275)
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
|
2024-12-18 23:38:02 -08:00 |
|
|
|
8936316d58
|
[Kernel] Refactor Cutlass c3x (#10049)
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
|
2024-12-19 07:00:18 +00:00 |
|
|
Cyrus LeungandGitHub
|
6142ef0ada
|
[VLM] Merged multimodal processor for Qwen2-Audio (#11303)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-19 06:14:17 +00:00 |
|
|
Chen ZhangandGitHub
|
c6b0a7d3ba
|
[V1] Simplify prefix caching logic by removing num_evictable_computed_blocks (#11310)
|
2024-12-19 04:17:12 +00:00 |
|
|
Michael GoinandGitHub
|
a30482f054
|
[CI] Expand test_guided_generate to test all backends (#11313)
Signed-off-by: mgoin <michael@neuralmagic.com>
|
2024-12-19 04:00:38 +00:00 |
|
|
Travis JohnsonandGitHub
|
17ca964273
|
[Model] IBM Granite 3.1 (#11307)
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
|
2024-12-19 11:27:24 +08:00 |
|
|
Tyler Michael SmithandGitHub
|
5a9da2e6e9
|
[Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311)
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2024-12-19 02:43:30 +00:00 |
|
|
Alexander MatveevandGitHub
|
fdea8ec167
|
[V1] VLM - enable processor cache by default (#11305)
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
|
2024-12-18 18:54:46 -05:00 |
|
|
Joe RundeandGitHub
|
ca5f54a9b9
|
[Bugfix] fix minicpmv test (#11304)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
|
2024-12-18 10:34:26 -08:00 |
|
|
Kunshang JiandGitHub
|
f954fe0e65
|
[FIX] update openai version (#11287)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
|
2024-12-18 10:17:05 -08:00 |
|
|
Simon MoandGitHub
|
362cff1eb3
|
[CI][Misc] Remove Github Action Release Workflow (#11274)
|
2024-12-18 10:16:53 -08:00 |
|
|
Isotr0pyandGitHub
|
996aa70f00
|
[Bugfix] Fix broken phi3-v mm_processor_kwargs tests (#11263)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2024-12-18 10:16:40 -08:00 |
|
|
|
60508ffda9
|
[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995)
Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
|
2024-12-18 09:57:16 -05:00 |
|
|
Yan MaandGitHub
|
f04e407e6b
|
[MISC][XPU]update ipex link for CI fix (#11278)
|
2024-12-17 22:34:23 -08:00 |
|
|
Wallas HenriqueandGitHub
|
8b79f9e107
|
[Bugfix] Fix guided decoding with tokenizer mode mistral (#11046)
|
2024-12-17 22:34:08 -08:00 |
|
|
Konrad ZaworaandGitHub
|
866fa4550d
|
[Bugfix] Restore support for larger block sizes (#11259)
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
|
2024-12-17 16:39:07 -08:00 |
|
|
Cody YuandGitHub
|
bf8717ebae
|
[V1] Prefix caching for vision language models (#11187)
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
|
2024-12-17 16:37:59 -08:00 |
|
|
Michael GoinandGitHub
|
c77eb8a33c
|
[Bugfix] Set temperature=0.7 in test_guided_choice_chat (#11264)
|
2024-12-17 16:34:06 -08:00 |
|
|
Joe RundeandGitHub
|
2d1b9baa8f
|
[Bugfix] Fix request cancellation without polling (#11190)
|
2024-12-17 12:26:32 -08:00 |
|
|
Isotr0pyandGitHub
|
f9ecbb18bf
|
[Misc] Allow passing logits_soft_cap for xformers backend (#11252)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2024-12-17 00:37:04 -08:00 |
|
|
|
02222a0256
|
[Misc] Kernel Benchmark for RMSNorm (#11241)
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiaoyu Zhang <BBuf@users.noreply.github.com>
|
2024-12-17 06:57:02 +00:00 |
|
|
Tyler Michael SmithandGitHub
|
2bfdbf2a36
|
[V1][Core] Use weakref.finalize instead of atexit (#11242)
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2024-12-16 22:11:33 -08:00 |
|
|
wangxiyuanandGitHub
|
e88db68cf5
|
[Platform] platform agnostic for EngineArgs initialization (#11225)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
|
2024-12-16 22:11:06 -08:00 |
|
|
|
59c9b6ebeb
|
[V1][VLM] Proper memory profiling for image language models (#11210)
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
|
2024-12-16 22:10:57 -08:00 |
|
|
|
66d4b16724
|
[Frontend] Add OpenAI API support for input_audio (#11027)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-16 22:09:58 -08:00 |
|
|
Michael GoinandGitHub
|
0064f697d3
|
[CI] Add test case with JSON schema using references + use xgrammar by default with OpenAI parse (#10935)
Signed-off-by: mgoin <michael@neuralmagic.com>
|
2024-12-17 11:39:58 +08:00 |
|
|
youkaichaoandGitHub
|
35bae114a8
|
fix gh200 tests on main (#11246)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-16 17:22:38 -08:00 |
|
|
|
88a412ed3d
|
[torch.compile] fast inductor (#11108)
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2024-12-16 16:15:22 -08:00 |
|
|
youkaichaoandGitHub
|
c301616ed2
|
[ci][tests] add gh200 tests (#11244)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-16 15:53:18 -08:00 |
|
|
|
35ffa682b1
|
[Docs] hint to enable use of GPU performance counters in profiling tools for multi-node distributed serving (#11235)
Co-authored-by: Michael Goin <michael@neuralmagic.com>
|
2024-12-16 22:20:39 +00:00 |
|
|
youkaichaoandGitHub
|
551603feff
|
[core] overhaul memory profiling and fix backward compatibility (#10511)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-16 13:32:25 -08:00 |
|
|
|
efbce85f4d
|
[misc] Layerwise profile updates (#10242)
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
|
2024-12-16 18:14:57 +00:00 |
|
|
Isotr0pyandGitHub
|
2ca830dbaa
|
[Doc] Reorder vision language examples in alphabet order (#11228)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2024-12-16 11:23:33 +00:00 |
|
|
|
d927dbcd88
|
[Model] Refactor Ultravox to use merged input processor (#11198)
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2024-12-16 10:09:53 +00:00 |
|
|
Jani MonosesandGitHub
|
bddbbcb132
|
[Model] Support Cohere2ForCausalLM (Cohere R7B) (#11203)
|
2024-12-16 09:56:19 +00:00 |
|
|
|
b3b1526f03
|
WIP: [CI/Build] simplify Dockerfile build for ARM64 / GH200 (#11212)
Signed-off-by: drikster80 <ed.sealing@gmail.com>
Co-authored-by: drikster80 <ed.sealing@gmail.com>
|
2024-12-16 09:20:49 +00:00 |
|
|
yansh97andGitHub
|
17138af7c4
|
[Bugfix] Fix the default value for temperature in ChatCompletionRequest (#11219)
|
2024-12-16 00:15:40 -08:00 |
|
|
chenqianfzhandGitHub
|
69ba344de8
|
[Bugfix] Fix block size validation (#10938)
|
2024-12-15 16:38:40 -08:00 |
|
|
AlexHe99andGitHub
|
da6f409246
|
Update deploying_with_k8s.rst (#10922)
|
2024-12-15 16:33:58 -08:00 |
|
|
Woosuk KwonandGitHub
|
25ebed2f8c
|
[V1][Minor] Cache np arange to reduce input preparation overhead (#11214)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-12-15 13:33:00 -08:00 |
|
|
shangmingcandGitHub
|
d263bd9df7
|
[Core] Support disaggregated prefill with Mooncake Transfer Engine (#10884)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2024-12-15 21:28:18 +00:00 |
|
|
Kuntai DuandGitHub
|
38e599d6a8
|
[Doc] add documentation for disaggregated prefilling (#11197)
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
|
2024-12-15 13:31:16 -06:00 |
|
|
Cyrus LeungandGitHub
|
96d673e0f8
|
[Bugfix] Fix error handling of unsupported sliding window (#11213)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-15 10:59:42 -07:00 |
|
|
Cyrus LeungandGitHub
|
b10609e6a1
|
[Misc] Clean up multi-modal processor (#11207)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-15 06:30:28 +00:00 |
|
|
youkaichaoandGitHub
|
a1c02058ba
|
[torch.compile] allow tracking forward time (#11081)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-14 19:45:00 -08:00 |
|