 Russell BryantandGitHub
|
d65798f78c
|
[Core] zmq: bind only to 127.0.0.1 for local-only usage (#8543)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
|
2024-09-18 16:10:27 +00:00 |
|
|
afeldman-nmandGitHub
|
a8c1d161a7
|
[Core] *Prompt* logprobs support in Multi-step (#8199)
|
2024-09-18 08:38:43 -07:00 |
|
|
|
7c7714d856
|
[Core][Bugfix][Perf] Introduce MQLLMEngine to avoid asyncio OH (#8157)
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
|
2024-09-18 13:56:58 +00:00 |
|
|
|
9d104b5beb
|
[CI/Build] Update Ruff version (#8469)
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2024-09-18 11:00:56 +00:00 |
|
|
Cyrus LeungandGitHub
|
6ffa3f314c
|
[CI/Build] Avoid CUDA initialization (#8534)
|
2024-09-18 10:38:11 +00:00 |
|
|
Jiaxin ShanandGitHub
|
e351572900
|
[Misc] Add argument to disable FastAPI docs (#8554)
|
2024-09-18 09:51:59 +00:00 |
|
|
DanieleandGitHub
|
95965d31b6
|
[CI/Build] fix Dockerfile.cpu on podman (#8540)
|
2024-09-18 10:49:53 +08:00 |
|
|
Tyler Michael SmithandGitHub
|
8110e44529
|
[Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012)
|
2024-09-17 23:44:27 +00:00 |
|
|
Alexey Kondratiev(AMD)andGitHub
|
09deb4721f
|
[CI/Build] Excluding kernels/test_gguf.py from ROCm (#8520)
|
2024-09-17 16:40:29 -07:00 |
|
|
|
fa0c114fad
|
[doc] improve installation doc (#8550)
Co-authored-by: Andy Dai <76841985+Imss27@users.noreply.github.com>
|
2024-09-17 16:24:06 -07:00 |
|
|
Joe RundeandGitHub
|
98f9713399
|
[Bugfix] Fix TP > 1 for new granite (#8544)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
|
2024-09-17 23:17:08 +00:00 |
|
|
Nick HillandGitHub
|
56c3de018c
|
[Misc] Don't dump contents of kvcache tensors on errors (#8527)
|
2024-09-17 12:24:29 -07:00 |
|
|
|
a54ed80249
|
[Model] Add mistral function calling format to all models loaded with "mistral" format (#8515)
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2024-09-17 17:50:37 +00:00 |
|
|
chenqianfzhandGitHub
|
9855b99502
|
[Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434)
|
2024-09-17 08:09:12 -07:00 |
|
|
sroy745andGitHub
|
1009e93c5d
|
[Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631)
|
2024-09-17 07:35:01 -07:00 |
|
|
Isotr0pyandGitHub
|
1b6de8352b
|
[Benchmark] Support sample from HF datasets and image input for benchmark_serving (#8495)
|
2024-09-17 07:34:27 +00:00 |
|
|
Rui QiaoandGitHub
|
cbdb252259
|
[Misc] Limit to ray[adag] 2.35 to avoid backward incompatible change (#8509)
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
|
2024-09-17 00:06:26 -07:00 |
|
|
youkaichaoandGitHub
|
99aa4eddaf
|
[torch.compile] register allreduce operations as custom ops (#8526)
|
2024-09-16 22:57:57 -07:00 |
|
|
Roger WangandGitHub
|
ee2bceaaa6
|
[Misc][Bugfix] Disable guided decoding for mistral tokenizer (#8521)
|
2024-09-16 22:22:45 -07:00 |
|
|
Alex BrooksandGitHub
|
1c1bb388e0
|
[Frontend] Improve Nullable kv Arg Parsing (#8525)
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
|
2024-09-17 04:17:32 +00:00 |
|
|
Simon MoandGitHub
|
546034b466
|
[refactor] remove triton based sampler (#8524)
|
2024-09-16 20:04:48 -07:00 |
|
|
Joe RundeandGitHub
|
cca61642e0
|
[Bugfix] Fix 3.12 builds on main (#8510)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
|
2024-09-17 00:01:45 +00:00 |
|
|
Simon MoandGitHub
|
5ce45eb54d
|
[misc] small qol fixes for release process (#8517)
|
2024-09-16 15:11:27 -07:00 |
|
|
Simon MoandGitHub
|
5478c4b41f
|
[perf bench] set timeout to debug hanging (#8516)
|
2024-09-16 14:30:02 -07:00 |
|
|
Kevin LinandGitHub
|
47f5e03b5b
|
[Bugfix] Bind api server port before starting engine (#8491)
|
2024-09-16 13:56:28 -07:00 |
|
|
youkaichaoandGitHub
|
2759a43a26
|
[doc] update doc on testing and debugging (#8514)
|
2024-09-16 12:10:23 -07:00 |
|
|
Luka GovedičandGitHub
|
5d73ae49d6
|
[Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270)
|
2024-09-16 11:52:40 -07:00 |
|
|
sasha0552andGitHub
|
781e3b9a42
|
[Bugfix][Kernel] Fix build for sm_60 in GGUF kernel (#8506)
|
2024-09-16 12:15:57 -06:00 |
|
|
Nick HillandGitHub
|
acd5511b6d
|
[BugFix] Fix clean shutdown issues (#8492)
|
2024-09-16 09:33:46 -07:00 |
|
|
lewtunandGitHub
|
837c1968f9
|
[Frontend] Expose revision arg in OpenAI server (#8501)
|
2024-09-16 15:55:26 +00:00 |
|
|
|
a091e2da3e
|
[Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032)
Co-authored-by: Dipika <dipikasikka1@gmail.com>
|
2024-09-16 09:47:19 -06:00 |
|
|
Isotr0pyandGitHub
|
fc990f9795
|
[Bugfix][Kernel] Add IQ1_M quantization implementation to GGUF kernel (#8357)
|
2024-09-15 16:51:44 -06:00 |
|
|
ChrisandGitHub
|
3724d5f6b5
|
[Bugfix][Model] Fix Python 3.8 compatibility in Pixtral model by updating type annotations (#8490)
|
2024-09-15 04:20:05 +00:00 |
|
|
Woosuk KwonandGitHub
|
50e9ec41fc
|
[TPU] Implement multi-step scheduling (#8489)
|
2024-09-14 16:58:31 -07:00 |
|
|
youkaichaoandGitHub
|
47790f3e32
|
[torch.compile] add a flag to disable custom op (#8488)
|
2024-09-14 13:07:16 -07:00 |
|
|
youkaichaoandGitHub
|
a36e070dad
|
[torch.compile] fix functionalization (#8480)
|
2024-09-14 09:46:04 -07:00 |
|
|
|
8a0cf1ddc3
|
[Model] support minicpm3 (#8297)
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-09-14 14:50:26 +00:00 |
|
|
Charlie FuandGitHub
|
1ef0d2efd0
|
[Kernel][Hardware][Amd]Custom paged attention kernel for rocm (#8310)
|
2024-09-13 17:01:11 -07:00 |
|
|
|
851725202a
|
[Hardware][intel GPU] bump up ipex version to 2.3 (#8365)
Co-authored-by: Yan Ma <yan.ma@intel.com>
|
2024-09-13 16:54:34 -07:00 |
|
|
Simon MoandGitHub
|
9ba0817ff1
|
bump version to v0.6.1.post2 (#8473)
|
2024-09-13 11:35:00 -07:00 |
|
|
Nick HillandGitHub
|
18e9e1f7b3
|
[HotFix] Fix final output truncation with stop string + streaming (#8468)
|
2024-09-13 11:31:12 -07:00 |
|
|
Isotr0pyandGitHub
|
f57092c00b
|
[Doc] Add oneDNN installation to CPU backend documentation (#8467)
|
2024-09-13 18:06:30 +00:00 |
|
|
Cyrus LeungandGitHub
|
a84e598e21
|
[CI/Build] Reorganize models tests (#7820)
|
2024-09-13 10:20:06 -07:00 |
|
|
youkaichaoandGitHub
|
0a4806f0a9
|
[plugin][torch.compile] allow to add custom compile backend (#8445)
|
2024-09-13 09:32:42 -07:00 |
|
|
Cyrus LeungandGitHub
|
ecd7a1d5b6
|
[Installation] Gate FastAPI version for Python 3.8 (#8456)
|
2024-09-13 09:02:26 -07:00 |
|
|
youkaichaoandGitHub
|
a2469127db
|
[misc][ci] fix quant test (#8449)
|
2024-09-13 17:20:14 +08:00 |
|
|
Jee Jee LiandGitHub
|
06311e2956
|
[Misc] Skip loading extra bias for Qwen2-VL GPTQ-Int8 (#8442)
|
2024-09-13 07:58:28 +00:00 |
|
|
youkaichaoandGitHub
|
cab69a15e4
|
[doc] recommend pip instead of conda (#8446)
|
2024-09-12 23:52:41 -07:00 |
|
|
Isotr0pyandGitHub
|
9b4a3b235e
|
[CI/Build] Enable InternVL2 PP test only on single node (#8437)
|
2024-09-13 06:35:20 +00:00 |
|
|
Simon MoandGitHub
|
acda0b35d0
|
bump version to v0.6.1.post1 (#8440)
|
2024-09-12 21:39:49 -07:00 |
|