Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• 28e1299e60 rename PromptInputs and inputs with backward compatibility (#8760) Cyrus LeungandGitHub 2024-09-26 00:36:47 +08:00
• 0c4d2ad5e6 [VLM][Bugfix] internvl with num_scheduler_steps > 1 (#8614) DefTruthandGitHub 2024-09-26 00:35:53 +08:00
• c6f2485c82 [[Misc]] Add extra deps for openai server image (#8792) Jee Jee LiandGitHub 2024-09-26 00:35:23 +08:00
• 300da09177 [Kernel] Fullgraph and opcheck tests (#8479) bnellnmandGitHub 2024-09-25 10:35:52 -04:00
• 1c046447a6 [CI/Build][Bugfix][Doc][ROCm] CI fix and doc update after ROCm 6.2 upgrade (#8777) Hongxia YangandGitHub 2024-09-25 10:26:37 -04:00
• 8fae5ed7f6 [Misc] Fix minor typo in scheduler (#8765) Woo-Yeon LeeandGitHub 2024-09-25 16:53:03 +09:00
• 3368c3ab36 [Bugfix] Ray 2.9.x doesn't expose available_resources_per_node (#8767) David NewmanandGitHub 2024-09-25 17:52:26 +10:00
• 1ac3de09cd [Frontend] OpenAI server: propagate usage accounting to FastAPI middleware layer (#8672) Adam TilghmanandGitHub 2024-09-25 00:49:26 -07:00
• 3e073e66f1 [Bugfix] load fc bias from config for eagle (#8790) sohamparikhandGitHub 2024-09-25 02:16:30 -04:00
• c23953675f [Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (#8770) Isotr0pyandGitHub 2024-09-25 14:16:11 +08:00
• e3dd0692fa [BugFix] Propagate 'trust_remote_code' setting in internvl and minicpmv (#8250) zifeitongandGitHub 2024-09-24 22:53:43 -07:00
• fc3afc20df Fix tests in test_chunked_prefill_scheduler which fail with BlockManager V2 (#8752) sroy745andGitHub 2024-09-24 21:26:36 -07:00
• b4522474a3 [Bugfix][Kernel] Implement acquire/release polyfill for Pascal (#8776) sasha0552andGitHub 2024-09-25 04:26:33 +00:00
• ee777d9c30 Fix test_schedule_swapped_simple in test_scheduler.py (#8780) sroy745andGitHub 2024-09-24 21:26:18 -07:00
• 6e0c9d6bd0 [Bugfix] Use heartbeats instead of health checks (#8583) Joe RundeandGitHub 2024-09-24 21:37:38 -06:00
• 6da1ab6b41 [Core] Adding Priority Scheduling (#5958) Archit PatkeandGitHub 2024-09-24 21:50:50 -05:00
• 01b6f9e1f0 [Core][Bugfix] Support prompt_logprobs returned with speculative decoding (#8047) Travis JohnsonandGitHub 2024-09-24 18:29:56 -06:00
• 13f9f7a3d0 [[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (#8768) Jee Jee LiandGitHub 2024-09-25 08:08:55 +08:00
• 1e7d5c01f5 [misc] soft drop beam search (#8763) youkaichaoandGitHub 2024-09-24 15:48:39 -07:00
• 2467b642dd [CI/Build] fix setuptools-scm usage (#8771) DanieleandGitHub 2024-09-24 21:38:12 +02:00
• 72fc97a0f1 [Bugfix] Fix torch dynamo fixes caused by replace_parameters (#8748) Lucas WilkinsonandGitHub 2024-09-24 14:33:21 -04:00
• 2529d09b5a [Frontend] Batch inference for llm.chat() API (#8648) 6 people 2024-09-24 12:44:11 -04:00
• a928ded995 [Kernel] Split Marlin MoE kernels into multiple files (#8661) 3 people 2024-09-24 18:31:42 +02:00
• cc4325b66a [Bugfix] Fix potentially unsafe custom allreduce synchronization (#8558) Hanzhi ZhouandGitHub 2024-09-24 01:08:14 -07:00
• 8ff7ced996 [Model] Expose Phi3v num_crops as a mm_processor_kwarg (#8658) 4 people 2024-09-24 01:36:46 -06:00
• 3f06bae907 [Core][Model] Support loading weights by ID within models (#7931) Peter SalasandGitHub 2024-09-24 00:14:15 -07:00
• b8747e8a7c [MISC] Skip dumping inputs when unpicklable (#8744) Cody YuandGitHub 2024-09-23 23:10:03 -07:00
• 3185fb0cca Revert "[Core] Rename PromptInputs to PromptType, and inputs to prompt" (#8750) Simon MoandGitHub 2024-09-23 22:45:20 -07:00
• 0250dd68c5 re-implement beam search on top of vllm core (#8726) 3 people 2024-09-23 22:08:12 -07:00
• 88577ac928 Fix tests in test_scheduler.py that fail with BlockManager V2 (#8728) sroy745andGitHub 2024-09-23 21:43:13 -07:00
• 530821d00c [Hardware][AMD] ROCm6.2 upgrade (#8674) Hongxia YangandGitHub 2024-09-23 21:52:39 -04:00
• 1a2aef3e59 Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335) Alexander MatveevandGitHub 2024-09-23 18:38:04 -04:00
• 5f7bb58427 Fix typical acceptance sampler with correct recovered token ids (#8562) jiqing-fengandGitHub 2024-09-24 03:32:27 +08:00
• b05f5c9238 [Core] Allow IPv6 in VLLM_HOST_IP with zmq (#8575) Russell BryantandGitHub 2024-09-23 15:15:41 -04:00
• 9b0e3ec970 [Kernel][LoRA] Add assertion for punica sgmv kernels (#7585) Jee Jee LiandGitHub 2024-09-24 02:57:42 +08:00
• 86e9c8df29 [Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701) 5 people 2024-09-23 13:46:26 -04:00
• ee5f34b1c2 [CI/Build] use setuptools-scm to set __version__ (#4738) 3 people 2024-09-23 18:44:26 +02:00
• f2bd246c17 [VLM] Fix paligemma, fuyu and persimmon with transformers 4.45 : use config.text_config.vocab_size (#8707) Jani MonosesandGitHub 2024-09-23 17:43:09 +03:00
• a79e522984 [Model] Support pp for qwen2-vl (#8696) Yanyi LiuandGitHub 2024-09-23 21:46:59 +08:00
• 3e83c12b5c [Bugfix][CPU] fix missing input intermediate_tensors in the cpu_model_runner (#8733) Li, JiangandGitHub 2024-09-23 21:15:16 +08:00
• e551ca1555 [Hardware][CPU] Refactor CPU model runner (#8729) Isotr0pyandGitHub 2024-09-23 20:12:20 +08:00
• 9b8c8ba119 [Core][Frontend] Support Passing Multimodal Processor Kwargs (#8657) Alex BrooksandGitHub 2024-09-23 01:44:48 -06:00
• d23679eb99 [Bugfix] fix docker build for xpu (#8652) Yan MaandGitHub 2024-09-23 13:54:18 +08:00
• 57a0702e63 [Bugfix] Fix CPU CMake build (#8723) 3 people 2024-09-22 23:40:46 -04:00
• 3dda7c2250 [Bugfix] Avoid some bogus messages RE CUTLASS's revision when building (#8702) Tyler Michael SmithandGitHub 2024-09-22 22:24:59 -04:00
• 92ba7e7477 [misc] upgrade mistral-common (#8715) youkaichaoandGitHub 2024-09-22 15:41:59 -07:00
• d4a2ac8302 [build] enable existing pytorch (for GH200, aarch64, nightly) (#8713) youkaichaoandGitHub 2024-09-22 12:47:54 -07:00
• c6bd70d772 [SpecDec][Misc] Cleanup, remove bonus token logic. (#8701) Lily LiuandGitHub 2024-09-22 12:34:14 -07:00
• 5b59532760 [Model][VLM] Add LLaVA-Onevision model support (#8486) 6 people 2024-09-23 01:51:44 +08:00
• ca2b628b3c [MISC] rename CudaMemoryProfiler to DeviceMemoryProfiler (#8703) Huazhong JiandGitHub 2024-09-23 01:44:09 +08:00
• 8ca5051b9a [Misc] Use NamedTuple in Multi-image example (#8705) Alex BrooksandGitHub 2024-09-22 06:56:20 -06:00
• 06ed2815e2 [Model] Refactor BLIP/BLIP-2 to support composite model loading (#8407) Cyrus LeungandGitHub 2024-09-22 20:24:21 +08:00
• 0e40ac9b7b [ci][build] fix vllm-flash-attn (#8699) youkaichaoandGitHub 2024-09-21 23:24:58 -07:00
• 13d88d4137 [Bugfix] Refactor composite weight loading logic (#8656) Isotr0pyandGitHub 2024-09-22 12:33:27 +08:00
• d66ac62854 [Kernel][Bugfix] Delete some more useless code in marlin_moe_ops.cu (#8643) Tyler Michael SmithandGitHub 2024-09-21 19:45:02 -04:00
• 9dc7c6c7f3 [dbrx] refactor dbrx experts to extend FusedMoe class (#8518) Divakar VermaandGitHub 2024-09-21 16:09:39 -05:00
• ec4aaad812 [Kernel][Triton][AMD] Remove tl.atomic_add from awq_gemm_kernel, 2-5x speedup MI300, minor improvement for MI250 (#8646) rasmithandGitHub 2024-09-21 04:20:54 -05:00
• 4dfdf43196 [Doc] Fix typo in AMD installation guide (#8689) Andy DaiandGitHub 2024-09-21 00:24:12 -07:00
• 5e85f4f82a [VLM] Use SequenceData.from_token_counts to create dummy data (#8687) Cyrus LeungandGitHub 2024-09-21 14:28:56 +08:00
• 71c60491f2 [Kernel] Build flash-attn from source (#8245) Luka GovedičandGitHub 2024-09-21 02:27:10 -04:00
• 0faab90eb0 [beam search] add output for manually checking the correctness (#8684) youkaichaoandGitHub 2024-09-20 19:55:33 -07:00
• 0455c46ed4 [Core] Factor out common code in SequenceData and Sequence (#8675) Cyrus LeungandGitHub 2024-09-21 10:30:39 +08:00
• d4bf085ad0 [MISC] add support custom_op check (#8557) 3 people 2024-09-21 10:03:55 +08:00
• 0057894ef7 [Core] Rename PromptInputs and inputs(#8673) Cyrus LeungandGitHub 2024-09-21 10:00:54 +08:00
• 0f961b3ce9 [Bugfix] Fix incorrect llava next feature size calculation (#8496) zyddnysandGitHub 2024-09-20 18:48:32 -04:00
• 7f9c8902e3 [Hardware][AWS] update neuron to 2.20 (#8676) omrishivandGitHub 2024-09-20 15:19:44 -07:00
• 7c8566aa4f [Doc] neuron documentation update (#8671) omrishivandGitHub 2024-09-20 15:04:37 -07:00
• b4e4eda92e [Bugfix][Core] Fix tekken edge case for mistral tokenizer (#8640) Patrick von PlatenandGitHub 2024-09-20 23:33:03 +02:00
• 2874bac618 [Bugfix] Config got an unexpected keyword argument 'engine' (#8556) Pastel！andGitHub 2024-09-21 05:00:45 +08:00
• 035fa895ec [Misc] Show AMD GPU topology in collect_env.py (#8649) Cyrus LeungandGitHub 2024-09-21 04:52:19 +08:00
• b28298f2f4 [Bugfix] Validate SamplingParam n is an int (#8548) saumya-saranandGitHub 2024-09-20 12:46:02 -07:00
• 2940afa04e [CI/Build] Removing entrypoints/openai/test_embedding.py test from ROCm build (#8670) Alexey Kondratiev(AMD)andGitHub 2024-09-20 13:27:44 -04:00
• 3b63de9353 [Model] Add OLMoE (#7922) Niklas MuennighoffandGitHub 2024-09-20 09:31:41 -07:00
• 260d40b5ea [Core] Support Lora lineage and base model metadata management (#6315) Jiaxin ShanandGitHub 2024-09-19 23:20:56 -07:00
• 9e5ec35b1f [bugfix] [AMD] add multi-step advance_step to ROCmFlashAttentionMetadata (#8474) William LinandGitHub 2024-09-19 20:49:54 -07:00
• 18ae428a0d [Bugfix] Fix Phi3.5 mini and MoE LoRA inference (#8571) Amit GargandGitHub 2024-09-19 17:54:02 -07:00
• de6f90a13d [Misc] guard against change in cuda library name (#8609) bnellnmandGitHub 2024-09-19 18:36:30 -04:00
• 6cb748e190 [CI/Build] Re-enabling Entrypoints tests on ROCm, excluding ones that fail (#8551) Alexey Kondratiev(AMD)andGitHub 2024-09-19 16:06:32 -04:00
• 9e99407e3c Create SECURITY.md (#8642) Simon MoandGitHub 2024-09-19 12:16:28 -07:00
• ea4647b7d7 [Doc] Add documentation for GGUF quantization (#8618) Isotr0pyandGitHub 2024-09-20 03:15:55 +08:00
• e42c634acb [Core] simplify logits resort in _apply_top_k_top_p (#8619) 盏一andGitHub 2024-09-20 02:28:25 +08:00
• 9cc373f390 [Kernel][Amd] Add fp8 kv cache support for rocm custom paged attention (#8577) Charlie FuandGitHub 2024-09-19 12:37:57 -05:00
• 76515f303b [Frontend] Use MQLLMEngine for embeddings models too (#8584) Nick HillandGitHub 2024-09-19 17:51:06 +01:00
• 855c8ae2c9 [MISC] remove engine_use_ray in benchmark_throughput.py (#8615) Kunshang JiandGitHub 2024-09-19 13:33:20 +08:00
• c52ec5f034 [Bugfix] fixing sonnet benchmark bug in benchmark_serving.py (#8616) Kuntai DuandGitHub 2024-09-18 22:24:24 -07:00
• 02c9afa2d0 Revert "[Misc][Bugfix] Disable guided decoding for mistral tokenizer" (#8593) Roger WangandGitHub 2024-09-18 21:14:28 -07:00
• 3118f63385 [Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction during decode of encoder-decoder models. (#8545) sroy745andGitHub 2024-09-18 19:24:15 -07:00
• 4c34ce8916 [Kernel] Remove marlin moe templating on thread_m_blocks (#8573) 3 people 2024-09-18 21:42:49 -04:00
• 0d47bf3bf4 [Bugfix] add dead_error property to engine client (#8574) Joe RundeandGitHub 2024-09-18 16:10:01 -06:00
• d9cd78eb71 [BugFix] Nonzero exit code if MQLLMEngine startup fails (#8572) Nick HillandGitHub 2024-09-18 21:17:55 +01:00
• db9120cded [Kernel] Change interface to Mamba selective_state_update for continuous batching (#8039) Tyler Michael SmithandGitHub 2024-09-18 16:05:06 -04:00
• b3195bc9e4 [AMD][ROCm]Quantization methods on ROCm; Fix _scaled_mm call (#8380) 4 people 2024-09-18 13:41:08 -04:00
• e18749ff09 [Model] Support Solar Model (#8386) 3 people 2024-09-19 02:04:00 +09:00
• d65798f78c [Core] zmq: bind only to 127.0.0.1 for local-only usage (#8543) Russell BryantandGitHub 2024-09-18 12:10:27 -04:00
• a8c1d161a7 [Core] *Prompt* logprobs support in Multi-step (#8199) afeldman-nmandGitHub 2024-09-18 11:38:43 -04:00
• 7c7714d856 [Core][Bugfix][Perf] Introduce MQLLMEngine to avoid asyncio OH (#8157) 6 people 2024-09-18 09:56:58 -04:00
• 9d104b5beb [CI/Build] Update Ruff version (#8469) 3 people 2024-09-18 07:00:56 -04:00
• 6ffa3f314c [CI/Build] Avoid CUDA initialization (#8534) Cyrus LeungandGitHub 2024-09-18 18:38:11 +08:00
• e351572900 [Misc] Add argument to disable FastAPI docs (#8554) Jiaxin ShanandGitHub 2024-09-18 02:51:59 -07:00
• 95965d31b6 [CI/Build] fix Dockerfile.cpu on podman (#8540) DanieleandGitHub 2024-09-18 04:49:53 +02:00