Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• cd5601ac37 [BugFix] Prevent exporting duplicate OpenTelemetry spans (#9017) Ronen SchafferandGitHub 2024-10-22 21:11:53 +03:00
• 434984e665 [Frontend] Support custom request_id from request (#9550) 3 people 2024-10-23 02:07:30 +08:00
• 32a1ee74a0 [Hardware][Intel CPU][DOC] Update docs for CPU backend (#6212) 5 people 2024-10-22 10:38:04 -07:00
• 08075c3448 [Bugfix] Eagle: change config name for fc bias (#9580) gopalsardaandGitHub 2024-10-22 21:44:22 +05:30
• bb392ea2d2 [Model][VLM] Initialize support for Mono-InternVL model (#9528) Isotr0pyandGitHub 2024-10-23 00:01:46 +08:00
• 9dbcce84a7 [Neuron] [Bugfix] Fix neuron startup (#9374) 3 people 2024-10-22 14:51:41 +02:00
• a48e3ec052 [CI/Build][LoRA] Temporarily fix long context failure issue (#9579) Jee Jee LiandGitHub 2024-10-22 19:32:51 +08:00
• 6c5af09b39 [V1] Implement vLLM V1 [1/N] (#9289) Woosuk KwonandGitHub 2024-10-22 01:24:07 -07:00
• 3ddbe25502 [Hardware][CPU] using current_platform.is_cpu (#9536) wangshuai09andGitHub 2024-10-22 15:50:43 +08:00
• 0d02747f2e support TP in qwen2 bnb (#9574) chenqianfzhandGitHub 2024-10-22 00:13:23 -07:00
• f7db5f0fa9 [Doc] Use shell code-blocks and fix section headers (#9508) Rafael VasquezandGitHub 2024-10-22 02:43:24 -04:00
• ca30c3c84b [Core] Remove evictor_v1 (#9572) Kuntai DuandGitHub 2024-10-21 23:55:49 -05:00
• c0292211ce [CI/Build] Replaced some models on tests for smaller ones (#9570) Wallas HenriqueandGitHub 2024-10-22 01:52:14 -03:00
• 74692421f7 [Bugfix]: phi.py get rope_theta from config file (#9503) 3 people 2024-10-22 04:53:36 +02:00
• 29acd2c34c [Bugfix][OpenVINO] fix_dockerfile_openvino (#9552) ngrozaeandGitHub 2024-10-22 04:47:52 +02:00
• f085995a7b [CI/Build] Remove unnecessary fork_new_process (#9484) Cyrus LeungandGitHub 2024-10-22 10:47:29 +08:00
• b729901139 [Bugfix]: serialize config by value for --trust-remote-code (#6751) 3 people 2024-10-21 20:46:24 -06:00
• 76a5e13270 [core] move parallel sampling out from vllm core (#9302) youkaichaoandGitHub 2024-10-21 17:31:44 -07:00
• ef7faad1b8 🐛 Fixup more test failures from memory profiling (#9563) Joe RundeandGitHub 2024-10-21 19:10:56 -05:00
• 575dcebe9a [CI] Make format checker error message more user-friendly by using emoji (#9564) Kuntai DuandGitHub 2024-10-21 18:45:15 -05:00
• 711f3a7806 [Frontend] Don't log duplicate error stacktrace for every request in the batch (#9023) Wallas HenriqueandGitHub 2024-10-21 18:49:41 -03:00
• 15713e3b75 [BugFix] Update draft model TP size check to allow matching target TP size (#9394) 3 people 2024-10-21 22:14:29 +01:00
• d621c43df7 [doc] fix format (#9562) youkaichaoandGitHub 2024-10-21 13:54:57 -07:00
• 9d9186be97 [Frontend] Reduce frequency of client cancellation checking (#7959) Nick HillandGitHub 2024-10-21 21:28:10 +01:00
• 5241aa1494 [Model][Bugfix] Fix batching with multi-image in PixtralHF (#9518) Michael GoinandGitHub 2024-10-21 14:20:07 -04:00
• ec6bd6c4c6 [BugFix] Use correct python3 binary in Docker.ppc64le entrypoint (#9492) Varad AhirwadkarandGitHub 2024-10-21 23:13:02 +05:30
• 8ca8954841 [Bugfix][Misc]: fix graph capture for decoder (#9549) yudian0504andGitHub 2024-10-22 01:33:30 +08:00
• f6b97293aa [Model] FalconMamba Support (#9325) Dhia Eddine RhaiemandGitHub 2024-10-21 20:50:16 +04:00
• 496e991da8 [Doc] Consistent naming of attention backends (#9498) Thomas ParnellandGitHub 2024-10-21 16:29:57 +02:00
• 696b01af8f [CI/Build] Split up decoder-only LM tests (#9488) 3 people 2024-10-21 12:27:50 +08:00
• 855e0e6f97 [Frontend][Misc] Goodput metric support (#9338) Andy DaiandGitHub 2024-10-20 11:39:32 -07:00
• 4fa3e33349 [Kernel] Support sliding window in flash attention backend (#9403) Chen ZhangandGitHub 2024-10-20 10:57:52 -07:00
• 962d2c6349 [Model][Pixtral] Use memory_efficient_attention for PixtralHFVision (#9520) Michael GoinandGitHub 2024-10-20 01:29:14 -04:00
• 5b59fe0f08 [Bugfix] Pass json-schema to GuidedDecodingParams and make test stronger (#9530) Chen ZhangandGitHub 2024-10-19 17:05:02 -07:00
• 8e3e7f2713 [Model][Pixtral] Optimizations for input_processor_for_pixtral_hf (#9514) Michael GoinandGitHub 2024-10-19 10:44:29 -04:00
• 263d8ee150 [Bugfix] Fix missing task for speculative decoding (#9524) Cyrus LeungandGitHub 2024-10-19 14:49:40 +08:00
• c5eea3c8ba [Frontend] Support simpler image input format (#9478) Yue ZhangandGitHub 2024-10-18 23:17:07 -07:00
• 85dc92fc98 [CI/Build] Configure matcher for actionlint workflow (#9511) Russell BryantandGitHub 2024-10-19 02:04:18 -04:00
• dfd951ed9b [CI/Build] Add error matching for ruff output (#9513) Russell BryantandGitHub 2024-10-19 01:42:20 -04:00
• 82c25151ec [Doc] update gpu-memory-utilization flag docs (#9507) Joe RundeandGitHub 2024-10-18 22:26:36 -05:00
• 1325872ec8 [Frontend] Avoid creating guided decoding LogitsProcessor unnecessarily (#9521) Nick HillandGitHub 2024-10-19 04:21:01 +01:00
• 380e18639f 🐛 fix torch memory profiling (#9516) Joe RundeandGitHub 2024-10-18 20:25:19 -05:00
• 337ed76671 [Bugfix] Fix offline mode when using mistral_common (#9457) sasha0552andGitHub 2024-10-19 01:12:32 +00:00
• 0c9a5258f9 [Kernel] Add env variable to force flashinfer backend to enable tensor cores (#9497) 4 people 2024-10-19 02:55:48 +02:00
• d11bf435a0 [MISC] Consolidate cleanup() and refactor offline_inference_with_prefix.py (#9510) Cody YuandGitHub 2024-10-18 14:30:55 -07:00
• 9bb10a7d27 [MISC] Add lora requests to metrics (#9477) 3 people 2024-10-18 13:50:18 -07:00
• 3921a2f29e [Model] Support Pixtral models in the HF Transformers format (#9036) Michael GoinandGitHub 2024-10-18 15:29:56 -04:00
• 67a7e5ef38 [CI/Build] Add error matching config for mypy (#9512) Russell BryantandGitHub 2024-10-18 15:17:53 -04:00
• 051eaf6db3 [Model] Add user-configurable task for models that support both generation and embedding (#9424) Cyrus LeungandGitHub 2024-10-19 02:31:58 +08:00
• 7dbe738d65 [Misc] benchmark: Add option to set max concurrency (#9390) Russell BryantandGitHub 2024-10-18 14:15:28 -04:00
• ae8b633ba3 [Bugfix] Fix offline_inference_with_prefix.py (#9505) Tyler Michael SmithandGitHub 2024-10-18 12:59:19 -04:00
• 1bbbcc0b1d [CI/Build] Fix lint errors in mistral tokenizer (#9504) Cyrus LeungandGitHub 2024-10-19 00:09:35 +08:00
• 25aeb7d4c9 [BugFix] Fix and simplify completion API usage streaming (#9475) Nick HillandGitHub 2024-10-18 15:10:26 +01:00
• d2b1bf55ec [Frontend][Feature] Add jamba tool parser (#9154) tomeras91andGitHub 2024-10-18 13:27:48 +03:00
• 1ffc8a7362 [BugFix] Typing fixes to RequestOutput.prompt and beam search (#9473) Nick HillandGitHub 2024-10-18 08:19:53 +01:00
• 944dd8edaf [CI/Build] Use commit hash references for github actions (#9430) Russell BryantandGitHub 2024-10-18 00:54:58 -04:00
• 154a8ae880 [Qwen2.5] Support bnb quant for Qwen2.5 (#9467) Haoyu WangandGitHub 2024-10-18 12:40:14 +08:00
• de4008e2ab [Bugfix][Core] Use torch.cuda.memory_stats() to profile peak memory usage (#9352) Joe RundeandGitHub 2024-10-17 21:47:27 -05:00
• 48138a8415 [BugFix] Stop silent failures on compressed-tensors parsing (#9381) Dipika SikkaandGitHub 2024-10-17 21:54:00 -04:00
• 343f8e0905 Support BERTModel (first encoder-only embedding model) (#9056) 9 people 2024-10-17 19:21:01 -04:00
• bb76538bbd [Hardwware][Neuron] Simplify model load for transformers-neuronx library (#9380) Shashwat SrijanandGitHub 2024-10-17 15:39:39 -07:00
• d615b5c9f8 [Bugfix] Print warnings related to mistral_common tokenizer only once (#9468) sasha0552andGitHub 2024-10-17 21:44:20 +00:00
• d65049daab [Bugfix] Add random_seed to sample_hf_requests in benchmark_serving script (#9013) 3 people 2024-10-17 14:11:11 -07:00
• eca2c5f7c0 [Bugfix] Fix support for dimension like integers and ScalarType (#9299) bnellnmandGitHub 2024-10-17 15:08:34 -04:00
• 0f41fbe5a3 [torch.compile] Fine-grained CustomOp enabling mechanism (#9300) Luka GovedičandGitHub 2024-10-17 14:36:37 -04:00
• 7871659abb [Misc] Remove commit id file (#9470) Cyrus LeungandGitHub 2024-10-18 01:34:37 +08:00
• a2c71c5405 [CI/Build] remove .github from .dockerignore, add dirty repo check (#9375) DanieleandGitHub 2024-10-17 19:25:06 +02:00
• 81ede99ca4 [Core] Deprecating block manager v1 and make block manager v2 default (#8704) Kuntai DuandGitHub 2024-10-17 11:38:15 -05:00
• 5eda21e773 [Hardware][CPU] compressed-tensor INT8 W8A8 AZP support (#9344) Li, JiangandGitHub 2024-10-18 00:21:04 +08:00
• 8e1cddcd44 [TPU] Call torch._sync(param) during weight loading (#9437) Woosuk KwonandGitHub 2024-10-17 09:00:11 -07:00
• 5e443b594f [Bugfix] Allow prefill of assistant response when using mistral_common (#9446) sasha0552andGitHub 2024-10-17 15:06:37 +00:00
• 9d30a056e7 [misc] CUDA Time Layerwise Profiler (#8337) 4 people 2024-10-17 10:36:09 -04:00
• 390be74649 [Misc] Print stack trace using logger.exception (#9461) Cyrus LeungandGitHub 2024-10-17 21:55:48 +08:00
• e312e52b44 [Kernel] Add Exllama as a backend for compressed-tensors (#9395) Lucas WilkinsonandGitHub 2024-10-17 09:48:26 -04:00
• dbfa8d31d5 Add notes on the use of Slack (#9442) Yuan TangandGitHub 2024-10-17 00:46:46 -04:00
• 92d86da217 [BugFix] [Kernel] Fix GPU SEGV occurring in int8 kernels (#9391) rasmithandGitHub 2024-10-16 20:34:06 -05:00
• c3fab5f769 [Bugfix][Kernel] Prevent integer overflow in fp8 dynamic per-token quantize kernel (#9425) Tyler Michael SmithandGitHub 2024-10-16 19:46:06 -04:00
• 776dbd74f1 [CI/Build] mypy: Resolve some errors from checking vllm/engine (#9267) Russell BryantandGitHub 2024-10-16 18:55:59 -04:00
• 8345045833 [Performance][Spec Decode] Optimize ngram lookup performance (#9333) Lily LiuandGitHub 2024-10-16 12:37:45 -07:00
• 5b8a1fde84 [Model][Bugfix] Add FATReLU activation and support for openbmb/MiniCPM-S-1B-sft (#9396) Junhao LiandGitHub 2024-10-16 12:40:24 -04:00
• fb60ae9b91 [Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189) Mor ZusmanandGitHub 2024-10-17 00:12:43 +08:00
• 415f76a9cb Support mistral interleaved attn (#9414) Patrick von PlatenandGitHub 2024-10-16 15:28:30 +02:00
• cf1d62a644 [Model] Support SDPA attention for Molmo vision backbone (#9410) Isotr0pyandGitHub 2024-10-16 19:52:01 +08:00
• 59230ef32b [Misc] Consolidate example usage of OpenAI client for multimodal models (#9412) 3 people 2024-10-16 04:20:51 -07:00
• cee711fdbb [Core] Rename input data types (#8688) Cyrus LeungandGitHub 2024-10-16 18:49:37 +08:00
• 1de76a0e55 [CI/Build] Test VLM embeddings (#9406) Cyrus LeungandGitHub 2024-10-16 17:44:30 +08:00
• 7abba39ee6 [Model] VLM2Vec, the first multimodal embedding model in vLLM (#9303) Cyrus LeungandGitHub 2024-10-16 14:31:00 +08:00
• 7e7eae338d [Misc] Standardize RoPE handling for Qwen2-VL (#9250) Cyrus LeungandGitHub 2024-10-16 13:56:17 +08:00
• ed920135c8 [Bugfix] Molmo text-only input bug fix (#9397) 5 people 2024-10-15 21:56:09 -07:00
• 717a5f82cd [Bugfix][CI/Build] Fix CUDA 11.8 Build (#9386) Lucas WilkinsonandGitHub 2024-10-15 20:15:21 -04:00
• ba30942240 [Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034) 3 people 2024-10-15 15:40:43 -07:00
• 22f8a69549 [Misc] Directly use compressed-tensors for checkpoint definitions (#8909) 3 people 2024-10-15 18:40:25 -04:00
• 5d264f4ab8 pass ignore_eos parameter to all benchmark_serving calls (#9349) Grace HoandGitHub 2024-10-15 13:30:44 -07:00
• e9d517f276 [BugFix] Fix chat API continuous usage stats (#9357) Nick HillandGitHub 2024-10-15 07:19:48 +01:00
• 55e081fbad [Bugfix] Update InternVL input mapper to support image embeds (#9351) hhzhang16andGitHub 2024-10-14 21:29:19 -07:00
• 8e836d982a [Doc] Fix code formatting in spec_decode.rst (#9348) Michael GoinandGitHub 2024-10-15 00:29:11 -04:00
• 44eaa5a5d9 [Frontend] Clarify model_type error messages (#9345) Steve GrubbandGitHub 2024-10-15 00:29:01 -04:00
• 169b530607 [Bugfix] Clean up some cruft in mamba.py (#9343) Tyler Michael SmithandGitHub 2024-10-14 20:24:25 -04:00
• f0fe4fe86d [Model] Make llama3.2 support multiple and interleaved images (#9095) Xiang XuandGitHub 2024-10-14 15:24:26 -07:00
• 4d31cd424b [Frontend] merge beam search implementations (#9296) Brendan WongandGitHub 2024-10-14 15:05:52 -07:00