Cyrus LeungandGitHub f230cc2ca6 [Bugfix] Fix broadcasting logic for multi_modal_kwargs (#6836)

2024-07-31 10:38:45 +08:00

..

__init__.py

[Bugfix] Add __init__.py files for vllm/core/block/ and vllm/spec_decode/ (#3798)

2024-04-02 12:35:31 -07:00

batch_expansion.py

[BugFix] Fix use of per-request seed with pipeline parallel (#6698)

2024-07-30 10:40:08 -07:00

draft_model_runner.py

[Bugfix] Fix broadcasting logic for multi_modal_kwargs (#6836)

2024-07-31 10:38:45 +08:00

interfaces.py

[BUGFIX] Raise an error for no draft token case when draft_tp>1 (#6369)

2024-07-19 06:01:09 -07:00

medusa_worker.py

[BugFix] Fix use of per-request seed with pipeline parallel (#6698)

2024-07-30 10:40:08 -07:00

metrics.py

[Bugfix] [SpecDecode] AsyncMetricsCollector: update time since last collection (#6578)

2024-07-19 14:01:03 -07:00

mlp_speculator_worker.py

[BugFix] Fix use of per-request seed with pipeline parallel (#6698)

2024-07-30 10:40:08 -07:00

multi_step_worker.py

[Core] draft_model_runner: Implement prepare_inputs on GPU for advance_step (#6338)

2024-07-17 14:30:28 -07:00

ngram_worker.py

[BugFix] Fix use of per-request seed with pipeline parallel (#6698)

2024-07-30 10:40:08 -07:00

proposer_worker_base.py

[Misc][Speculative decoding] Typos and typing fixes (#6467)

2024-07-17 07:17:07 +00:00

smaller_tp_proposer_worker.py

[Speculative Decoding] Enabling bonus token in speculative decoding for KV cache based models (#5765)

2024-07-10 16:02:47 -07:00

spec_decode_worker.py

[BugFix] Fix use of per-request seed with pipeline parallel (#6698)

2024-07-30 10:40:08 -07:00

target_model_runner.py

[Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both draft and target models. (#6485)

2024-07-20 23:58:58 -07:00

top1_proposer.py

[BUGFIX] Raise an error for no draft token case when draft_tp>1 (#6369)

2024-07-19 06:01:09 -07:00

util.py

[Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both draft and target models. (#6485)

2024-07-20 23:58:58 -07:00