[Speculative decoding 7/9] Speculative decoding end-to-end correctness tests. (#3951)

2026-08-06 13:40:27 +08:00 · 2024-04-23 08:02:36 +00:00
parent 050f285ff6
commit 62b8aebc6f
22 changed files with 1164 additions and 175 deletions
@@ -83,6 +83,7 @@ class GPUExecutor(ExecutorBase):
            scheduler_config=self.scheduler_config,
            device_config=self.device_config,
            cache_config=self.cache_config,
+            # TODO allow draft-model specific load config.
            load_config=self.load_config,
            local_rank=0,
            rank=0,