vllm/tests/compile/piecewise/piecewise_compilation_config.json at fe15729a2b77d760fcf99da76f15806c5eab33df - vllm - Gitea: Git with a cup of tea

wassname/vllm

mirror of https://github.com/wassname/vllm.git synced 2026-07-28 11:27:56 +08:00

Files

T

330e82d34a [v1][torch.compile] support managing cudagraph buffer (#10203 )

Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

2024-11-11 11:10:27 -08:00

5 lines

110 B

JSON

Raw Blame History

 {
     "use_cudagraph": true,
     "non_cudagraph_ops": ["silly.attention"],
     "cudagraph_copy_inputs": true
 }