@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 43b05fa314
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 05:45 UTC

[torch.compile][misc] fix comments (#10993)

Signed-off-by: youkaichao <youkaichao@gmail.com>

youkaichao committed Dec 8, 2024 at 11:18 UTC 43b05fa314e90e551d87211e8bdde2e2bb5a0bdc
1 file changed +2 -2
vllm/config.py
+2 -2
@@ -2177,8 +2177,8 @@ class CompilationConfig(BaseModel):
2177 TODO: move outside cudagraph logic into compilation.
2178 torch.compile will handle cudagraph capture logic in the future.
2179 - cudagraph_capture_sizes: sizes to capture cudagraph.
2180 - - None: capture sizes are inferred from compilation context.
2181 - - List[int]: capture sizes are specified.
2180 + - None (default): capture sizes are inferred from vllm config.
2181 + - List[int]: capture sizes are specified as given.
2182 - cudagraph_num_of_warmups: number of warmup runs for cudagraph.
2183 It means the first several runs will be treated as warmup runs.
2184 Only after that, the execution will be recorded, and the recorded