@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 53be4a8634

Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 15, 19:45 UTC

[V1] Allow sliding window + prefix caching (#13069)

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Woosuk Kwon committed Mar 12, 2025 at 11:21 UTC 53be4a863486d02bd96a59c674bbec23eec508f6

1 file changed +1 -1

vllm/config.py

+1 -1

@@ -1146,7 +1146,7 @@ class CacheConfig:

1146 if not self.enable_prefix_caching:

1147 return

1148

1149 - if self.sliding_window is not None:

1149 + if self.sliding_window is not None and not envs.VLLM_USE_V1:

1150 raise NotImplementedError(

1151 "Prefix caching is not supported with sliding window. "

1152 "Run with --disable-sliding-window to use prefix caching.")