A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 11, 07:30 UTC
[Docs] fix function name (#30748)
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
rongfu.leng committed
Dec 17, 2025 at 20:14 UTC
9e67c4ce985b0b8852603cfe3fcaf8f37de137ed
1 file changed
+1
-1
docs/design/plugin_system.md
+1
-1
@@ -109,7 +109,7 @@ Every plugin has three parts:
109
- `init_device`: This function is called to set up the device for the worker.
110
- `initialize_cache`: This function is called to set cache config for the worker.
111
- `load_model`: This function is called to load the model weights to device.
112
- - `get_kv_cache_spaces`: This function is called to generate the kv cache spaces for the model.
112
+ - `get_kv_cache_spec`: This function is called to generate the kv cache spec for the model.
113
- `determine_available_memory`: This function is called to profiles the peak memory usage of the model to determine how much memory can be used for KV cache without OOMs.
114
- `initialize_from_config`: This function is called to allocate device KV cache with the specified kv_cache_config
115
- `execute_model`: This function is called every step to inference the model.