@tduncan / Nvidia-Optimization-Stack-VLLM / commits / e83d57d8ae

[Doc] Sync KV event medium terminology after #48123 (#51646)

Signed-off-by: Alex <jihui.huang@daocloud.io> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Or Ozeri <oro@il.ibm.com>

AlexHuang committed Sep 10, 2026 at 19:42 UTC e83d57d8ae14e240e2779c37db58b6dacd21bab2

1 file changed +5 -5

docs/features/kv_offloading_usage.md

+5 -5

@@ -117,9 +117,9 @@ Then set `"eviction_policy": "my_policy"` in `kv_connector_extra_config`, the sa

117

118 Each entry in `secondary_tiers` is a dict with a required `type` field plus tier-specific fields.

119

120 -The filesystem and object-store tiers can publish hash-only `BlockStored` KV events for blocks they successfully store, tagged with a stable per-tier `medium` (`FS` for the filesystem tier, `OBJ` for the object-store tier). Set `enable_kv_events: true` in the tier's entry to opt in; events are published only when KV cache events are also enabled globally via `--kv-events-config`.

120 +The filesystem and object-store tiers can publish hash-only `BlockStored` KV events for blocks they successfully store. Both tiers use the coarse-grained wire medium value `STORAGE`; the medium does not distinguish filesystem from object-store storage. To recover location semantics, set the optional `locality` field (`LOCAL` / `REMOTE`) on the tier entry — that field, not the medium, tells consumers whether the tier's blocks are local to the publishing vLLM instance. Set `enable_kv_events: true` in the tier's entry to opt in; events are published only when KV cache events are also enabled globally via `--kv-events-config`.

121

122 -Set the optional `locality` tier field to `LOCAL` or `REMOTE` to describe the tier's storage location relative to the publishing vLLM instance. `LOCAL` marks storage local to that instance, while `REMOTE` marks storage that is not local to it. When the setting is omitted, locality is unspecified. vLLM does not infer it from the tier type, so an OBJ tier is not implicitly `REMOTE`. A KV event includes `locality` only when the tier explicitly configures it. This metadata describes the tier property without implying that a consumer can already route requests to its blocks.

122 +Set the optional `locality` tier field to `LOCAL` or `REMOTE` to describe the tier's storage location relative to the publishing vLLM instance. `LOCAL` marks storage local to that instance, while `REMOTE` marks storage that is not local to it. When the setting is omitted, locality is unspecified. vLLM does not infer it from the tier type, so an `obj` tier is not implicitly `REMOTE`. A KV event includes `locality` only when the tier explicitly configures it. This metadata describes the tier property without implying that a consumer can already route requests to its blocks.

123

124 ### Filesystem (FS)

125

@@ -131,7 +131,7 @@ The filesystem tier (`type: "fs"`) writes blocks to a filesystem directory.

131 | `root_dir` | yes | — | Base directory; vLLM creates subdirectories beneath it (see [On-Disk Layout](#on-disk-layout)). |

132 | `n_read_threads` | no | `16` | Read-priority I/O threads (load path). |

133 | `n_write_threads` | no | `16` | Write-priority I/O threads (store path). |

136

137 Each thread group prefers its own queue but pulls from the other when its primary queue is empty, so a write-heavy or read-heavy burst won't leave the off-priority queue waiting. Size the totals to your storage's effective concurrency.

@@ -174,8 +174,8 @@ The object-store tier (`type: "obj"`) offloads blocks to an S3-compatible object

174 | `store_config` | yes | — | Object store connection parameters (see below). |

175 | `prefix` | no | `""` | Key prefix prepended to all object keys. |

176 | `io_threads` | no | `4` | Number of NIXL OBJ backend I/O threads. |

179

180 `store_config` fields:

181