@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / f05f3664c3

Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 4, 13:45 UTC

[Doc] Add missing API endpoints to security documentation (#40532)

Signed-off-by: Russell Bryant <rbryant@redhat.com>

Russell Bryant committed Apr 28, 2026 at 17:53 UTC f05f3664c35804bf2b5b64eecd17ddfdbb8ed5e3

1 file changed +26 -5

docs/usage/security.md

+26 -5

@@ -138,14 +138,22 @@ When `--api-key` is configured, the following `/v1` endpoints require Bearer tok

138

139 - `/v1/models` - List available models

140 - `/v1/chat/completions` - Chat completions

141 +- `/v1/chat/completions/batch` - Batch chat completions

142 +- `/v1/chat/completions/render` - Render chat completion requests

143 - `/v1/completions` - Text completions

144 +- `/v1/completions/render` - Render completion requests

145 - `/v1/embeddings` - Generate embeddings

146 - `/v1/audio/transcriptions` - Audio transcription

147 - `/v1/audio/translations` - Audio translation

148 - `/v1/messages` - Anthropic-compatible messages API

146 -- `/v1/responses` - Response management

149 +- `/v1/messages/count_tokens` - Count tokens for Anthropic messages

150 +- `/v1/responses` - Create a response

151 +- `/v1/responses/{response_id}` - Retrieve a response

152 +- `/v1/responses/{response_id}/cancel` - Cancel a response

153 - `/v1/score` - Scoring API

154 - `/v1/rerank` - Reranking API

155 +- `/v1/load_lora_adapter` - Load a LoRA adapter (can alter model behavior; only available when `--enable-lora` is set and `VLLM_ALLOW_RUNTIME_LORA_UPDATING=True`)

156 +- `/v1/unload_lora_adapter` - Unload a LoRA adapter (can alter model behavior; only available when `--enable-lora` is set and `VLLM_ALLOW_RUNTIME_LORA_UPDATING=True`)

157

158 ### Unprotected Endpoints (No API Key Required)

159

@@ -155,16 +163,23 @@ The following endpoints **do not require authentication** even when `--api-key`

163

164 - `/invocations` - SageMaker-compatible endpoint (routes to the same inference functions as `/v1` endpoints)

165 - `/inference/v1/generate` - Generate completions

166 +- `/generative_scoring` - Generative scoring API

167 - `/pooling` - Pooling API

168 - `/classify` - Classification API

169 - `/score` - Scoring API (non-`/v1` variant)

170 - `/rerank` - Reranking API (non-`/v1` variant)

171

163 -**Operational control endpoints (always enabled):**

172 +**Operational control endpoints (only when `"generate"` task is supported):**

173

174 - `/pause` - Pause generation (causes denial of service)

175 - `/resume` - Resume generation

176 +- `/is_paused` - Check if generation is paused

177 - `/scale_elastic_ep` - Trigger scaling operations

178 +- `/is_scaling_elastic_ep` - Check if scaling is in progress

179 +- `/init_weight_transfer_engine` - Initialize weight transfer engine for RLHF

180 +- `/update_weights` - Update model weights (can alter model behavior)

181 +- `/get_world_size` - Get distributed world size

182 +- `/abort_requests` - Abort in-flight requests (only when `--tokens-only` is also set)

183

184 **Utility endpoints:**

185

@@ -207,9 +222,9 @@ These endpoints are only available when profiling is enabled and should only be

222

223 An attacker who can reach the vLLM HTTP server can:

224

210 -1. **Bypass authentication** by using non-`/v1` endpoints like `/invocations`, `/inference/v1/generate`, `/pooling`, `/classify`, `/score`, or `/rerank` to run arbitrary inference without credentials

211 -2. **Cause denial of service** by calling `/pause` or `/scale_elastic_ep` without a token

212 -3. **Access operational controls** to manipulate server state (e.g., pausing generation)

225 +1. **Bypass authentication** by using non-`/v1` endpoints like `/invocations`, `/inference/v1/generate`, `/generative_scoring`, `/pooling`, `/classify`, `/score`, or `/rerank` to run arbitrary inference without credentials

226 +2. **Cause denial of service** by calling `/pause`, `/scale_elastic_ep`, or `/abort_requests` without a token

227 +3. **Access operational controls** to manipulate server state (e.g., pausing generation, updating model weights via `/update_weights`)

228 4. **If `--enable-tokenizer-info-endpoint` is set:** Access sensitive tokenizer configuration including chat templates, which may reveal prompt engineering strategies or other implementation details

229 5. **If `VLLM_SERVER_DEV_MODE=1` is set:** Execute arbitrary RPC commands via `/collective_rpc`, reset caches, put the engine to sleep, and access detailed server configuration

230

@@ -288,6 +303,12 @@ To disable the Python code interpreter specifically, omit `code_interpreter` fro

303

304 **Consider a custom implementation**: The GPT-OSS Python tool is a reference implementation. For production deployments, consider implementing a custom code execution sandbox with stricter isolation guarantees. See the [GPT-OSS documentation](https://github.com/openai/gpt-oss?tab=readme-ov-file#python) for guidance.

305

306 +## Dynamic LoRA Loading

307 +

308 +vLLM supports dynamically loading and unloading LoRA adapters at runtime via the `/v1/load_lora_adapter` and `/v1/unload_lora_adapter` API endpoints. This functionality is **not enabled by default** — it requires both `--enable-lora` and the environment variable `VLLM_ALLOW_RUNTIME_LORA_UPDATING=True` to be set.

309 +

310 +**Warning:** Dynamic LoRA loading is not a secure operation and should not be enabled in deployments exposed to untrusted clients. If you must enable dynamic LoRA loading, restrict access to the `/v1/load_lora_adapter` and `/v1/unload_lora_adapter` endpoints to trusted administrators only, using a reverse proxy or network-level access controls. Do not expose these endpoints to end users. For details on configuring LoRA adapters, see the [LoRA Adapters documentation](../features/lora.md).

311 +

312 ## Reporting Security Vulnerabilities

313

314 If you believe you have found a security vulnerability in vLLM, please report it following the project's security policy. For more information on how to report security issues and the project's security policy, please see the [vLLM Security Policy](https://github.com/vllm-project/vllm/blob/main/SECURITY.md).