A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 12:15 UTC
[Docs] Add a security guide (#17230)
Signed-off-by: Russell Bryant <rbryant@redhat.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Russell Bryant committed
Apr 28, 2025 at 11:12 UTC
72dfe4c74f507b5ad02afd757d11f6581cba9f3c
2 files changed
+59
docs/source/deployment/security.md
new
+58
@@ -0,0 +1,58 @@
1
+# Security Guide
2
+
3
+## Inter-Node Communication
4
+
5
+All communications between nodes in a multi-node vLLM deployment are **insecure by default** and must be protected by placing the nodes on an isolated network. This includes:
6
+
7
+1. PyTorch Distributed communications
8
+2. KV cache transfer communications
9
+3. Tensor, Pipeline, and Data parallel communications
10
+
11
+### Configuration Options for Inter-Node Communications
12
+
13
+The following options control inter-node communications in vLLM:
14
+
15
+1. **Environment Variables:**
16
+ - `VLLM_HOST_IP`: Sets the IP address for vLLM processes to communicate on
17
+
18
+2. **KV Cache Transfer Configuration:**
19
+ - `--kv-ip`: The IP address for KV cache transfer communications (default: 127.0.0.1)
20
+ - `--kv-port`: The port for KV cache transfer communications (default: 14579)
21
+
22
+3. **Data Parallel Configuration:**
23
+ - `data_parallel_master_ip`: IP of the data parallel master (default: 127.0.0.1)
24
+ - `data_parallel_master_port`: Port of the data parallel master (default: 29500)
25
+
26
+### Notes on PyTorch Distributed
27
+
28
+vLLM uses PyTorch's distributed features for some inter-node communication. For
29
+detailed information about PyTorch Distributed security considerations, please
30
+refer to the [PyTorch Security
31
+Guide](https://github.com/pytorch/pytorch/security/policy#using-distributed-features).
32
+
33
+Key points from the PyTorch security guide:
34
+- PyTorch Distributed features are intended for internal communication only
35
+- They are not built for use in untrusted environments or networks
36
+- No authorization protocol is included for performance reasons
37
+- Messages are sent unencrypted
38
+- Connections are accepted from anywhere without checks
39
+
40
+### Security Recommendations
41
+
42
+1. **Network Isolation:**
43
+ - Deploy vLLM nodes on a dedicated, isolated network
44
+ - Use network segmentation to prevent unauthorized access
45
+ - Implement appropriate firewall rules
46
+
47
+2. **Configuration Best Practices:**
48
+ - Always set `VLLM_HOST_IP` to a specific IP address rather than using defaults
49
+ - Configure firewalls to only allow necessary ports between nodes
50
+
51
+3. **Access Control:**
52
+ - Restrict physical and network access to the deployment environment
53
+ - Implement proper authentication and authorization for management interfaces
54
+ - Follow the principle of least privilege for all system components
55
+
56
+## Reporting Security Vulnerabilities
57
+
58
+If you believe you have found a security vulnerability in vLLM, please report it following the project's security policy. For more information on how to report security issues and the project's security policy, please see the [vLLM Security Policy](https://github.com/vllm-project/vllm/blob/main/SECURITY.md).
docs/source/index.md
+1
@@ -132,6 +132,7 @@ serving/integrations/index
132
:caption: Deployment
133
:maxdepth: 1
134
135
+deployment/security
136
deployment/docker
137
deployment/k8s
138
deployment/nginx