A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 07:15 UTC
[doc] fix the list rendering issue - security.md (#18982)
Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>
Reid committed
May 31, 2025 at 18:39 UTC
749f5bdd3879e8e7ebce1c56f9960481a6cf63bb
1 file changed
+6
-6
docs/usage/security.md
+6
-6
@@ -12,14 +12,14 @@ All communications between nodes in a multi-node vLLM deployment are **insecure
12
13
The following options control inter-node communications in vLLM:
14
15
-1. **Environment Variables:**
15
+#### 1. **Environment Variables:**
16
- `VLLM_HOST_IP`: Sets the IP address for vLLM processes to communicate on
17
18
-2. **KV Cache Transfer Configuration:**
18
+#### 2. **KV Cache Transfer Configuration:**
19
- `--kv-ip`: The IP address for KV cache transfer communications (default: 127.0.0.1)
20
- `--kv-port`: The port for KV cache transfer communications (default: 14579)
21
22
-3. **Data Parallel Configuration:**
22
+#### 3. **Data Parallel Configuration:**
23
- `data_parallel_master_ip`: IP of the data parallel master (default: 127.0.0.1)
24
- `data_parallel_master_port`: Port of the data parallel master (default: 29500)
25
@@ -39,16 +39,16 @@ Key points from the PyTorch security guide:
39
40
### Security Recommendations
41
42
-1. **Network Isolation:**
42
+#### 1. **Network Isolation:**
43
- Deploy vLLM nodes on a dedicated, isolated network
44
- Use network segmentation to prevent unauthorized access
45
- Implement appropriate firewall rules
46
47
-2. **Configuration Best Practices:**
47
+#### 2. **Configuration Best Practices:**
48
- Always set `VLLM_HOST_IP` to a specific IP address rather than using defaults
49
- Configure firewalls to only allow necessary ports between nodes
50
51
-3. **Access Control:**
51
+#### 3. **Access Control:**
52
- Restrict physical and network access to the deployment environment
53
- Implement proper authentication and authorization for management interfaces
54
- Follow the principle of least privilege for all system components