A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 11:00 UTC
[Docs] Expand security doc with firewall info (#18081)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Russell Bryant committed
May 13, 2025 at 15:36 UTC
0189a65a2e76b94ee4cc872dbad2b732256e0e9a
1 file changed
+39
docs/source/deployment/security.md
+39
@@ -53,6 +53,45 @@ Key points from the PyTorch security guide:
53
- Implement proper authentication and authorization for management interfaces
54
- Follow the principle of least privilege for all system components
55
56
+## Security and Firewalls: Protecting Exposed vLLM Systems
57
+
58
+While vLLM is designed to allow unsafe network services to be isolated to
59
+private networks, there are components—such as dependencies and underlying
60
+frameworks—that may open insecure services listening on all network interfaces,
61
+sometimes outside of vLLM's direct control.
62
+
63
+A major concern is the use of `torch.distributed`, which vLLM leverages for
64
+distributed communication, including when using vLLM on a single host. When vLLM
65
+uses TCP initialization (see [PyTorch TCP Initialization
66
+documentation](https://docs.pytorch.org/docs/stable/distributed.html#tcp-initialization)),
67
+PyTorch creates a `TCPStore` that, by default, listens on all network
68
+interfaces. This means that unless additional protections are put in place,
69
+these services may be accessible to any host that can reach your machine via any
70
+network interface.
71
+
72
+**From a PyTorch perspective, any use of `torch.distributed` should be
73
+considered insecure by default.** This is a known and intentional behavior from
74
+the PyTorch team.
75
+
76
+### Firewall Configuration Guidance
77
+
78
+The best way to protect your vLLM system is to carefully configure a firewall to
79
+expose only the minimum network surface area necessary. In most cases, this
80
+means:
81
+
82
+- **Block all incoming connections except to the TCP port the API server is
83
+listening on.**
84
+
85
+- Ensure that ports used for internal communication (such as those for
86
+`torch.distributed` and KV cache transfer) are only accessible from trusted
87
+hosts or networks.
88
+
89
+- Never expose these internal ports to the public internet or untrusted
90
+networks.
91
+
92
+Consult your operating system or application platform documentation for specific
93
+firewall configuration instructions.
94
+
95
## Reporting Security Vulnerabilities
96
97
If you believe you have found a security vulnerability in vLLM, please report it following the project's security policy. For more information on how to report security issues and the project's security policy, please see the [vLLM Security Policy](https://github.com/vllm-project/vllm/blob/main/SECURITY.md).