A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 00:00 UTC
Document NixlConnector backend selection via kv_connector_extra_config (#33552)
Signed-off-by: KrxGu <krishom70@gmail.com>
Krish Gupta committed
Feb 3, 2026 at 19:19 UTC
2df2b3499dee2025f3f5aa12fb68ea07013c0aa7
1 file changed
+29
docs/features/nixl_connector_usage.md
+29
@@ -36,6 +36,35 @@ export UCX_NET_DEVICES=all # or specify network devices like "mlx5_0:1,mlx5_1:1
36
!!! tip
37
When using UCX as the transport backend, NCCL environment variables (like `NCCL_IB_HCA`, `NCCL_SOCKET_IFNAME`) are not applicable to NixlConnector, so configure UCX-specific environment variables instead of NCCL variables.
38
39
+#### Selecting a NIXL transport backend (plugin)
40
+
41
+NixlConnector can use different NIXL transport backends (plugins). By default, NixlConnector uses UCX as the transport backend.
42
+
43
+To select a different backend, set `kv_connector_extra_config.backends` in `--kv-transfer-config`.
44
+
45
+### Example: using LIBFABRIC backend
46
+
47
+```bash
48
+vllm serve <MODEL> \
49
+ --kv-transfer-config '{
50
+ "kv_connector":"NixlConnector",
51
+ "kv_role":"kv_both",
52
+ "kv_connector_extra_config":{"backends":["LIBFABRIC"]}
53
+ }'
54
+```
55
+
56
+You can also pass JSON keys individually using dotted arguments, and you can append list elements using `+`:
57
+
58
+```bash
59
+vllm serve <MODEL> \
60
+ --kv-transfer-config.kv_connector NixlConnector \
61
+ --kv-transfer-config.kv_role kv_both \
62
+ --kv-transfer-config.kv_connector_extra_config.backends+ LIBFABRIC
63
+```
64
+
65
+!!! note
66
+ Backend availability depends on how NIXL was built and what plugins are present in your environment. Refer to the [NIXL repository](https://github.com/ai-dynamo/nixl) for available backends and build instructions.
67
+
68
## Basic Usage (on the same host)
69
70
### Producer (Prefiller) Configuration