| 1 | # b12x Linear and MoE Backends |
| 2 | |
| 3 | [b12x](https://pypi.org/project/b12x/) provides optional CUDA kernels for |
| 4 | NVIDIA SM120 and SM121 GPUs. Install the dependency with: |
| 5 | |
| 6 | ```bash |
| 7 | uv pip install "vllm[b12x]" |
| 8 | ``` |
| 9 | |
| 10 | b12x linear kernels participate in automatic selection after established |
| 11 | optimized backends and before emulation. Select the linear and MoE backends |
| 12 | explicitly with: |
| 13 | |
| 14 | ```bash |
| 15 | vllm serve <model> \ |
| 16 | --linear-backend b12x \ |
| 17 | --moe-backend b12x |
| 18 | ``` |
| 19 | |
| 20 | Only pass `--moe-backend b12x` for a compatible NVFP4 or MXFP4 MoE model. The |
| 21 | linear and MoE backends can be selected independently. |
| 22 | |
| 23 | b12x uses MXFP8 activations by default for MXFP4 MoE and the checkpoint's |
| 24 | activation format for NVFP4 MoE. MXFP4 falls back to BF16 when its A8 path does |
| 25 | not support the model configuration. Set `VLLM_B12X_MOE_FP4_FORCE_A16=1` to |
| 26 | force BF16 activations for either FP4 weight format. |
| 27 | |
| 28 | ## Supported Configurations |
| 29 | |
| 30 | | Backend | Supported configurations | |
| 31 | | ------- | ------------------------ | |
| 32 | | Linear | Per-tensor FP8, 128x128 block FP8, MXFP8, NVFP4, and MXFP4 | |
| 33 | | MoE | Tensor-parallel MXFP4 weights with BF16 or MXFP8 activations; NVFP4 weights with BF16, NVFP4, or MXFP8 activations | |
| 34 | |
| 35 | Dense W4A16 layers are not handled by b12x and continue to use another |
| 36 | compatible backend such as Marlin. The b12x MoE backend does not support expert |
| 37 | parallelism, expert maps, EXL3, or NF3. |