A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 6, 00:45 UTC
[Docs] Adds vllm-musa to custom_op.md (#37840)
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
R0CKSTAR committed
Mar 25, 2026 at 19:54 UTC
242c93f74494e32569cc836d5075a88ccddc53a7
1 file changed
+2
-2
docs/design/custom_op.md
+2
-2
@@ -266,7 +266,7 @@ Currently, thanks to [vLLM's hardware-plugin mechanism](./plugin_system.md), the
266
267
- **Official device plugins:** [vllm-ascend](https://github.com/vllm-project/vllm-ascend) (for Huawei Ascend NPU), [vllm-spyre](https://github.com/vllm-project/vllm-spyre)
268
(for Spyre), [vllm-gaudi](https://github.com/vllm-project/vllm-gaudi) (for Intel Gaudi), [vllm-neuron](https://github.com/vllm-project/vllm-neuron) (for AWS Neuron), [vllm-meta](https://github.com/vllm-project/vllm-metal) (for Apple Silicon), etc.
269
-- **Non-official device plugins:** [vllm-metax](https://github.com/MetaX-MACA/vLLM-metax) (for MetaX GPU), [vllm-kunlun](https://github.com/baidu/vLLM-Kunlun) (for Baidu Kunlun XPU), etc.
269
+- **Non-official device plugins:** [vllm-metax](https://github.com/MetaX-MACA/vLLM-metax) (for MetaX GPU), [vllm-kunlun](https://github.com/baidu/vLLM-Kunlun) (for Baidu Kunlun XPU), [vllm-musa](https://github.com/MooreThreads/vllm-musa) (for Moore Threads GPU), etc.
270
271
In this case, `CustomOp` can enable these hardware manufacturers to seamlessly replace vLLM's operations with their deep-optimized kernels for specific devices at runtime, by just registering an OOT `CustomOp` and implementing the `forward_oot()` method.
272
@@ -289,7 +289,7 @@ Taking `MMEncoderAttention` as an example:
289
290
def __init__(...):
291
super().__init__(...)
292
-
292
+
293
def forward_oot(...):
294
# Call optimized device-specific kernels.
295
...