@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 7ed27f3cb5

Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 12, 23:30 UTC

[Doc]: fix typos in various files (#28945)

Signed-off-by: Didier Durand <durand.didier@gmail.com>

Didier Durand committed Nov 19, 2025 at 07:52 UTC 7ed27f3cb55e3f64614300ec7acde1b382a48541

6 files changed +7 -7

docs/design/moe_kernel_features.md

+2 -2

@@ -4,7 +4,7 @@ The purpose of this document is to provide an overview of the various MoE kernel

5 ## Fused MoE Modular All2All backends

7 -There are a number of all2all communication backends that are used to implement expert parallelism (EP) for the `FusedMoE` layer. The different `FusedMoEPrepareAndFinalize` sub-classes provide an interface for each all2all backend.

7 +There are a number of all2all communication backends that are used to implement expert parallelism (EP) for the `FusedMoE` layer. The different `FusedMoEPrepareAndFinalize` subclasses provide an interface for each all2all backend.

9 The following table describes the relevant features of each backend, i.e. activation format, supported quantization schemes and async support.

@@ -68,7 +68,7 @@ Modular kernels are supported by the following `FusedMoEMethodBase` classes.

69 ## Fused MoE Experts Kernels

71 -The are a number of MoE experts kernel implementations for different quantization types and architectures. Most follow the general API of the base Triton [`fused_experts`][vllm.model_executor.layers.fused_moe.fused_moe.fused_experts] function. Many have modular kernel adapters so they can be used with compatible all2all backends. This table lists each experts kernel and its particular properties.

71 +There are a number of MoE experts kernel implementations for different quantization types and architectures. Most follow the general API of the base Triton [`fused_experts`][vllm.model_executor.layers.fused_moe.fused_moe.fused_experts] function. Many have modular kernel adapters so they can be used with compatible all2all backends. This table lists each experts kernel and its particular properties.

73 Each kernel must be provided with one of the supported input activation formats. Some flavors of kernels support both standard and batched formats through different entry points, e.g. `TritonExperts` and `BatchedTritonExperts`. Batched format kernels are currently only needed for matching with certain all2all backends, e.g. `pplx`, `DeepEPLLPrepareAndFinalize`.

docs/design/plugin_system.md

+1 -1

@@ -49,7 +49,7 @@ Every plugin has three parts:

50 - **Platform plugins** (with group name `vllm.platform_plugins`): The primary use case for these plugins is to register custom, out-of-the-tree platforms into vLLM. The plugin function should return `None` when the platform is not supported in the current environment, or the platform class's fully qualified name when the platform is supported.

52 -- **IO Processor plugins** (with group name `vllm.io_processor_plugins`): The primary use case for these plugins is to register custom pre/post processing of the model prompt and model output for pooling models. The plugin function returns the IOProcessor's class fully qualified name.

52 +- **IO Processor plugins** (with group name `vllm.io_processor_plugins`): The primary use case for these plugins is to register custom pre-/post-processing of the model prompt and model output for pooling models. The plugin function returns the IOProcessor's class fully qualified name.

54 - **Stat logger plugins** (with group name `vllm.stat_logger_plugins`): The primary use case for these plugins is to register custom, out-of-the-tree loggers into vLLM. The entry point should be a class that subclasses StatLoggerBase.

docs/features/quantization/quark.md

+1 -1

@@ -306,7 +306,7 @@ As examples, we provide some ready-to-use quantized mixed precision model to sho

306

307 ### 2. inference the quantized mixed precision model in vLLM

308

309 -Models quantized with AMD Quark using mixed precision can natively be reload in vLLM, and e.g. evaluated using lm-evaluation-harness as follow:

309 +Models quantized with AMD Quark using mixed precision can natively be reload in vLLM, and e.g. evaluated using lm-evaluation-harness as follows:

310

311 ```bash

312 lm_eval --model vllm \

examples/online_serving/prometheus_grafana/README.md

+1 -1

@@ -46,7 +46,7 @@ Navigate to [`http://localhost:3000`](http://localhost:3000). Log in with the de

47 Navigate to [`http://localhost:3000/connections/datasources/new`](http://localhost:3000/connections/datasources/new) and select Prometheus.

49 -On Prometheus configuration page, we need to add the `Prometheus Server URL` in `Connection`. For this setup, Grafana and Prometheus are running in separate containers, but Docker creates DNS name for each containers. You can just use `http://prometheus:9090`.

49 +On Prometheus configuration page, we need to add the `Prometheus Server URL` in `Connection`. For this setup, Grafana and Prometheus are running in separate containers, but Docker creates DNS name for each container. You can just use `http://prometheus:9090`.

51 Click `Save & Test`. You should get a green check saying "Successfully queried the Prometheus API.".

vllm/engine/arg_utils.py

+1 -1

@@ -1500,7 +1500,7 @@ class EngineArgs:

1500 # Local DP rank = 1, use pure-external LB.

1501 if data_parallel_external_lb:

1502 assert self.data_parallel_rank is not None, (

1503 - "data_parallel_rank or node_rank must be spefified if "

1503 + "data_parallel_rank or node_rank must be specified if "

1504 "data_parallel_external_lb is enable."

1505 )

1506 assert self.data_parallel_size_local in (1, None), (

vllm/envs.py

+1 -1

@@ -1261,7 +1261,7 @@ environment_variables: dict[str, Callable[[], Any]] = {

1261 # MoE routing strategy selector.

1262 # See `RoutingSimulator.get_available_strategies()` # for available

1263 # strategies.

1264 - # Cutstom routing strategies can be registered by

1264 + # Custom routing strategies can be registered by

1265 # RoutingSimulator.register_strategy()

1266 # Note: custom strategies may not produce correct model outputs

1267 "VLLM_MOE_ROUTING_SIMULATION_STRATEGY": lambda: os.environ.get(