A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 13, 03:15 UTC
[Doc]: fix typos in various files (#28567)
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Didier Durand committed
Nov 15, 2025 at 20:27 UTC
2bb4435cb7e2e2317b0f20803347690fb38fe6b4
5 files changed
+6
-6
docs/design/moe_kernel_features.md
+1
-1
@@ -68,7 +68,7 @@ Modular kernels are supported by the following `FusedMoEMethodBase` classes.
68
69
## Fused MoE Experts Kernels
70
71
-The are a number of MoE experts kernel implementations for different quantization types and architectures. Most follow the general API of the base Triton [`fused_experts`][vllm.model_executor.layers.fused_moe.fused_moe.fused_experts] function. Many have modular kernel adatpers so they can be used with compatible all2all backends. This table lists each experts kernel and its particular properties.
71
+The are a number of MoE experts kernel implementations for different quantization types and architectures. Most follow the general API of the base Triton [`fused_experts`][vllm.model_executor.layers.fused_moe.fused_moe.fused_experts] function. Many have modular kernel adapters so they can be used with compatible all2all backends. This table lists each experts kernel and its particular properties.
72
73
Each kernel must be provided with one of the supported input activation formats. Some flavors of kernels support both standard and batched formats through different entry points, e.g. `TritonExperts` and `BatchedTritonExperts`. Batched format kernels are currently only needed for matching with certain all2all backends, e.g. `pplx`, `DeepEPLLPrepareAndFinalize`.
74
docs/features/quantization/quark.md
+1
-1
@@ -298,7 +298,7 @@ There are two steps to generate and deploy a mixed precision model quantized wit
298
299
Firstly, the layerwise mixed-precision configuration for a given LLM model is searched and then quantized using AMD Quark. We will provide a detailed tutorial with Quark APIs later.
300
301
-As examples, we provide some ready-to-use quantized mixed precision model to show the usage in vLLM and the accuracy benifits. They are:
301
+As examples, we provide some ready-to-use quantized mixed precision model to show the usage in vLLM and the accuracy benefits. They are:
302
303
- amd/Llama-2-70b-chat-hf-WMXFP4FP8-AMXFP4FP8-AMP-KVFP8
304
- amd/Mixtral-8x7B-Instruct-v0.1-WMXFP4FP8-AMXFP4FP8-AMP-KVFP8
vllm/compilation/compiler_interface.py
+1
-1
@@ -299,7 +299,7 @@ class InductorAdaptor(CompilerInterface):
299
self.base_cache_dir = cache_dir[: -len(prefix)] if prefix else cache_dir
300
if disable_cache:
301
return
302
- # redirect the cache directory to a sub-directory
302
+ # redirect the cache directory to a subdirectory
303
# set flags so that Inductor and Triton store their cache
304
# in the cache_dir, then users only need to copy the cache_dir
305
# to another machine to reuse the cache.
vllm/compilation/decorators.py
+2
-2
@@ -159,7 +159,7 @@ def support_torch_compile(
159
160
`mark_unbacked_dims` is a dictionary that maps argument names with a dynamic
161
dim to be decorated with `mark_unbacked`. This is useful if we would like to
162
- enforce that dynamo do not specialize on 0/1 values in the case of dummy input
162
+ enforce that dynamo does not specialize on 0/1 values in the case of dummy input
163
such as for vision model compilation
164
"""
165
@@ -483,7 +483,7 @@ def maybe_use_cudagraph_partition_wrapper(vllm_config: VllmConfig):
483
Context manager to set/unset customized cudagraph partition wrappers.
484
485
If we're using Inductor-based graph partitioning, we currently have the
486
- whole `fx.Graph` before Inductor lowering and and the piecewise
486
+ whole `fx.Graph` before Inductor lowering and the piecewise
487
splitting happens after all graph passes and fusions. Here, we add
488
a custom hook for Inductor to wrap each partition with our static
489
graph wrapper class to maintain more control over static graph
vllm/v1/worker/gpu_model_runner.py
+1
-1
@@ -2871,7 +2871,7 @@ class GPUModelRunner(
2871
"gpu_model_runner: set_async_sampled_token_ids"
2872
):
2873
# Save ref of sampled_token_ids CPU tensor if the batch contains
2874
- # any requests with sampling params that that require output ids.
2874
+ # any requests with sampling params that require output ids.
2875
self.input_batch.set_async_sampled_token_ids(
2876
async_output.sampled_token_ids_cpu,
2877
async_output.async_copy_ready_event,