b78fbab4be · tduncan/Nvidia-Optimization-Stack-VLLM

docs/benchmarking/cli.md

+2 -2

@@ -147,7 +147,7 @@ With standard decoding, each streamed output usually contains one token, so ITL

147 and TPOT are typically similar.

148

149 With speculative decoding, one streamed output can contain multiple tokens,

150 -such as several accepted draft tokens within a single engine tstep. ITL records

150 +such as several accepted draft tokens within a single engine step. ITL records

151 only the gaps between streamed outputs; it does not add zero-duration gaps for

152 tokens in the same output. TPOT instead amortizes the request's decoding time

153 over every output token.

@@ -451,7 +451,7 @@ vllm bench serve \

451 --num-prompts -1

452 ```

453

454 -Available categories include `[high_entropy, mixed, low_entropy]`, where high entropy data contains unstructued data such as creative writing while low entropy data contains more structured data such as coding, more details are in the dataset card.

454 +Available categories include `[high_entropy, mixed, low_entropy]`, where high entropy data contains unstructured data such as creative writing while low entropy data contains more structured data such as coding, more details are in the dataset card.

455

456 #### BFCL (Tool-Calling) Benchmark

457

docs/benchmarking/dashboard.md

+1 -1

@@ -36,7 +36,7 @@ When run, benchmark script generates results under **benchmark/results** folder,

36 - `ON_CPU`: set the value to '1' on Intel® Xeon® and Arm® Neoverse™ Processors. Default value is 0.

37 - `SERVING_JSON`: JSON file to use for the serving tests. Default value is empty string (use default file).

38 - `LATENCY_JSON`: JSON file to use for the latency tests. Default value is empty string (use default file).

39 -- `THROUGHPUT_JSON`: JSON file to use for the throughout tests. Default value is empty string (use default file).

39 +- `THROUGHPUT_JSON`: JSON file to use for the throughput tests. Default value is empty string (use default file).

40 - `REMOTE_HOST`: IP for the remote vLLM service to benchmark. Default value is empty string.

41 - `REMOTE_PORT`: Port for the remote vLLM service to benchmark. Default value is empty string.

42 - `PROMPTS_PER_CONCURRENCY`: Multiplier to compute `num_prompts` for serving tests (`num_prompts = max_concurrency × value`). Overrides JSON `num_prompts`. Default is NULL.

docs/contributing/README.md

+1 -1

@@ -50,7 +50,7 @@ To rebuild only the Rust frontend binary:

50 ./build_rust.sh --debug # faster build for development

51 ```

52

53 -If you are developing vLLM's Python and CUDA/C++ code, install Pytorch first:

53 +If you are developing vLLM's Python and CUDA/C++ code, install PyTorch first:

54

55 ```bash

56 uv pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu129

docs/design/metrics.md

+1 -1

@@ -670,7 +670,7 @@ fall under the more general heading of "Observability".

670 vLLM has support for OpenTelemetry tracing:

671

672 - Added by <https://github.com/vllm-project/vllm/pull/4687> and reinstated by <https://github.com/vllm-project/vllm/pull/20372>

673 -- Configured with `--oltp-traces-endpoint` and `--collect-detailed-traces`

673 +- Configured with `--otlp-traces-endpoint` and `--collect-detailed-traces`

674 - [OpenTelemetry blog post](https://opentelemetry.io/blog/2024/llm-observability/)

675 - [User-facing docs](../../examples/observability/opentelemetry/README.md)

676 - [Blog post](https://medium.com/@ronen.schaffer/follow-the-trail-supercharging-vllm-with-opentelemetry-distributed-tracing-aa655229b46f)

docs/features/quantization/quark.md

+1 -1

@@ -100,7 +100,7 @@ kv-cache and the quantization algorithm is AutoSmoothQuant.

100

101 !!! note

102 Note the quantization algorithm needs a JSON config file and the config file is located in

103 - [Quark Pytorch examples](https://quark.docs.amd.com/latest/pytorch/pytorch_examples.html),

103 + [Quark PyTorch examples](https://quark.docs.amd.com/latest/pytorch/pytorch_examples.html),

104 under the directory `examples/torch/language_modeling/llm_ptq/models`. For example,

105 AutoSmoothQuant config file for Llama is

106 `examples/torch/language_modeling/llm_ptq/models/llama/autosmoothquant_config.json`.

docs/features/speculative_decoding/draft_model.md

+1 -1

@@ -96,7 +96,7 @@ The code used to request completions as a client remains unchanged:

96 },

97 gpu_memory_utilization=0.5,

98 )

99 -outputs = llm.generate(prompts，sampling_params)

99 +outputs = llm.generate(prompts, sampling_params)

100

101 for output in outputs:

102 prompt = output.prompt

docs/getting_started/installation/cpu.x86.inc.md

+1 -1

@@ -33,7 +33,7 @@ uv pip install "https://github.com/vllm-project/vllm/releases/download/v${VLLM_V

33 pip install "https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}+cpu-cp38-abi3-manylinux_2_34_x86_64.whl" --extra-index-url https://download.pytorch.org/whl/cpu

34 ```

35 !!! warning "set `LD_PRELOAD`"

36 - Before use vLLM CPU installed via wheels, make Intel OpenMP is added to `LD_PRELOAD`:

36 + Before use vLLM CPU installed via wheels, make sure Intel OpenMP is added to `LD_PRELOAD`:

37 ```bash

38 # manually find the path

39 sudo find / -iname *libiomp5.so

docs/governance/committers.md

+1 -1

@@ -2,7 +2,7 @@

2

3 This document lists the current committers of the vLLM project and the core areas they maintain.

4 Committers have write access to the vLLM repository and are responsible for reviewing and merging PRs.

5 -You can also refer to the [CODEOWNERS](https://github.com/vllm-project/vllm/blob/main/.github/CODEOWNERS) file for concrete file-level ownership and reviewers. Both this documents and the CODEOWNERS file are living documents and they complement each other.

5 +You can also refer to the [CODEOWNERS](https://github.com/vllm-project/vllm/blob/main/.github/CODEOWNERS) file for concrete file-level ownership and reviewers. Both these documents and the CODEOWNERS file are living documents and they complement each other.

6

7 ## Active Committers

8

docs/training/weight_transfer/base.md

+1 -1

@@ -233,7 +233,7 @@ Two implementations ship with vLLM:

233 | `RayVLLMWeightSyncClient(handle)` | One or more `AsyncLLM`/`LLM` Ray actors. Accepts a list and fans each call out to every handle, blocking on all of them, so a multi-actor (e.g. multi-DP) deployment is driven as one unit |

234 | `HTTPVLLMWeightSyncClient(base_url, timeout=300)` | A vLLM server over the RLHF HTTP routes |

235

236 -Custom weight sync clients can be implement like so:

236 +Custom weight sync clients can be implemented like so:

237

238 ```python

239 class MyFrameworkWeightSyncClient:

[Docs] Fix the typos in the document (#57190)