@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 93db3256a4

Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 11, 15:00 UTC

Give pooling examples better names (#30488)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Harry Mellor committed Dec 11, 2025 at 16:22 UTC 93db3256a4c56cbf8647b6c0caca78abdf926130

5 files changed +3 -3

docs/models/supported_models.md

+1 -1

@@ -568,7 +568,7 @@ These models primarily support the [`LLM.score`](./pooling_models.md#llmscore) A

568 ```

569

570 !!! note

571 - Load the official original `Qwen3 Reranker` by using the following command. More information can be found at: [examples/pooling/score/qwen3_reranker.py](../../examples/pooling/score/qwen3_reranker.py).

571 + Load the official original `Qwen3 Reranker` by using the following command. More information can be found at: [examples/pooling/score/offline_reranker.py](../../examples/pooling/score/offline_reranker.py).

572

573 ```bash

574 vllm serve Qwen/Qwen3-Reranker-0.6B --hf_overrides '{"architectures": ["Qwen3ForSequenceClassification"],"classifier_from_token": ["no", "yes"],"is_original_qwen3_reranker": true}'

docs/serving/openai_compatible_server.md

+1 -1

@@ -851,7 +851,7 @@ endpoints are compatible with both [Jina AI's re-rank API interface](https://jin

851 [Cohere's re-rank API interface](https://docs.cohere.com/v2/reference/rerank) to ensure compatibility with

852 popular open-source tools.

853

854 -Code example: [examples/pooling/score/jinaai_rerank_client.py](../../examples/pooling/score/jinaai_rerank_client.py)

854 +Code example: [examples/pooling/score/openai_reranker.py](../../examples/pooling/score/openai_reranker.py)

855

856 #### Example Request

857

examples/pooling/score/offline_reranker.py renamed

examples/pooling/score/openai_reranker.py renamed

vllm/model_executor/models/config.py

+1 -1

@@ -214,7 +214,7 @@ class Qwen3ForSequenceClassificationConfig(VerifyAndUpdateConfig):

214 tokens = getattr(config, "classifier_from_token", None)

215 assert tokens is not None and len(tokens) == 2, (

216 "Try loading the original Qwen3 Reranker?, see: "

217 - "https://github.com/vllm-project/vllm/tree/main/examples/offline_inference/qwen3_reranker.py"

217 + "https://github.com/vllm-project/vllm/tree/main/examples/offline_inference/offline_reranker.py"

218 )

219 vllm_config.model_config.hf_config.method = "from_2_way_softmax"

220