[Pooling] Fix Rust pooling endpoint for benchmark (#53352)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Taneem Ibrahim committed
Aug 22, 2026 at 14:45 UTC
a014e35f38c80fb0652387740193ad2147fed6a3
3 files changed
+3
-3
rust/src/bench/README.md
+1
-1
@@ -419,7 +419,7 @@ vllm-bench \
419
| --------- | ------------- | ------------- |
420
| `openai-embeddings` | `/v1/embeddings` | Text embedding (accepts text or token IDs) |
421
| `openai-embeddings-chat` | `/v1/embeddings` | Chat-format embedding (supports multimodal content) |
422
-| `vllm-pooling` | `/v1/pooling` | vLLM native pooling endpoint |
422
+| `vllm-pooling` | `/pooling` | vLLM native pooling endpoint |
423
| `vllm-rerank` | `/v1/rerank` | vLLM reranking (query from prompt, documents via `--extra-body`) |
424
425
Pooling backends are non-streaming and report E2EL (end-to-end latency) only. Use `--dataset-name sharegpt`, `sonnet`, or `hf` for text-based embedding/rerank benchmarks, or `random` for token-ID-based embedding benchmarks.
rust/src/bench/src/backends/pooling.rs
+1
-1
@@ -8,7 +8,7 @@
8
//! - `openai-embeddings`: Standard OpenAI `/v1/embeddings` with text input
9
//! - `openai-embeddings-chat`: OpenAI `/v1/embeddings` with chat message format (supports
10
//! multimodal)
11
-//! - `vllm-pooling`: vLLM `/v1/pooling` endpoint
11
+//! - `vllm-pooling`: vLLM `/pooling` endpoint
12
//! - `vllm-rerank`: vLLM `/v1/rerank` endpoint (query + documents)
13
14
use std::time::Instant;
rust/src/bench/src/cli.rs
+1
-1
@@ -682,7 +682,7 @@ impl BenchServeArgs {
682
BackendKind::OpenaiEmbeddings | BackendKind::OpenaiEmbeddingsChat => {
683
"/v1/embeddings".to_string()
684
}
685
- BackendKind::VllmPooling => "/v1/pooling".to_string(),
685
+ BackendKind::VllmPooling => "/pooling".to_string(),
686
BackendKind::VllmRerank => "/v1/rerank".to_string(),
687
}
688
}