[Docs] Document pooling config resolution (#48497)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Taneem Ibrahim committed
Jul 15, 2026 at 14:24 UTC
de100ffb622f435b727177fc751c8de5f41c4d3b
1 file changed
+59
-4
docs/models/pooling_models/README.md
+59
-4
@@ -279,11 +279,66 @@ the pooler assigned to each task has the following attributes by default:
279
| `embed` | `LAST` | ✅︎ | ❌ |
280
| `classify` | `LAST` | ❌ | ✅︎ |
281
282
-When loading [Sentence Transformers](https://huggingface.co/sentence-transformers) models,
283
-its Sentence Transformers configuration file (`modules.json`) takes priority over the model's defaults.
282
+#### Resolution precedence
283
285
-You can further customize this via the `--pooler-config` option,
286
-which takes priority over both the model's and Sentence Transformers' defaults.
284
+The pooling method and `use_activation` are resolved per field. An explicitly
285
+set field in `--pooler-config` takes precedence over Sentence Transformers
286
+metadata, which in turn takes precedence over the model architecture or task
287
+default. Fields left unset continue through the chain independently.
288
+
289
+The current `PoolerConfig` has no `normalize` or `activation` field.
290
+`use_activation` controls whether the task's constructed normalization or
291
+classification activation is applied.
292
+
293
+| Field | Source precedence | How to override |
294
+| ----- | ----------------- | --------------- |
295
+| Pooling method (`pooling_type`) | `--pooler-config` > boolean `pooling_mode_*` fields in the Pooling module referenced by Sentence Transformers `modules.json` > architecture default (`LAST` for sequence pooling and `ALL` for token pooling unless the architecture overrides it) | Set `{"pooling_type": "CLS"}`, or set `seq_pooling_type` / `tok_pooling_type` explicitly. |
296
+| Embedding normalization (`use_activation`) | `--pooler-config` > Sentence Transformers modules (`true` when a Normalize module is present, otherwise `false`) > pooling-task default (`true`) when no Sentence Transformers Pooling module is found | Set `{"use_activation": false}` to return unnormalized embeddings. |
297
+| Classification activation function | Hugging Face `problem_type` > Sentence Transformers activation metadata > sigmoid or softmax selected from the label count | The function cannot be selected through `--pooler-config`; set `{"use_activation": false}` to return logits instead. |
298
+
299
+Sentence Transformers configurations using the newer compact `pooling_mode`
300
+string are not currently parsed; see [issue #45995](https://github.com/vllm-project/vllm/issues/45995).
301
+
302
+For converted models and predefined models using the standard DispatchPooler
303
+adapters, `embed` and `token_embed` construct an L2-normalization head, while
304
+`classify` and `token_classify` construct the selected classification activation.
305
+In both cases, `use_activation` controls whether that head is applied. Models with
306
+custom poolers can implement different behavior.
307
+
308
+To inspect the resolved fields without loading model weights:
309
+
310
+```python
311
+from vllm.config import ModelConfig, PoolerConfig
312
+from vllm.model_executor.layers.pooler.activations import get_act_fn
313
+
314
+
315
+def inspect(requested: PoolerConfig) -> None:
316
+ model_config = ModelConfig(
317
+ "intfloat/e5-small",
318
+ runner="pooling",
319
+ pooler_config=requested,
320
+ )
321
+ resolved = model_config.pooler_config
322
+ assert resolved is not None
323
+ print(
324
+ {
325
+ "seq_pooling_type": resolved.seq_pooling_type,
326
+ "tok_pooling_type": resolved.tok_pooling_type,
327
+ "use_activation": resolved.use_activation,
328
+ "sequence_classification_activation": type(
329
+ get_act_fn(model_config.hf_config)
330
+ ).__name__,
331
+ }
332
+ )
333
+
334
+
335
+inspect(PoolerConfig())
336
+inspect(PoolerConfig(pooling_type="CLS", use_activation=False))
337
+```
338
+
339
+For `intfloat/e5-small`, the first result contains `MEAN`, `ALL`, and `True`.
340
+The second contains `CLS`, `ALL`, and `False`. Both report the classification
341
+activation that the standard sequence-classification adapter would construct.
342
343
## Removed Features
344