@tduncan / Nvidia-Optimization-Stack-VLLM / commits / dee37d8911

[Frontend] Add `--tool-strict-level` for server-side control for structural tag activation (#56268)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com> Signed-off-by: lazymio <mio@lazym.io> Signed-off-by: Bugen Zhao <i@bugenzhao.com>

lazymio committed Sep 18, 2026 at 12:32 UTC dee37d89115db4c94a820a79a78a7828e141c910
35 files changed +742 -61
docs/features/tool_calling.md
+30 -10
@@ -109,18 +109,22 @@ vLLM supports the `tool_choice='none'` option in the chat completion API. When t
109
110 ## Constrained Decoding Behavior
111
112 -Whether vLLM enforces the tool parameter schema during generation depends on the `tool_choice` mode and the per-tool `strict` field:
112 +Structural-tag parsers resolve call obligation, grammar activation, and argument-schema enforcement separately. With structural-tag enforcement enabled:
113
114 -| `tool_choice` value | Schema-constrained decoding | Behavior |
114 +| `tool_choice` value | Call obligation | Structural-tag activation |
115 | --- | --- | --- |
116 -| Named function | Yes (via structured outputs backend) | Arguments are guaranteed to be valid JSON conforming to the function's parameter schema. |
117 -| `"required"` | Yes (via structured outputs backend) | Same as named function. The model must produce at least one tool call. |
118 -| `"auto"` | Only when `strict: true` is set on at least one tool | Structural-tag parsers constrain tool-call arguments when a tool opts in with `strict: true`. Without it, the model generates freely and tool calls are extracted from raw text. |
119 -| `"none"` | N/A | No tool calls are produced. |
116 +| Named function | Call the selected function | Always |
117 +| `"required"` | Produce at least one tool call | Always |
118 +| `"auto"` | Tool calls are optional | When at least one tool sets `strict: true`, or `--tool-strict-level` is `function` or `parameter` |
119 +| `"none"` | Tool calling is disabled | Disabled |
120 +
121 +When a structural tag applies, each tool's declared parameter schema is enforced only when that tool sets `strict: true` or the server uses `--tool-strict-level parameter`. Tools with omitted or false `strict` receive broad argument-syntax constraints at levels `auto` and `function`, including for required and named calls.
122 +
123 +For parsers using schema-derived JSON constraints, required and named calls continue to enforce the declared parameter schemas.
124
125 ### Strict Mode
126
123 -For `tool_choice="required"` or named function calling, structural-tag constraints are always applied regardless of the `strict` field. For `tool_choice="auto"`, setting `strict: true` on at least one tool opts in to structural-tag constraints; without it, the model generates freely and tool calls are extracted from raw text. The `strict` field is supported across all three API surfaces: Chat Completion, Responses, and Anthropic Messages.
127 +Structural-tag constraints have two layers. The tool-call *envelope* (markup and function name) is constrained whenever a structural tag applies: always for `tool_choice="required"` and named function calling, and for `tool_choice="auto"` once at least one tool sets `strict: true` or the server raises the floor. The *argument schema* of an individual tool is pinned only when that tool sets `strict: true` (or under `--tool-strict-level parameter`); a tool that omits `strict` keeps its arguments unconstrained, even when another tool in the same request is strict. The `strict` field is supported across all three API surfaces: Chat Completion, Responses, and Anthropic Messages.
128
129 For best compatibility with strict schema enforcement, define tool parameter schemas in the OpenAI strict-schema style:
130
@@ -134,6 +138,22 @@ vLLM also provides a global toggle via the `VLLM_ENFORCE_STRICT_TOOL_CALLING` en
138 VLLM_ENFORCE_STRICT_TOOL_CALLING=false vllm serve ...
139 ```
140
141 +### Server-Side Strictness Floor
142 +
143 +Most OpenAI-compatible clients and agent frameworks never set `strict` on their tools, so with `tool_choice="auto"` the model generates tool calls without any grammar and malformed markup can leak into the response. The `--tool-strict-level` option lets the server operator raise the floor for every request that carries tools, independently of what the client declares:
144 +
145 +| Value | Behavior |
146 +| --- | --- |
147 +| `auto` (default) | Follow the request's tool choice and per-tool strictness. Required/named choices activate structural tags; `tool_choice="auto"` activates them when at least one tool sets `strict: true`. |
148 +| `function` | Constrain the tool-call envelope (markup and the function name) for every request with tools, leaving argument contents free unless the client marked the tool `strict: true`. |
149 +| `parameter` | Additionally pin argument schemas for every tool, as if every tool had `strict: true`. |
150 +
151 +```bash
152 +vllm serve ... --tool-strict-level function
153 +```
154 +
155 +The floor never relaxes a constraint the request would already receive: tools the client marked `strict: true` keep their schemas at every level. With `tool_choice="auto"`, the grammar does not force a tool call; a plain text response stays valid. `VLLM_ENFORCE_STRICT_TOOL_CALLING=false` disables structural tags entirely and takes precedence over this option.
156 +
157 ## Automatic Function Calling
158
159 To enable this feature, you should set the following flags:
@@ -152,7 +172,7 @@ from HuggingFace; and you can find an example of this in a `tokenizer_config.jso
172 If your favorite tool-calling model is not supported, please feel free to contribute a parser & tool use chat template!
173
174 !!! note
155 - With `tool_choice="auto"`, schema-level constraint requires both `VLLM_ENFORCE_STRICT_TOOL_CALLING=true` (the default) and at least one tool with `strict: true`. When these conditions are met and the selected parser supports structural tags, vLLM constrains tool-call arguments. Otherwise, vLLM extracts tool calls from raw text, so arguments may occasionally be malformed or violate the function's parameter schema.
175 + With `tool_choice="auto"`, structural-tag constraints require both `VLLM_ENFORCE_STRICT_TOOL_CALLING=true` (the default) and at least one tool with `strict: true`, or a server-side floor set via `--tool-strict-level`. When these conditions are met and the selected parser supports structural tags, vLLM constrains the tool-call envelope and pins the argument schema of each tool that sets `strict: true` (or of every tool under `--tool-strict-level parameter`). Otherwise, vLLM extracts tool calls from raw text, so arguments may occasionally be malformed or violate the function's parameter schema.
176
177 ### Hermes Models (`hermes`)
178
@@ -162,8 +182,8 @@ All Nous Research Hermes-series models newer than Hermes 2 Pro should be support
182 * `NousResearch/Hermes-2-Theta-*`
183 * `NousResearch/Hermes-3-*`
184
165 -_Note that the Hermes 2 **Theta** models are known to have degraded tool call quality and capabilities due to the merge
166 -step in their creation_.
185 +*Note that the Hermes 2 **Theta** models are known to have degraded tool call quality and capabilities due to the merge
186 +step in their creation*.
187
188 Flags: `--tool-call-parser hermes`
189
rust/src/chat/src/backend/hf.rs
+2
@@ -134,6 +134,7 @@ impl ChatBackend for HfChatBackend {
134 self.tokenizer.clone(),
135 options.tool_call_parser,
136 options.reasoning_parser,
137 + options.tool_strict_level,
138 )?))
139 }
140 }
@@ -343,6 +344,7 @@ mod tests {
344 let error = match backend.new_chat_output_processor(
345 &mut request,
346 NewChatOutputProcessorOptions {
347 + tool_strict_level: crate::ToolStrictLevel::Auto,
348 tool_call_parser: &ParserSelection::Explicit("json".to_string()),
349 reasoning_parser: &ParserSelection::Auto,
350 },
rust/src/chat/src/backend/mod.rs
+2 -1
@@ -13,7 +13,7 @@ use crate::multimodal::{MmLimitPerPrompt, MultimodalModelInfo};
13 use crate::output::DynChatOutputProcessor;
14 use crate::renderer::DynChatRenderer;
15 use crate::request::ChatRequest;
16 -use crate::{ChatTemplateContentFormatOption, ParserSelection, RendererSelection};
16 +use crate::{ChatTemplateContentFormatOption, ParserSelection, RendererSelection, ToolStrictLevel};
17
18 pub mod hf;
19
@@ -21,6 +21,7 @@ pub mod hf;
21 pub struct NewChatOutputProcessorOptions<'a> {
22 pub tool_call_parser: &'a ParserSelection,
23 pub reasoning_parser: &'a ParserSelection,
24 + pub tool_strict_level: ToolStrictLevel,
25 }
26
27 /// Minimal prompt-processing backend needed by `vllm-chat`.
rust/src/chat/src/lib.rs
+18 -1
@@ -30,7 +30,7 @@ pub use parser::reasoning::{
30 ReasoningDelta, ReasoningError, ReasoningParser, ReasoningParserFactory,
31 };
32 pub use parser::tool::{ToolParser, ToolParserError, ToolParserFactory};
33 -pub use parser::{ParserSelection, validate_parser_overrides};
33 +pub use parser::{ParserSelection, ToolStrictLevel, validate_parser_overrides};
34 pub use reasoning::EffortValue;
35 pub use renderer::hf::ChatTemplateContentFormatOption;
36 pub use renderer::{
@@ -74,6 +74,8 @@ pub struct ChatRequestProcessor {
74 tool_call_parser: ParserSelection,
75 /// Reasoning parser selection used when preparing generation requests.
76 reasoning_parser: ParserSelection,
77 + /// Server-side floor for tool-call structural tags.
78 + tool_strict_level: ToolStrictLevel,
79 }
80
81 impl ChatRequestProcessor {
@@ -85,6 +87,7 @@ impl ChatRequestProcessor {
87 model_dtype: Some(model_dtype),
88 tool_call_parser: ParserSelection::Auto,
89 reasoning_parser: ParserSelection::Auto,
90 + tool_strict_level: ToolStrictLevel::Auto,
91 }
92 }
93
@@ -95,6 +98,7 @@ impl ChatRequestProcessor {
98 model_dtype: None,
99 tool_call_parser: ParserSelection::Auto,
100 reasoning_parser: ParserSelection::Auto,
101 + tool_strict_level: ToolStrictLevel::Auto,
102 }
103 }
104
@@ -109,6 +113,12 @@ impl ChatRequestProcessor {
113 self
114 }
115
116 + /// Configure the server-side floor for tool-call structural tags.
117 + pub fn with_tool_strict_level(mut self, tool_strict_level: ToolStrictLevel) -> Self {
118 + self.tool_strict_level = tool_strict_level;
119 + self
120 + }
121 +
122 async fn finalize_rendered_prompt(
123 &self,
124 request: &ChatRequest,
@@ -207,6 +217,7 @@ impl ChatRequestProcessor {
217 NewChatOutputProcessorOptions {
218 tool_call_parser: &self.tool_call_parser,
219 reasoning_parser: &self.reasoning_parser,
220 + tool_strict_level: self.tool_strict_level,
221 },
222 )?;
223 let text_request = self.prepare_text_request(request).await?;
@@ -255,6 +266,12 @@ impl ChatLlm {
266 self
267 }
268
269 + /// Set the server-side floor for tool-call structural tags.
270 + pub fn with_tool_strict_level(mut self, tool_strict_level: ToolStrictLevel) -> Self {
271 + self.processor.tool_strict_level = tool_strict_level;
272 + self
273 + }
274 +
275 /// Override the effective model dtype used for multimodal tensor encoding.
276 pub fn with_model_dtype(mut self, model_dtype: ModelDtype) -> Self {
277 self.processor.model_dtype = Some(model_dtype);
rust/src/chat/src/output/default/mod.rs
+12 -3
@@ -18,10 +18,10 @@ use self::unified::unified_event_stream;
18 use super::structured::structured_chat_event_stream;
19 use crate::error::Result;
20 use crate::output::{ChatOutputProcessor, DynChatEventStream, DynDecodedTextEventStream};
21 -use crate::parser::ParserSelection;
21 use crate::parser::reasoning::{ReasoningParser, ReasoningParserFactory};
22 use crate::parser::tool::{ToolParser, ToolParserFactory};
23 use crate::parser::unified::UnifiedParserFactory;
24 +use crate::parser::{ParserSelection, ToolStrictLevel};
25 use crate::request::{ChatRequest, ChatTool};
26 use crate::{Error, Result as ChatResult};
27
@@ -49,6 +49,7 @@ impl DefaultChatOutputProcessor {
49 tokenizer: DynTokenizer,
50 tool_call_parser: &ParserSelection,
51 reasoning_parser: &ParserSelection,
52 + tool_strict_level: ToolStrictLevel,
53 ) -> ChatResult<Self> {
54 let parser = if let Some(parser) = Self::resolve_optional_unified_parser(
55 request.tools(),
@@ -73,7 +74,11 @@ impl DefaultChatOutputProcessor {
74 Box::new(CombinedParser::new(reasoning_parser, tool_parser)) as Box<dyn UnifiedParser>
75 };
76
76 - apply_structural_tag_constraint(request, parser.structural_tag_builder())?;
77 + apply_structural_tag_constraint(
78 + request,
79 + parser.structural_tag_builder(),
80 + tool_strict_level,
81 + )?;
82
83 if parser.preserve_special_tokens() {
84 request.decode_options.skip_special_tokens = false;
@@ -191,7 +196,7 @@ mod tests {
196 use vllm_tokenizer::test_utils::TestTokenizer;
197
198 use super::DefaultChatOutputProcessor;
194 - use crate::parser::ParserSelection;
199 + use crate::parser::{ParserSelection, ToolStrictLevel};
200 use crate::request::ChatRequest;
201
202 fn tokenizer() -> Arc<TestTokenizer> {
@@ -213,6 +218,7 @@ mod tests {
218 tokenizer(),
219 &selection,
220 &selection,
221 + ToolStrictLevel::Auto,
222 )
223 .unwrap();
224 }
@@ -227,6 +233,7 @@ mod tests {
233 tokenizer(),
234 &ParserSelection::Auto,
235 &ParserSelection::Auto,
236 + ToolStrictLevel::Auto,
237 )
238 .unwrap();
239 }
@@ -244,6 +251,7 @@ mod tests {
251 tokenizer(),
252 tool,
253 reasoning,
254 + ToolStrictLevel::Auto,
255 )
256 .unwrap();
257 }
@@ -258,6 +266,7 @@ mod tests {
266 tokenizer(),
267 &ParserSelection::Auto,
268 &ParserSelection::Explicit("gemma4".to_string()),
269 + ToolStrictLevel::Auto,
270 ) {
271 Ok(_) => panic!("expected mixed Gemma4 parser selection to fail"),
272 Err(error) => error,
rust/src/chat/src/output/default/structural_tag.rs
+191 -29
@@ -14,19 +14,21 @@ use xgrammar_structural_tag::{
14 build_structural_tag,
15 };
16
17 +use crate::parser::ToolStrictLevel;
18 use crate::request::{ChatRequest, ChatToolChoice};
19 use crate::{Error, Result as ChatResult};
20
21 /// Apply structural tag constraints to the request based on the tool parser's structural tag
21 -/// support and the request's tool choice.
22 +/// support, the request's tool choice, and the server-side strictness floor.
23 pub(super) fn apply_structural_tag_constraint(
24 request: &mut ChatRequest,
25 builder: Option<&dyn StructuralTagBuilder>,
26 + strict_level: ToolStrictLevel,
27 ) -> ChatResult<()> {
28 let Some(builder) = builder else {
29 return Ok(());
30 };
29 - let Some(tool_choice) = structural_tag_tool_choice(request) else {
31 + let Some(tool_choice) = structural_tag_tool_choice(request, strict_level) else {
32 return Ok(());
33 };
34
@@ -34,11 +36,14 @@ pub(super) fn apply_structural_tag_constraint(
36 .tools()
37 .iter()
38 .map(|tool| {
39 + // A tool without an explicit `strict` is non-strict: its call envelope is
40 + // constrained, but its arguments stay free unless the server level is `parameter`.
41 + let strict = strict_level >= ToolStrictLevel::Parameter || tool.strict == Some(true);
42 ToolParam::Function(FunctionToolParam::new(FunctionDefinition {
43 name: tool.name.clone(),
44 description: tool.description.clone(),
45 parameters: Some(tool.parameters.clone()),
41 - strict: tool.strict,
46 + strict: Some(strict),
47 }))
48 })
49 .collect::<Vec<_>>();
@@ -66,17 +71,21 @@ pub(super) fn apply_structural_tag_constraint(
71 /// Resolve the tool choice used for [`xgrammar_structural_tag`] based on the request.
72 ///
73 /// Returns `None` if no structural tag constraints should be applied.
69 -fn structural_tag_tool_choice(request: &ChatRequest) -> Option<StructuralTagToolChoice> {
74 +fn structural_tag_tool_choice(
75 + request: &ChatRequest,
76 + strict_level: ToolStrictLevel,
77 +) -> Option<StructuralTagToolChoice> {
78 if request.tools().is_empty() {
79 return None;
80 }
81
82 match request.tool_choice() {
75 - // For `Auto`, only apply the structural tag if there's at least one strict tool.
76 - ChatToolChoice::Auto if request.tools().iter().any(|tool| tool.strict == Some(true)) => {
77 - Some(StructuralTagToolChoice::auto())
78 - }
79 - ChatToolChoice::Auto | ChatToolChoice::None => None,
83 + ChatToolChoice::None => None,
84 + // For `Auto`, apply the structural tag only when the server raises the floor or at
85 + // least one tool opts in with `strict: true`.
86 + ChatToolChoice::Auto => (strict_level >= ToolStrictLevel::Function
87 + || request.tools().iter().any(|tool| tool.strict == Some(true)))
88 + .then(StructuralTagToolChoice::auto),
89
90 ChatToolChoice::Required => Some(StructuralTagToolChoice::required()),
91 ChatToolChoice::Function { name } => Some(StructuralTagToolChoice::function(name.clone())),
@@ -148,8 +157,12 @@ mod tests {
157 let mut request = request(ChatToolChoice::Auto, vec![chat_tool("search", Some(true))]);
158 let parser = qwen3_coder_parser(request.tools());
159
151 - apply_structural_tag_constraint(&mut request, parser.structural_tag_builder())
152 - .expect("structural tag should build");
160 + apply_structural_tag_constraint(
161 + &mut request,
162 + parser.structural_tag_builder(),
163 + ToolStrictLevel::Auto,
164 + )
165 + .expect("structural tag should build");
166
167 let tag = structural_tag_value(&request);
168 assert_eq!(tag["type"], "structural_tag");
@@ -172,8 +185,12 @@ mod tests {
185 };
186 let parser = qwen3_coder_parser(request.tools());
187
175 - apply_structural_tag_constraint(&mut request, parser.structural_tag_builder())
176 - .expect("structural tag should build");
188 + apply_structural_tag_constraint(
189 + &mut request,
190 + parser.structural_tag_builder(),
191 + ToolStrictLevel::Auto,
192 + )
193 + .expect("structural tag should build");
194
195 let tag = structural_tag_value(&request);
196 assert!(tag.to_string().contains("lookup"));
@@ -199,8 +216,12 @@ mod tests {
216 };
217 let parser = qwen3_coder_parser(request.tools());
218
202 - apply_structural_tag_constraint(&mut request, parser.structural_tag_builder())
203 - .expect("structural tag should build");
219 + apply_structural_tag_constraint(
220 + &mut request,
221 + parser.structural_tag_builder(),
222 + ToolStrictLevel::Auto,
223 + )
224 + .expect("structural tag should build");
225
226 let tag = structural_tag_value(&request).to_string();
227 assert!(tag.contains("search"));
@@ -212,8 +233,12 @@ mod tests {
233 let mut request = request(ChatToolChoice::Auto, vec![chat_tool("search", None)]);
234 let parser = qwen3_coder_parser(request.tools());
235
215 - apply_structural_tag_constraint(&mut request, parser.structural_tag_builder())
216 - .expect("structural tag decision should succeed");
236 + apply_structural_tag_constraint(
237 + &mut request,
238 + parser.structural_tag_builder(),
239 + ToolStrictLevel::Auto,
240 + )
241 + .expect("structural tag decision should succeed");
242
243 assert!(request.sampling_params.structured_outputs.is_none());
244 }
@@ -227,8 +252,12 @@ mod tests {
252 });
253 let parser = qwen3_coder_parser(request.tools());
254
230 - apply_structural_tag_constraint(&mut request, parser.structural_tag_builder())
231 - .expect("structural tag should build");
255 + apply_structural_tag_constraint(
256 + &mut request,
257 + parser.structural_tag_builder(),
258 + ToolStrictLevel::Auto,
259 + )
260 + .expect("structural tag should build");
261
262 let params = structured_outputs(&request);
263 assert!(params.constraint.is_structural_tag());
@@ -242,8 +271,12 @@ mod tests {
271 let mut request = request(ChatToolChoice::Required, vec![chat_tool("search", None)]);
272 let parser = qwen3_coder_parser(request.tools());
273
245 - apply_structural_tag_constraint(&mut request, parser.structural_tag_builder())
246 - .expect("structural tag should build");
274 + apply_structural_tag_constraint(
275 + &mut request,
276 + parser.structural_tag_builder(),
277 + ToolStrictLevel::Auto,
278 + )
279 + .expect("structural tag should build");
280
281 let tag = structural_tag_value(&request);
282 assert_eq!(tag["type"], "structural_tag");
@@ -259,8 +292,12 @@ mod tests {
292 });
293 let parser = qwen3_coder_parser(request.tools());
294
262 - apply_structural_tag_constraint(&mut request, parser.structural_tag_builder())
263 - .expect("structural tag should build");
295 + apply_structural_tag_constraint(
296 + &mut request,
297 + parser.structural_tag_builder(),
298 + ToolStrictLevel::Auto,
299 + )
300 + .expect("structural tag should build");
301
302 let params = structured_outputs(&request);
303 assert!(params.constraint.is_structural_tag());
@@ -279,8 +316,12 @@ mod tests {
316 );
317 let parser = qwen3_coder_parser(request.tools());
318
282 - apply_structural_tag_constraint(&mut request, parser.structural_tag_builder())
283 - .expect("structural tag should build");
319 + apply_structural_tag_constraint(
320 + &mut request,
321 + parser.structural_tag_builder(),
322 + ToolStrictLevel::Auto,
323 + )
324 + .expect("structural tag should build");
325
326 let tag = structural_tag_value(&request).to_string();
327 assert!(tag.contains("lookup"));
@@ -292,8 +333,12 @@ mod tests {
333 let mut request = request(ChatToolChoice::None, vec![chat_tool("search", Some(true))]);
334 let parser = qwen3_coder_parser(request.tools());
335
295 - apply_structural_tag_constraint(&mut request, parser.structural_tag_builder())
296 - .expect("structural tag decision should succeed");
336 + apply_structural_tag_constraint(
337 + &mut request,
338 + parser.structural_tag_builder(),
339 + ToolStrictLevel::Auto,
340 + )
341 + .expect("structural tag decision should succeed");
342
343 assert!(request.sampling_params.structured_outputs.is_none());
344 }
@@ -307,10 +352,127 @@ mod tests {
352 });
353 let parser = qwen3_coder_parser(request.tools());
354
310 - apply_structural_tag_constraint(&mut request, parser.structural_tag_builder())
311 - .expect("structural tag decision should succeed");
355 + apply_structural_tag_constraint(
356 + &mut request,
357 + parser.structural_tag_builder(),
358 + ToolStrictLevel::Auto,
359 + )
360 + .expect("structural tag decision should succeed");
361
362 let params = structured_outputs(&request);
363 assert!(params.constraint.is_json_object());
364 }
365 +
366 + fn build(tool_choice: ChatToolChoice, tools: Vec<Tool>, level: ToolStrictLevel) -> ChatRequest {
367 + let mut request = request(tool_choice, tools);
368 + let parser = qwen3_coder_parser(request.tools());
369 +
370 + apply_structural_tag_constraint(&mut request, parser.structural_tag_builder(), level)
371 + .expect("structural tag decision should succeed");
372 + request
373 + }
374 +
375 + fn tag_string(request: &ChatRequest) -> String {
376 + structural_tag_value(request).to_string()
377 + }
378 +
379 + #[test]
380 + fn function_level_constrains_auto_without_strict_tools() {
381 + let request = build(
382 + ChatToolChoice::Auto,
383 + vec![chat_tool("search", None)],
384 + ToolStrictLevel::Function,
385 + );
386 +
387 + let tag = structural_tag_value(&request);
388 + assert_eq!(tag["type"], "structural_tag");
389 + assert!(tag.to_string().contains("search"));
390 + }
391 +
392 + #[test]
393 + fn absent_strict_is_non_strict() {
394 + let unset = build(
395 + ChatToolChoice::Required,
396 + vec![chat_tool("search", None)],
397 + ToolStrictLevel::Auto,
398 + );
399 + let non_strict = build(
400 + ChatToolChoice::Required,
401 + vec![chat_tool("search", Some(false))],
402 + ToolStrictLevel::Auto,
403 + );
404 + let strict = build(
405 + ChatToolChoice::Required,
406 + vec![chat_tool("search", Some(true))],
407 + ToolStrictLevel::Auto,
408 + );
409 +
410 + assert_eq!(tag_string(&unset), tag_string(&non_strict));
411 + assert_ne!(tag_string(&unset), tag_string(&strict));
412 + }
413 +
414 + #[test]
415 + fn strict_tool_does_not_pin_its_neighbour() {
416 + let mixed = build(
417 + ChatToolChoice::Auto,
418 + vec![chat_tool("weather", Some(true)), chat_tool("search", None)],
419 + ToolStrictLevel::Auto,
420 + );
421 + let explicit = build(
422 + ChatToolChoice::Auto,
423 + vec![
424 + chat_tool("weather", Some(true)),
425 + chat_tool("search", Some(false)),
426 + ],
427 + ToolStrictLevel::Auto,
428 + );
429 +
430 + assert_eq!(tag_string(&mixed), tag_string(&explicit));
431 + }
432 +
433 + #[test]
434 + fn parameter_level_pins_argument_schemas() {
435 + let pinned = build(
436 + ChatToolChoice::Auto,
437 + vec![chat_tool("search", None)],
438 + ToolStrictLevel::Parameter,
439 + );
440 + let strict = build(
441 + ChatToolChoice::Auto,
442 + vec![chat_tool("search", Some(true))],
443 + ToolStrictLevel::Auto,
444 + );
445 + let envelope_only = build(
446 + ChatToolChoice::Auto,
447 + vec![chat_tool("search", None)],
448 + ToolStrictLevel::Function,
449 + );
450 +
451 + assert_eq!(tag_string(&pinned), tag_string(&strict));
452 + assert_ne!(tag_string(&pinned), tag_string(&envelope_only));
453 + }
454 +
455 + #[test]
456 + fn tool_strict_level_parses_case_insensitively() {
457 + assert_eq!("AUTO".parse::<ToolStrictLevel>(), Ok(ToolStrictLevel::Auto));
458 + assert_eq!(
459 + "PARAMETER".parse::<ToolStrictLevel>(),
460 + Ok(ToolStrictLevel::Parameter)
461 + );
462 + assert_eq!(
463 + "function".parse::<ToolStrictLevel>(),
464 + Ok(ToolStrictLevel::Function)
465 + );
466 + assert_eq!(ToolStrictLevel::default(), ToolStrictLevel::Auto);
467 + assert_eq!(
468 + serde_json::to_value(ToolStrictLevel::Auto).unwrap(),
469 + json!("auto")
470 + );
471 + assert_eq!(
472 + serde_json::from_value::<ToolStrictLevel>(json!("auto")).unwrap(),
473 + ToolStrictLevel::Auto
474 + );
475 + assert!("off".parse::<ToolStrictLevel>().is_err());
476 + assert!("strict".parse::<ToolStrictLevel>().is_err());
477 + }
478 }
rust/src/chat/src/parser/mod.rs
+60
@@ -57,6 +57,66 @@ impl ParserSelection {
57 }
58 }
59
60 +/// Server-side floor for tool-call structural tags (`--tool-strict-level`).
61 +#[derive(
62 + Debug,
63 + Clone,
64 + Copy,
65 + PartialEq,
66 + Eq,
67 + PartialOrd,
68 + Ord,
69 + Default,
70 + DeserializeFromStr,
71 + SerializeDisplay,
72 +)]
73 +pub enum ToolStrictLevel {
74 + /// Derive tool constraints from the request's tool choice and per-tool strictness.
75 + #[default]
76 + Auto,
77 + /// Constrain the tool-call envelope for every request with tools.
78 + Function,
79 + /// Additionally pin argument schemas, as if every tool were `strict: true`.
80 + Parameter,
81 +}
82 +
83 +impl ToolStrictLevel {
84 + pub const AUTO_LITERAL: &str = "auto";
85 + pub const FUNCTION_LITERAL: &str = "function";
86 + pub const PARAMETER_LITERAL: &str = "parameter";
87 +}
88 +
89 +impl FromStr for ToolStrictLevel {
90 + type Err = String;
91 +
92 + fn from_str(value: &str) -> Result<Self, Self::Err> {
93 + if value.eq_ignore_ascii_case(Self::AUTO_LITERAL) {
94 + Ok(Self::Auto)
95 + } else if value.eq_ignore_ascii_case(Self::FUNCTION_LITERAL) {
96 + Ok(Self::Function)
97 + } else if value.eq_ignore_ascii_case(Self::PARAMETER_LITERAL) {
98 + Ok(Self::Parameter)
99 + } else {
100 + Err(format!(
101 + "unknown tool strict level {value:?}; expected one of {}, {}, {}",
102 + Self::AUTO_LITERAL,
103 + Self::FUNCTION_LITERAL,
104 + Self::PARAMETER_LITERAL
105 + ))
106 + }
107 + }
108 +}
109 +
110 +impl fmt::Display for ToolStrictLevel {
111 + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
112 + f.write_str(match self {
113 + Self::Auto => Self::AUTO_LITERAL,
114 + Self::Function => Self::FUNCTION_LITERAL,
115 + Self::Parameter => Self::PARAMETER_LITERAL,
116 + })
117 + }
118 +}
119 +
120 /// Validate explicit parser override names without starting request processing.
121 pub fn validate_parser_overrides(
122 tool_call_parser: &ParserSelection,
rust/src/chat/tests/chat.rs
+1
@@ -227,6 +227,7 @@ impl ChatBackend for FakeChatBackend {
227 self.tokenizer(),
228 options.tool_call_parser,
229 options.reasoning_parser,
230 + options.tool_strict_level,
231 )?))
232 }
233 }
rust/src/chat/tests/roundtrip.rs
+1
@@ -717,6 +717,7 @@ async fn parse_completion(
717 let processor = backends.chat_backend.new_chat_output_processor(
718 &mut request,
719 NewChatOutputProcessorOptions {
720 + tool_strict_level: vllm_chat::ToolStrictLevel::Auto,
721 tool_call_parser: &case.tool_call_parser,
722 reasoning_parser: &case.reasoning_parser,
723 },
rust/src/cmd/src/cli.rs
+16
@@ -24,6 +24,7 @@ use serde_with::{DefaultOnNull, OneOrMany, serde_as};
24 use thiserror_ext::AsReport as _;
25 use uuid::Uuid;
26 use vllm_chat::GenerationConfigMode;
27 +use vllm_chat::ToolStrictLevel;
28 use vllm_chat::multimodal::MmLimitPerPrompt;
29 use vllm_engine_core_client::TransportMode;
30 use vllm_managed_engine::ManagedEngineConfig;
@@ -134,6 +135,10 @@ pub struct RenderArgs {
135 /// `none` to disable parsing.
136 #[arg(long, default_value_t)]
137 reasoning_parser: ParserSelection,
138 + /// Server-side floor for structural-tag based tool calling: `auto`,
139 + /// `function`, or `parameter`.
140 + #[arg(long, default_value_t)]
141 + tool_strict_level: ToolStrictLevel,
142 /// Select the native chat renderer implementation.
143 #[arg(long = "tokenizer-mode", default_value_t)]
144 renderer: RendererSelection,
@@ -171,6 +176,7 @@ impl RenderArgs {
176 port: self.port,
177 tool_call_parser: self.tool_call_parser,
178 reasoning_parser: self.reasoning_parser,
179 + tool_strict_level: self.tool_strict_level,
180 renderer: self.renderer,
181 chat_template: self.chat_template,
182 default_chat_template_kwargs: self.default_chat_template_kwargs.unwrap_or_default(),
@@ -241,6 +247,14 @@ pub struct SharedRuntimeArgs {
247 #[arg(long, default_value_t)]
248 #[serde(default = "default_py_bootstrap_parser_selection")]
249 pub reasoning_parser: ParserSelection,
250 + /// Server-side floor for structural-tag based tool calling, applied on
251 + /// top of the per-tool `strict` field: `auto` follows the request's
252 + /// tool choice and per-tool strictness,
253 + /// `function` constrains the tool-call envelope for every request with
254 + /// tools, `parameter` additionally pins argument schemas.
255 + #[arg(long, default_value_t)]
256 + #[serde(default)]
257 + pub tool_strict_level: ToolStrictLevel,
258 /// Select the chat renderer implementation.
259 #[arg(long = "tokenizer-mode", default_value_t)]
260 #[serde(default, rename = "tokenizer_mode")]
@@ -517,6 +531,7 @@ impl SharedRuntimeArgs {
531 listener_mode: HttpListenerMode::InheritedFd { fd: listen_fd },
532 tool_call_parser: self.tool_call_parser,
533 reasoning_parser: self.reasoning_parser,
534 + tool_strict_level: self.tool_strict_level,
535 renderer: self.renderer,
536 language_model_only: self.language_model_only,
537 chat_template: self.chat_template,
@@ -574,6 +589,7 @@ impl SharedRuntimeArgs {
589 listener_mode,
590 tool_call_parser: self.tool_call_parser,
591 reasoning_parser: self.reasoning_parser,
592 + tool_strict_level: self.tool_strict_level,
593 renderer: self.renderer,
594 language_model_only: self.language_model_only,
595 chat_template: self.chat_template,
rust/src/cmd/src/cli/tests.rs
+6
@@ -184,6 +184,7 @@ fn serve_args_forward_python_flags_with_separator() {
184 engine_ready_timeout_secs: 600,
185 tool_call_parser: Auto,
186 reasoning_parser: Auto,
187 + tool_strict_level: Auto,
188 renderer: Auto,
189 language_model_only: false,
190 max_logprobs: None,
@@ -1023,6 +1024,7 @@ fn frontend_args_accept_json() {
1024 engine_ready_timeout_secs: 600,
1025 tool_call_parser: None,
1026 reasoning_parser: None,
1027 + tool_strict_level: Auto,
1028 renderer: Auto,
1029 language_model_only: false,
1030 max_logprobs: None,
@@ -1693,6 +1695,7 @@ fn serve_args_accept_handshake_aliases() {
1695 engine_ready_timeout_secs: 600,
1696 tool_call_parser: Auto,
1697 reasoning_parser: Auto,
1698 + tool_strict_level: Auto,
1699 renderer: Auto,
1700 language_model_only: false,
1701 max_logprobs: None,
@@ -1850,6 +1853,7 @@ fn serve_frontend_config_uses_dp_address_as_advertised_host() {
1853 },
1854 tool_call_parser: Auto,
1855 reasoning_parser: Auto,
1856 + tool_strict_level: Auto,
1857 renderer: Auto,
1858 language_model_only: false,
1859 chat_template: None,
@@ -1942,6 +1946,7 @@ fn serve_frontend_config_keeps_tcp_transport_for_non_local_only_topology() {
1946 },
1947 tool_call_parser: Auto,
1948 reasoning_parser: Auto,
1949 + tool_strict_level: Auto,
1950 renderer: Auto,
1951 language_model_only: false,
1952 chat_template: None,
@@ -2055,6 +2060,7 @@ fn frontend_config_uses_external_coordinator_when_coordinator_address_is_present
2060 },
2061 tool_call_parser: None,
2062 reasoning_parser: None,
2063 + tool_strict_level: Auto,
2064 renderer: Auto,
2065 language_model_only: false,
2066 chat_template: None,
rust/src/server/examples/external_engine_openai_qwen.rs
+2
@@ -16,6 +16,7 @@ use clap::Parser;
16 use futures::StreamExt as _;
17 use tokio_util::sync::CancellationToken;
18 use tracing_subscriber::EnvFilter;
19 +use vllm_chat::ToolStrictLevel;
20 use vllm_engine_core_client::TransportMode;
21 use vllm_server::{
22 ApiServerOptions, ChatTemplateContentFormatOption, Config, CoordinatorMode, CorsConfig,
@@ -70,6 +71,7 @@ async fn main() -> Result<()> {
71 },
72 tool_call_parser: ParserSelection::Auto,
73 reasoning_parser: ParserSelection::Auto,
74 + tool_strict_level: ToolStrictLevel::Auto,
75 renderer: RendererSelection::Auto,
76 language_model_only: false,
77 chat_template: None,
rust/src/server/src/config.rs
+3
@@ -14,6 +14,7 @@ use serde_json::Value;
14 use vllm_chat::multimodal::MmLimitPerPrompt;
15 use vllm_chat::{
16 ChatTemplateContentFormatOption, GenerationConfigMode, ParserSelection, RendererSelection,
17 + ToolStrictLevel,
18 };
19 use vllm_engine_core_client::{CoordinatorMode as EngineCoreCoordinatorMode, TransportMode};
20 use vllm_text::backend::hf::HfOverrides;
@@ -229,6 +230,8 @@ pub struct Config {
230 pub tool_call_parser: ParserSelection,
231 /// Reasoning parser selection.
232 pub reasoning_parser: ParserSelection,
233 + /// Server-side floor for tool-call structural tags.
234 + pub tool_strict_level: ToolStrictLevel,
235 /// Chat renderer selection.
236 pub renderer: RendererSelection,
237 /// Disable frontend-side multimodal preprocessing and render the model as
rust/src/server/src/grpc/tests.rs
+2
@@ -244,6 +244,7 @@ impl ChatBackend for FakeTextBackend {
244 self.tokenizer(),
245 options.tool_call_parser,
246 options.reasoning_parser,
247 + options.tool_strict_level,
248 )?))
249 }
250 }
@@ -292,6 +293,7 @@ impl ChatBackend for FakeMultimodalBackend {
293 self.tokenizer(),
294 options.tool_call_parser,
295 options.reasoning_parser,
296 + options.tool_strict_level,
297 )?))
298 }
299 }
rust/src/server/src/lib.rs
+2 -1
@@ -139,7 +139,8 @@ async fn build_state(config: &Config) -> Result<Arc<AppState>> {
139
140 let chat = ChatLlm::new(text, chat_backend)
141 .with_tool_call_parser(config.tool_call_parser.clone())
142 - .with_reasoning_parser(config.reasoning_parser.clone());
142 + .with_reasoning_parser(config.reasoning_parser.clone())
143 + .with_tool_strict_level(config.tool_strict_level);
144
145 let state = Arc::new(
146 AppState::new(served_model_names, chat)
rust/src/server/src/render.rs
+9 -5
@@ -10,7 +10,7 @@ use tokio_util::sync::CancellationToken;
10 use tracing::info;
11 use vllm_chat::{
12 ChatRequestProcessor, ChatTemplateContentFormatOption, LoadModelBackendsOptions,
13 - ParserSelection, RendererSelection, load_model_backends,
13 + ParserSelection, RendererSelection, ToolStrictLevel, load_model_backends,
14 };
15 use vllm_text::TextRequestProcessor;
16 use vllm_text::backend::hf::HfOverrides;
@@ -32,6 +32,7 @@ pub struct RenderConfig {
32 pub port: u16,
33 pub tool_call_parser: ParserSelection,
34 pub reasoning_parser: ParserSelection,
35 + pub tool_strict_level: ToolStrictLevel,
36 pub renderer: RendererSelection,
37 pub chat_template: Option<String>,
38 pub default_chat_template_kwargs: HashMap<String, Value>,
@@ -88,10 +89,12 @@ async fn build_state(config: &RenderConfig) -> Result<Arc<RenderState>> {
89 let max_model_len = config.max_model_len.unwrap_or(u32::MAX);
90 let text = TextRequestProcessor::new(loaded.text_backend, max_model_len)
91 .with_max_logprobs(config.max_logprobs);
91 - let chat = ChatRequestProcessor::render_only(loaded.chat_backend).with_parser_selections(
92 - config.tool_call_parser.clone(),
93 - config.reasoning_parser.clone(),
94 - );
92 + let chat = ChatRequestProcessor::render_only(loaded.chat_backend)
93 + .with_parser_selections(
94 + config.tool_call_parser.clone(),
95 + config.reasoning_parser.clone(),
96 + )
97 + .with_tool_strict_level(config.tool_strict_level);
98 Ok(Arc::new(RenderState {
99 model: config.model.clone(),
100 served_model_names,
@@ -154,6 +157,7 @@ mod tests {
157 shutdown.cancel();
158 let error = serve_render(
159 RenderConfig {
160 + tool_strict_level: ToolStrictLevel::Auto,
161 model: "test-model".to_string(),
162 revision: None,
163 hf_overrides: Default::default(),
rust/src/server/src/routes/http_client_tests.rs
+1
@@ -168,6 +168,7 @@ impl ChatBackend for FakeChatBackend {
168 self.tokenizer(),
169 options.tool_call_parser,
170 options.reasoning_parser,
171 + options.tool_strict_level,
172 )?))
173 }
174 }
rust/src/server/src/routes/tests.rs
+1
@@ -517,6 +517,7 @@ impl ChatBackend for FakeChatBackend {
517 self.tokenizer(),
518 options.tool_call_parser,
519 options.reasoning_parser,
520 + options.tool_strict_level,
521 )?))
522 }
523 }
tests/tool_parsers/test_structural_tag_registry.py
+250 -4
@@ -1,6 +1,7 @@
1 # SPDX-License-Identifier: Apache-2.0
2 # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3
4 +import json
5 from types import SimpleNamespace
6 from unittest.mock import MagicMock
7
@@ -33,9 +34,11 @@ from vllm.tool_parsers.structural_tag_registry import (
34 SUPPORTED_STRUCTURAL_TAG_MODELS,
35 VLLM_BUILTIN_STRUCTURAL_TAG_MODELS,
36 XGRAMMAR_BUILTIN_STRUCTURAL_TAG_MODELS,
37 + ToolChoice,
38 get_function_parameters,
39 get_model_structural_tag,
40 )
41 +from vllm.tool_parsers.tool_strict_level import ToolStrictLevel
42
43
44 @pytest.fixture
@@ -231,11 +234,11 @@ def test_get_model_structural_tag_supports_all_xgrammar_builtins(
234
235
236 def test_get_model_structural_tag_supports_vllm_hermes(
234 - sample_tools: list[ChatCompletionToolsParam],
237 + sample_tools_strict: list[ChatCompletionToolsParam],
238 ):
239 tag = get_model_structural_tag(
240 model="hermes",
238 - tools=sample_tools,
241 + tools=sample_tools_strict,
242 tool_choice="required",
243 reasoning=False,
244 )
@@ -273,6 +276,22 @@ def test_get_model_structural_tag_supports_vllm_hermes(
276 assert content["json_schema"] == expected_schema
277
278
279 +def test_hermes_required_without_strict_leaves_arguments_free(
280 + sample_tools: list[ChatCompletionToolsParam],
281 +):
282 + tag = get_model_structural_tag(
283 + model="hermes",
284 + tools=sample_tools,
285 + tool_choice="required",
286 + reasoning=False,
287 + )
288 +
289 + assert isinstance(tag, StructuralTag)
290 + tags = tag.model_dump()["format"]["tags"]
291 + assert tags
292 + assert all(tag_dump["content"]["json_schema"] is True for tag_dump in tags)
293 +
294 +
295 def test_hermes_required_tool_calls_use_empty_separator():
296 tools = [
297 ChatCompletionToolsParam(
@@ -361,10 +380,19 @@ def _k3_tools(*calls: str) -> str:
380 return f"{_K3_TOOLS_OPEN}{''.join(calls)}{_K3_TOOLS_CLOSE}"
381
382
364 -def _k3_grammar(tool_choice, tools=None):
383 +def _as_strict(tools: list[ChatCompletionToolsParam]) -> list[ChatCompletionToolsParam]:
384 + for tool in tools:
385 + tool.function.strict = True
386 + return tools
387 +
388 +
389 +def _k3_grammar(tool_choice, tools=None, *, strict: bool = True):
390 + """Build the K3 grammar; tools are strict by default so the typed
391 + argument channel is exercised (an unset ``strict`` leaves arguments free)."""
392 + tools = tools if tools is not None else _k3_tools_by_name()
393 tag = get_model_structural_tag(
394 model="kimi_k3",
367 - tools=tools if tools is not None else _k3_tools_by_name(),
395 + tools=_as_strict(tools) if strict else tools,
396 tool_choice=tool_choice,
397 reasoning=False,
398 )
@@ -467,6 +495,18 @@ def test_kimi_k3_required_rejects_invalid(body: str):
495 assert not _is_grammar_accept_string(_k3_grammar("required"), body)
496
497
498 +def test_kimi_k3_required_without_strict_leaves_arguments_free():
499 + # An unset ``strict`` pins the call envelope only: a value the schema
500 + # would reject is accepted, and the declared-required argument may be
501 + # omitted.
502 + grammar = _k3_grammar("required", strict=False)
503 + body = _k3_response() + _k3_tools(
504 + _k3_call("get_weather", _k3_arg("days", "number", "abc"))
505 + )
506 + assert _is_grammar_accept_string(grammar, body)
507 + assert not _is_grammar_accept_string(_k3_grammar("required"), body)
508 +
509 +
510 def test_kimi_k3_schema_without_required_accepts_empty_call():
511 tools = [
512 ChatCompletionToolsParam(
@@ -829,3 +869,209 @@ def test_kimi_k3_forced_tool_choice_builds_single_mandatory_call():
869 response_only = _k3_response("no call here")
870 assert _is_grammar_accept_string(grammar, ok)
871 assert not _is_grammar_accept_string(grammar, response_only)
872 +
873 +
874 +def _pins_argument_schema(tag: StructuralTag) -> bool:
875 + return '"json_schema": {' in json.dumps(tag.model_dump(), ensure_ascii=False)
876 +
877 +
878 +def _dumped(tag: StructuralTag) -> str:
879 + return json.dumps(tag.model_dump(), ensure_ascii=False)
880 +
881 +
882 +def test_tool_strict_level_auto_is_the_default(
883 + sample_tools: list[ChatCompletionToolsParam],
884 +):
885 + """Auto + no strict tool gets no tag unless the operator raises the floor."""
886 + assert (
887 + get_model_structural_tag(
888 + model="deepseek_v4",
889 + tools=sample_tools,
890 + tool_choice="auto",
891 + reasoning=False,
892 + )
893 + is None
894 + )
895 +
896 +
897 +@pytest.mark.parametrize("model", sorted(SUPPORTED_STRUCTURAL_TAG_MODELS))
898 +def test_tool_strict_level_function_lifts_auto_gate(
899 + model: str,
900 + sample_tools: list[ChatCompletionToolsParam],
901 +):
902 + tag = get_model_structural_tag(
903 + model=model,
904 + tools=sample_tools,
905 + tool_choice="auto",
906 + reasoning=False,
907 + strict_level=ToolStrictLevel.FUNCTION,
908 + )
909 +
910 + assert tag is not None
911 +
912 +
913 +@pytest.mark.parametrize("model", sorted(XGRAMMAR_BUILTIN_STRUCTURAL_TAG_MODELS))
914 +def test_tool_strict_level_function_pins_envelope_only(
915 + model: str,
916 + sample_tools: list[ChatCompletionToolsParam],
917 +):
918 + """The request's own tools stay untouched; unset ``strict`` stays free."""
919 + tag = get_model_structural_tag(
920 + model=model,
921 + tools=sample_tools,
922 + tool_choice="auto",
923 + reasoning=False,
924 + strict_level=ToolStrictLevel.FUNCTION,
925 + )
926 +
927 + assert tag is not None
928 + assert not _pins_argument_schema(tag)
929 + assert all(tool.function.strict is None for tool in sample_tools)
930 +
931 +
932 +@pytest.mark.parametrize("model", sorted(XGRAMMAR_BUILTIN_STRUCTURAL_TAG_MODELS))
933 +def test_tool_strict_level_parameter_pins_argument_schemas(
934 + model: str,
935 + sample_tools: list[ChatCompletionToolsParam],
936 +):
937 + tag = get_model_structural_tag(
938 + model=model,
939 + tools=sample_tools,
940 + tool_choice="auto",
941 + reasoning=False,
942 + strict_level=ToolStrictLevel.PARAMETER,
943 + )
944 +
945 + assert tag is not None
946 + assert _pins_argument_schema(tag)
947 +
948 +
949 +def test_tool_strict_level_function_keeps_client_strict_tools_strict(
950 + sample_tools_strict: list[ChatCompletionToolsParam],
951 +):
952 + tag = get_model_structural_tag(
953 + model="deepseek_v4",
954 + tools=sample_tools_strict,
955 + tool_choice="auto",
956 + reasoning=False,
957 + strict_level=ToolStrictLevel.FUNCTION,
958 + )
959 +
960 + assert tag is not None
961 + assert _pins_argument_schema(tag)
962 +
963 +
964 +@pytest.mark.parametrize("model", sorted(XGRAMMAR_BUILTIN_STRUCTURAL_TAG_MODELS))
965 +def test_absent_strict_is_not_pinned_next_to_a_strict_tool(model: str):
966 + """A strict tool must not drag a neighbour with unset ``strict`` into
967 + schema enforcement."""
968 + weather = ChatCompletionToolsParam(
969 + type="function",
970 + function={
971 + "name": "get_weather",
972 + "strict": True,
973 + "parameters": {
974 + "type": "object",
975 + "properties": {"city": {"type": "string"}},
976 + "required": ["city"],
977 + },
978 + },
979 + )
980 + search = ChatCompletionToolsParam(
981 + type="function",
982 + function={
983 + "name": "search",
984 + "parameters": {
985 + "type": "object",
986 + "properties": {"query": {"type": "string"}},
987 + "required": ["query"],
988 + },
989 + },
990 + )
991 + tag = get_model_structural_tag(
992 + model=model,
993 + tools=[weather, search],
994 + tool_choice="auto",
995 + reasoning=False,
996 + )
997 +
998 + assert tag is not None
999 + dumped = _dumped(tag)
1000 + assert '"city"' in dumped
1001 + assert '"query"' not in dumped
1002 +
1003 +
1004 +@pytest.mark.parametrize(
1005 + "tool_choice",
1006 + [
1007 + "required",
1008 + ChatCompletionNamedToolChoiceParam(
1009 + function=ChatCompletionNamedFunction(name="get_weather")
1010 + ),
1011 + ],
1012 +)
1013 +@pytest.mark.parametrize("model", sorted(XGRAMMAR_BUILTIN_STRUCTURAL_TAG_MODELS))
1014 +def test_forced_tool_choice_pins_schema_only_for_strict_tools(
1015 + model: str,
1016 + tool_choice: ToolChoice,
1017 + sample_tools: list[ChatCompletionToolsParam],
1018 + sample_tools_strict: list[ChatCompletionToolsParam],
1019 +):
1020 + """Required / named always constrain the call; the argument schema
1021 + follows the per-tool ``strict`` (or the parameter level)."""
1022 + free = get_model_structural_tag(
1023 + model=model,
1024 + tools=sample_tools,
1025 + tool_choice=tool_choice,
1026 + reasoning=False,
1027 + strict_level=ToolStrictLevel.FUNCTION,
1028 + )
1029 + pinned = get_model_structural_tag(
1030 + model=model,
1031 + tools=sample_tools_strict,
1032 + tool_choice=tool_choice,
1033 + reasoning=False,
1034 + )
1035 + floored = get_model_structural_tag(
1036 + model=model,
1037 + tools=sample_tools,
1038 + tool_choice=tool_choice,
1039 + reasoning=False,
1040 + strict_level=ToolStrictLevel.PARAMETER,
1041 + )
1042 +
1043 + assert free is not None and not _pins_argument_schema(free)
1044 + assert pinned is not None and _pins_argument_schema(pinned)
1045 + assert floored is not None and _pins_argument_schema(floored)
1046 +
1047 +
1048 +def test_tool_strict_level_parameter_overrides_client_strict_false():
1049 + tools = [
1050 + ChatCompletionToolsParam(
1051 + type="function",
1052 + function={
1053 + "name": "get_weather",
1054 + "parameters": {"type": "object", "properties": {}},
1055 + "strict": False,
1056 + },
1057 + )
1058 + ]
1059 + tag = get_model_structural_tag(
1060 + model="deepseek_v4",
1061 + tools=tools,
1062 + tool_choice="required",
1063 + reasoning=False,
1064 + strict_level=ToolStrictLevel.PARAMETER,
1065 + )
1066 +
1067 + assert tag is not None
1068 + assert _pins_argument_schema(tag)
1069 +
1070 +
1071 +def test_tool_strict_level_from_name():
1072 + assert ToolStrictLevel.from_name("AUTO") is ToolStrictLevel.AUTO
1073 + assert ToolStrictLevel.from_name("Parameter") is ToolStrictLevel.PARAMETER
1074 + with pytest.raises(ValueError, match="expected one of auto, function, parameter"):
1075 + ToolStrictLevel.from_name("strict")
1076 + with pytest.raises(ValueError, match="expected one of auto, function, parameter"):
1077 + ToolStrictLevel.from_name("off")
vllm/entrypoints/anthropic/serving.py
+2
@@ -112,6 +112,7 @@ class AnthropicServingMessages(OpenAIServingChat):
112 reasoning_parser: str = "",
113 enable_auto_tools: bool = False,
114 tool_parser: str | None = None,
115 + tool_strict_level: str = "auto",
116 enable_prompt_tokens_details: bool = False,
117 enable_force_include_usage: bool = False,
118 default_chat_template_kwargs: dict[str, Any] | None = None,
@@ -127,6 +128,7 @@ class AnthropicServingMessages(OpenAIServingChat):
128 return_tokens_as_token_ids=return_tokens_as_token_ids,
129 reasoning_parser=reasoning_parser,
130 enable_auto_tools=enable_auto_tools,
131 + tool_strict_level=tool_strict_level,
132 tool_parser=tool_parser,
133 enable_prompt_tokens_details=enable_prompt_tokens_details,
134 enable_force_include_usage=enable_force_include_usage,
vllm/entrypoints/cohere/serving.py
+2
@@ -249,6 +249,7 @@ class CohereServingChatV2(OpenAIServingChat):
249 reasoning_parser: str = "",
250 enable_auto_tools: bool = False,
251 tool_parser: str | None = None,
252 + tool_strict_level: str = "auto",
253 enable_prompt_tokens_details: bool = False,
254 enable_force_include_usage: bool = False,
255 default_chat_template_kwargs: dict[str, Any] | None = None,
@@ -265,6 +266,7 @@ class CohereServingChatV2(OpenAIServingChat):
266 return_tokens_as_token_ids=return_tokens_as_token_ids,
267 reasoning_parser=reasoning_parser,
268 enable_auto_tools=enable_auto_tools,
269 + tool_strict_level=tool_strict_level,
270 tool_parser=tool_parser,
271 enable_prompt_tokens_details=enable_prompt_tokens_details,
272 enable_force_include_usage=enable_force_include_usage,
vllm/entrypoints/generate/api_router.py
+4
@@ -112,6 +112,7 @@ async def init_generate_state(
112 return_tokens_as_token_ids=args.return_tokens_as_token_ids,
113 enable_auto_tools=args.enable_auto_tool_choice,
114 tool_parser=args.tool_call_parser,
115 + tool_strict_level=args.tool_strict_level,
116 tool_server=state.tool_server,
117 reasoning_parser=args.structured_outputs_config.reasoning_parser,
118 enable_prompt_tokens_details=args.enable_prompt_tokens_details,
@@ -137,6 +138,7 @@ async def init_generate_state(
138 enable_auto_tools=args.enable_auto_tool_choice,
139 exclude_tools_when_tool_choice_none=args.exclude_tools_when_tool_choice_none,
140 tool_parser=args.tool_call_parser,
141 + tool_strict_level=args.tool_strict_level,
142 reasoning_parser=args.structured_outputs_config.reasoning_parser,
143 enable_prompt_tokens_details=args.enable_prompt_tokens_details,
144 enable_force_include_usage=args.enable_force_include_usage,
@@ -178,6 +180,7 @@ async def init_generate_state(
180 return_tokens_as_token_ids=args.return_tokens_as_token_ids,
181 enable_auto_tools=args.enable_auto_tool_choice,
182 tool_parser=args.tool_call_parser,
183 + tool_strict_level=args.tool_strict_level,
184 reasoning_parser=args.structured_outputs_config.reasoning_parser,
185 enable_prompt_tokens_details=args.enable_prompt_tokens_details,
186 enable_force_include_usage=args.enable_force_include_usage,
@@ -198,6 +201,7 @@ async def init_generate_state(
201 return_tokens_as_token_ids=args.return_tokens_as_token_ids,
202 enable_auto_tools=args.enable_auto_tool_choice,
203 tool_parser=args.tool_call_parser,
204 + tool_strict_level=args.tool_strict_level,
205 reasoning_parser=args.structured_outputs_config.reasoning_parser,
206 enable_prompt_tokens_details=args.enable_prompt_tokens_details,
207 enable_force_include_usage=args.enable_force_include_usage,
vllm/entrypoints/launchers/api_server/app_state.py
+2
@@ -95,6 +95,7 @@ async def init_app_state(
95 enable_auto_tools=args.enable_auto_tool_choice,
96 exclude_tools_when_tool_choice_none=args.exclude_tools_when_tool_choice_none,
97 tool_parser=args.tool_call_parser,
98 + tool_strict_level=args.tool_strict_level,
99 reasoning_parser=args.structured_outputs_config.reasoning_parser,
100 default_chat_template_kwargs=default_chat_template_kwargs,
101 log_error_stack=args.log_error_stack,
@@ -111,6 +112,7 @@ async def init_app_state(
112 enable_auto_tools=args.enable_auto_tool_choice,
113 exclude_tools_when_tool_choice_none=args.exclude_tools_when_tool_choice_none,
114 tool_parser=args.tool_call_parser,
115 + tool_strict_level=args.tool_strict_level,
116 reasoning_parser=args.structured_outputs_config.reasoning_parser,
117 default_chat_template_kwargs=default_chat_template_kwargs,
118 log_error_stack=args.log_error_stack,
vllm/entrypoints/launchers/cli_args.py
+8
@@ -20,6 +20,7 @@ from vllm.entrypoints.chat_utils import (
20 )
21 from vllm.entrypoints.openai.models.protocol import LoRAModulePath
22 from vllm.tool_parsers import ToolParserManager
23 +from vllm.tool_parsers.tool_strict_level import ToolStrictLevelName
24 from vllm.utils.argparse_utils import FlexibleArgumentParser
25
26 from .utils.constants import (
@@ -116,6 +117,13 @@ class BaseFrontendArgs:
117 """Special the tool parser plugin write to parse the model-generated tool
118 into OpenAI API format, the name register in this plugin can be used in
119 `--tool-call-parser`."""
120 + tool_strict_level: ToolStrictLevelName = "auto"
121 + """Server-side floor for structural-tag based tool calling, applied on top
122 + of the per-tool `strict` field. `auto` follows the request's tool choice
123 + and per-tool strictness; `function` constrains the
124 + tool-call envelope (markup and function name) for every request with
125 + tools; `parameter` additionally pins argument schemas, as if every tool
126 + were `strict: true`."""
127 tool_server: str | None = None
128 """Comma-separated list of host:port pairs (IPv4, IPv6, or hostname).
129 Examples: 127.0.0.1:8000, [::1]:8000, localhost:1234. Or `demo` for
vllm/entrypoints/launchers/render/app_state.py
+2
@@ -60,6 +60,7 @@ async def init_render_app_state(
60 enable_auto_tools=args.enable_auto_tool_choice,
61 exclude_tools_when_tool_choice_none=args.exclude_tools_when_tool_choice_none,
62 tool_parser=args.tool_call_parser,
63 + tool_strict_level=args.tool_strict_level,
64 reasoning_parser=args.reasoning_parser,
65 default_chat_template_kwargs=default_chat_template_kwargs,
66 log_error_stack=args.log_error_stack,
@@ -76,6 +77,7 @@ async def init_render_app_state(
77 enable_auto_tools=args.enable_auto_tool_choice,
78 exclude_tools_when_tool_choice_none=args.exclude_tools_when_tool_choice_none,
79 tool_parser=args.tool_call_parser,
80 + tool_strict_level=args.tool_strict_level,
81 reasoning_parser=args.reasoning_parser,
82 default_chat_template_kwargs=default_chat_template_kwargs,
83 log_error_stack=args.log_error_stack,
vllm/entrypoints/openai/chat_completion/serving.py
+2
@@ -132,6 +132,7 @@ class OpenAIServingChat(GenerateBaseServing):
132 enable_auto_tools: bool = False,
133 exclude_tools_when_tool_choice_none: bool = False,
134 tool_parser: str | None = None,
135 + tool_strict_level: str = "auto",
136 enable_prompt_tokens_details: bool = False,
137 enable_force_include_usage: bool = False,
138 enable_log_outputs: bool = False,
@@ -161,6 +162,7 @@ class OpenAIServingChat(GenerateBaseServing):
162 tool_parser_name=tool_parser,
163 reasoning_parser_name=reasoning_parser,
164 enable_auto_tools=enable_auto_tools,
165 + tool_strict_level=tool_strict_level,
166 model_name=self.model_config.model,
167 is_harmony=self.model_config.hf_config.model_type == "gpt_oss",
168 )
vllm/entrypoints/openai/responses/serving.py
+2
@@ -109,6 +109,7 @@ class OpenAIServingResponses(GenerateBaseServing):
109 reasoning_parser: str = "",
110 enable_auto_tools: bool = False,
111 tool_parser: str | None = None,
112 + tool_strict_level: str = "auto",
113 tool_server: ToolServer | None = None,
114 enable_prompt_tokens_details: bool = False,
115 enable_force_include_usage: bool = False,
@@ -135,6 +136,7 @@ class OpenAIServingResponses(GenerateBaseServing):
136 tool_parser_name=tool_parser,
137 reasoning_parser_name=reasoning_parser,
138 enable_auto_tools=enable_auto_tools,
139 + tool_strict_level=tool_strict_level,
140 model_name=self.model_config.model,
141 is_harmony=self.model_config.hf_config.model_type == "gpt_oss",
142 )
vllm/parser/abstract_parser.py
+4
@@ -36,6 +36,7 @@ from vllm.tool_parsers.streaming import (
36 extract_named_tool_call_streaming,
37 extract_required_tool_call_streaming,
38 )
39 +from vllm.tool_parsers.tool_strict_level import ToolStrictLevel
40
41 logger = init_logger(__name__)
42
@@ -95,6 +96,8 @@ class Parser:
96 # Subclasses should override these if they use specific parser classes
97 reasoning_parser_cls: type[ReasoningParser] | None = None
98 tool_parser_cls: type[ToolParser] | None = None
99 + # Server-side floor for tool-call structural tags (--tool-strict-level).
100 + tool_strict_level: ToolStrictLevel = ToolStrictLevel.AUTO
101
102 def __init__(
103 self,
@@ -415,6 +418,7 @@ class DelegatingParser(Parser):
418 structure_tag = self._tool_parser.get_structural_tag(
419 request,
420 reasoning=False,
421 + strict_level=self.tool_strict_level,
422 )
423 if structure_tag is None:
424 return request
vllm/parser/parser_manager.py
+10
@@ -6,6 +6,7 @@ from __future__ import annotations
6 from typing import TYPE_CHECKING
7
8 from vllm.logger import init_logger
9 +from vllm.tool_parsers.tool_strict_level import ToolStrictLevel
10
11 if TYPE_CHECKING:
12 from vllm.parser.abstract_parser import Parser
@@ -77,6 +78,7 @@ class ParserManager:
78 enable_auto_tools: bool = False,
79 model_name: str | None = None,
80 is_harmony: bool = False,
81 + tool_strict_level: str = "auto",
82 ) -> type[Parser] | None:
83 """Get a Parser that handles both reasoning and tool parsing.
84
@@ -89,6 +91,8 @@ class ParserManager:
91 model_name: The model name for parser-specific warnings.
92 is_harmony: Whether the selected model uses the Harmony format.
93 If True, HarmonyParser is always returned.
94 + tool_strict_level: Server-side floor for tool-call structural
95 + tags (``--tool-strict-level``).
96
97 Returns:
98 A Parser class, or None if neither parser is specified.
@@ -105,11 +109,14 @@ class ParserManager:
109 if reasoning_parser_cls is None and tool_parser_cls is None:
110 return None
111
112 + strict_level = ToolStrictLevel.from_name(tool_strict_level)
113 +
114 if is_harmony:
115 from vllm.parser.harmony import HarmonyParser
116
117 HarmonyParser.reasoning_parser_cls = reasoning_parser_cls
118 HarmonyParser.tool_parser_cls = tool_parser_cls
119 + HarmonyParser.tool_strict_level = strict_level
120 return HarmonyParser
121
122 if reasoning_parser_name == "kimi_k3" or tool_parser_name == "kimi_k3":
@@ -121,6 +128,7 @@ class ParserManager:
128 class _KimiK3Parser(KimiK3Parser):
129 reasoning_parser_cls = r_cls
130 tool_parser_cls = t_cls
131 + tool_strict_level = strict_level
132
133 return _KimiK3Parser
134
@@ -136,6 +144,7 @@ class ParserManager:
144 class _CohereCommandParser(CohereCommandParser):
145 reasoning_parser_cls = r_cls
146 tool_parser_cls = t_cls
147 + tool_strict_level = strict_level
148
149 return _CohereCommandParser
150
@@ -147,5 +156,6 @@ class ParserManager:
156 class _Parser(DelegatingParser):
157 reasoning_parser_cls = r_cls
158 tool_parser_cls = t_cls
159 + tool_strict_level = strict_level
160
161 return _Parser
vllm/renderers/online_derenderer.py
+2
@@ -54,6 +54,7 @@ class OnlineDerenderer:
54 exclude_tools_when_tool_choice_none: bool = False,
55 tool_parser: str | None = None,
56 reasoning_parser: str | None = None,
57 + tool_strict_level: str = "auto",
58 default_chat_template_kwargs: dict[str, Any] | None = None,
59 log_error_stack: bool = False,
60 ) -> None:
@@ -68,6 +69,7 @@ class OnlineDerenderer:
69 tool_parser_name=tool_parser,
70 reasoning_parser_name=reasoning_parser,
71 enable_auto_tools=enable_auto_tools,
72 + tool_strict_level=tool_strict_level,
73 model_name=model_config.model,
74 is_harmony=self.use_harmony,
75 )
vllm/renderers/online_renderer.py
+2
@@ -127,6 +127,7 @@ class OnlineRenderer:
127 exclude_tools_when_tool_choice_none: bool = False,
128 tool_parser: str | None = None,
129 reasoning_parser: str | None = None,
130 + tool_strict_level: str = "auto",
131 default_chat_template_kwargs: dict[str, Any] | None = None,
132 log_error_stack: bool = False,
133 ) -> None:
@@ -141,6 +142,7 @@ class OnlineRenderer:
142 tool_parser_name=tool_parser,
143 reasoning_parser_name=reasoning_parser,
144 enable_auto_tools=enable_auto_tools,
145 + tool_strict_level=tool_strict_level,
146 model_name=model_config.model,
147 is_harmony=self.use_harmony,
148 )
vllm/tool_parsers/abstract_tool_parser.py
+3
@@ -30,6 +30,7 @@ from vllm.sampling_params import (
30 StructuredOutputsParams,
31 )
32 from vllm.tokenizers import TokenizerLike
33 +from vllm.tool_parsers.tool_strict_level import ToolStrictLevel
34 from vllm.tool_parsers.utils import Tool, get_json_schema_from_tools
35 from vllm.utils.collection_utils import is_list_of
36 from vllm.utils.import_utils import import_plugin
@@ -168,6 +169,7 @@ class ToolParser:
169 request: ChatCompletionRequest | ResponsesRequest,
170 *,
171 reasoning: bool = False,
172 + strict_level: ToolStrictLevel = ToolStrictLevel.AUTO,
173 ):
174 if self.structural_tag_model is None:
175 return None
@@ -180,6 +182,7 @@ class ToolParser:
182 tools=request.tools,
183 tool_choice=request.tool_choice,
184 reasoning=reasoning,
185 + strict_level=strict_level,
186 )
187
188 def extract_tool_calls(
vllm/tool_parsers/hy_v4_tool_parser.py
+4
@@ -25,6 +25,7 @@ from vllm.entrypoints.openai.responses.protocol import ResponsesRequest
25 from vllm.logger import init_logger
26 from vllm.tokenizers import TokenizerLike
27 from vllm.tool_parsers.abstract_tool_parser import ToolParser
28 +from vllm.tool_parsers.tool_strict_level import ToolStrictLevel
29 from vllm.tool_parsers.utils import partial_tag_overlap
30
31 if TYPE_CHECKING:
@@ -994,6 +995,7 @@ class HYV4ToolParser(ToolParser):
995 request: ChatCompletionRequest | ResponsesRequest,
996 *,
997 reasoning: bool = False,
998 + strict_level: ToolStrictLevel = ToolStrictLevel.AUTO,
999 ) -> StructuralTag | None:
1000 """Build a structural tag matching HYV4's tool tokens.
1001
@@ -1009,6 +1011,7 @@ class HYV4ToolParser(ToolParser):
1011 Args:
1012 request: The request being adjusted.
1013 reasoning: Whether the grammar also covers the reasoning phase.
1014 + strict_level: Server-side floor from ``--tool-strict-level``.
1015
1016 Returns:
1017 The structural tag, or None when structural tagging does not apply.
@@ -1041,6 +1044,7 @@ class HYV4ToolParser(ToolParser):
1044 tool_choice=request.tool_choice,
1045 reasoning=reasoning,
1046 token_suffix=self._extractor.token_suffix,
1047 + strict_level=strict_level,
1048 )
1049 except Exception:
1050 logger.warning(
vllm/tool_parsers/structural_tag_registry.py
+52 -7
@@ -35,6 +35,7 @@ from vllm.entrypoints.openai.chat_completion.protocol import (
35 ChatCompletionNamedToolChoiceParam,
36 ChatCompletionToolsParam,
37 )
38 +from vllm.tool_parsers.tool_strict_level import ToolStrictLevel
39
40 ToolChoice: TypeAlias = (
41 Literal["none", "auto", "required"]
@@ -94,15 +95,57 @@ def register_vllm_structural_tag(
95 return decorator
96
97
98 +def _tool_is_strict(tool: ChatCompletionToolsParam | ResponsesTool) -> bool:
99 + if isinstance(tool, FunctionTool):
100 + return tool.strict is True
101 + if isinstance(tool, ChatCompletionToolsParam):
102 + return tool.function.strict is True
103 + return False
104 +
105 +
106 def _any_tool_strict(
107 tools: Sequence[ChatCompletionToolsParam | ResponsesTool],
108 ) -> bool:
100 - for tool in tools:
101 - if isinstance(tool, FunctionTool) and tool.strict is True:
102 - return True
103 - if isinstance(tool, ChatCompletionToolsParam) and tool.function.strict is True:
104 - return True
105 - return False
109 + return any(_tool_is_strict(tool) for tool in tools)
110 +
111 +
112 +def _with_tool_strict(
113 + tool: ChatCompletionToolsParam | ResponsesTool,
114 + strict: bool,
115 +) -> ChatCompletionToolsParam | ResponsesTool:
116 + """Return a copy of ``tool`` with ``strict`` set, leaving the request's alone."""
117 + if isinstance(tool, FunctionTool):
118 + return tool.model_copy(update={"strict": strict})
119 + if isinstance(tool, ChatCompletionToolsParam):
120 + return tool.model_copy(
121 + update={"function": tool.function.model_copy(update={"strict": strict})}
122 + )
123 + return tool
124 +
125 +
126 +def _resolve_tool_strictness(
127 + tools: Sequence[ChatCompletionToolsParam | ResponsesTool],
128 + tool_choice: ToolChoice,
129 + strict_level: ToolStrictLevel,
130 +) -> Sequence[ChatCompletionToolsParam | ResponsesTool] | None:
131 + """Decide whether a structural tag applies and pin each tool's ``strict``.
132 +
133 + A tool without an explicit ``strict`` is treated as non-strict: its call
134 + envelope is still constrained, but its arguments stay free unless the
135 + server level is PARAMETER. ``None`` means no structural tag.
136 + """
137 + if (
138 + tool_choice == "auto"
139 + and strict_level == ToolStrictLevel.AUTO
140 + and not _any_tool_strict(tools)
141 + ):
142 + return None
143 + return [
144 + _with_tool_strict(
145 + tool, strict_level >= ToolStrictLevel.PARAMETER or _tool_is_strict(tool)
146 + )
147 + for tool in tools
148 + ]
149
150
151 def get_model_structural_tag(
@@ -111,12 +154,14 @@ def get_model_structural_tag(
154 tool_choice: ToolChoice,
155 reasoning: bool,
156 token_suffix: str = "",
157 + strict_level: ToolStrictLevel = ToolStrictLevel.AUTO,
158 ) -> StructuralTag | None:
159 """Build a structural tag with xgrammar's builtin model templates."""
160 if not tools or tool_choice == "none":
161 return None
162
119 - if tool_choice == "auto" and not _any_tool_strict(tools):
163 + tools = _resolve_tool_strictness(tools, tool_choice, strict_level)
164 + if tools is None:
165 return None
166
167 dumped_tools = [_dump_tool_for_xgrammar(tool) for tool in tools]
vllm/tool_parsers/tool_strict_level.py new
+32
@@ -0,0 +1,32 @@
1 +# SPDX-License-Identifier: Apache-2.0
2 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3 +
4 +import enum
5 +from typing import Literal
6 +
7 +ToolStrictLevelName = Literal["auto", "function", "parameter"]
8 +
9 +
10 +class ToolStrictLevel(enum.IntEnum):
11 + """Server-side floor for tool-call structural tags (``--tool-strict-level``).
12 +
13 + AUTO: derive call constraints from ``tool_choice`` and argument-schema
14 + enforcement from each tool's explicit ``strict`` field.
15 + FUNCTION: constrain the tool-call envelope for every request with tools.
16 + PARAMETER: additionally pin argument schemas, as if every tool were
17 + ``strict: true``.
18 + """
19 +
20 + AUTO = 0
21 + FUNCTION = 1
22 + PARAMETER = 2
23 +
24 + @classmethod
25 + def from_name(cls, name: str) -> "ToolStrictLevel":
26 + try:
27 + return cls[name.strip().upper()]
28 + except KeyError:
29 + expected = ", ".join(level.name.lower() for level in cls)
30 + raise ValueError(
31 + f"Unknown tool strict level {name!r}; expected one of {expected}."
32 + ) from None