@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 06a760d6e8
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 02:00 UTC

[bugfix] catch xgrammar unsupported array constraints (#12210)

Signed-off-by: Jason Cheng <jasoncky96@gmail.com>

Cheng Kuan Yong Jason committed Jan 21, 2025 at 08:42 UTC 06a760d6e8bcd60dc98775678b5b12eef01d82bb
1 file changed +7
vllm/model_executor/guided_decoding/utils.py
+7
@@ -20,6 +20,13 @@ def has_xgrammar_unsupported_json_features(schema: dict) -> bool:
20 ]):
21 return True
22
23 + # Check for array unsupported keywords
24 + if obj.get("type") == "array" and any(key in obj for key in [
25 + "uniqueItems", "contains", "minContains", "maxContains",
26 + "minItems", "maxItems"
27 + ]):
28 + return True
29 +
30 # Recursively check all nested objects and arrays
31 for value in obj.values():
32 if isinstance(value, dict):