| 1 | # Structured Outputs |
| 2 | |
| 3 | This script demonstrates various structured output capabilities of vLLM's OpenAI-compatible server. |
| 4 | It can run individual constraint type or all of them. |
| 5 | It supports both streaming responses and concurrent non-streaming requests. |
| 6 | |
| 7 | To use this example, you must start an vLLM server with any model of your choice. |
| 8 | |
| 9 | ```bash |
| 10 | vllm serve Qwen/Qwen2.5-3B-Instruct |
| 11 | ``` |
| 12 | |
| 13 | To serve a reasoning model, you can use the following command: |
| 14 | |
| 15 | ```bash |
| 16 | vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-7B \ |
| 17 | --reasoning-parser deepseek_r1 |
| 18 | ``` |
| 19 | |
| 20 | If you want to run this script standalone with `uv`, you can use the following: |
| 21 | |
| 22 | ```bash |
| 23 | uvx --from git+https://github.com/vllm-project/vllm#subdirectory=examples/features/structured_outputs \ |
| 24 | structured-outputs |
| 25 | ``` |
| 26 | |
| 27 | See [feature docs](https://docs.vllm.ai/en/latest/features/structured_outputs.html) for more information. |
| 28 | |
| 29 | !!! tip |
| 30 | If vLLM is running remotely, then set `OPENAI_BASE_URL=<remote_url>` before running the script. |
| 31 | |
| 32 | ## Usage |
| 33 | |
| 34 | Run all constraints, non-streaming: |
| 35 | |
| 36 | ```bash |
| 37 | uv run structured_outputs_offline.py |
| 38 | ``` |
| 39 | |
| 40 | Run all constraints, streaming: |
| 41 | |
| 42 | ```bash |
| 43 | uv run structured_outputs_offline.py --stream |
| 44 | ``` |
| 45 | |
| 46 | Run certain constraints, for example `structural_tag` and `regex`, streaming: |
| 47 | |
| 48 | ```bash |
| 49 | uv run structured_outputs_offline.py \ |
| 50 | --constraint structural_tag regex \ |
| 51 | --stream |
| 52 | ``` |
| 53 | |
| 54 | Run all constraints, with reasoning models and streaming: |
| 55 | |
| 56 | ```bash |
| 57 | uv run structured_outputs_offline.py --reasoning --stream |
| 58 | ``` |