[Tools] vLLM Recipes conversion : support different data types variants and strategies (#51878)
Signed-off-by: louie-tsai <louie.tsai@intel.com>
Louie Tsai committed
Aug 11, 2026 at 18:42 UTC
9b3382685e85d6b7e6ab4adbe91f95a9924622fa
2 files changed
+167
-25
tools/recipes/README.md
+57
-17
@@ -9,12 +9,31 @@ Converts a hardware-specific vLLM Recipes JSON rendering into:
9
- `config.yml` — native configuration for `vllm serve --config`
10
- `env.sh` — environment variables required by the recipe
11
12
+The converter uses the Recipes JSON API as the source of truth. It does not reimplement model-variant, hardware, or strategy compatibility rules.
13
+
14
## Install Dependency
15
16
```bash
17
pip install pyyaml
18
```
19
20
+## Recipe Selection Model
21
+
22
+Discovery follows the JSON links published by vLLM Recipes:
23
+
24
+```text
25
+/models.json
26
+ -> /{hf_id}.json
27
+ -> recommended_command.by_hardware[{hardware}]
28
+ -> /{hf_id}/hw/{hardware}.json
29
+ -> recommended strategy
30
+ -> alternatives[{strategy}]
31
+```
32
+
33
+The per-hardware JSON is the recommended deployment rendering for that model and hardware. Alternative strategy renderings are referenced through the JSON's `alternatives` map.
34
+
35
+The converter follows those exact links instead of constructing strategy paths locally. This also handles the Recipes API's legacy default-hardware strategy path automatically.
36
+
37
## Interactive Recipe Discovery
38
39
If you do not know the recipe JSON URL, run the script without an input:
@@ -25,33 +44,39 @@ python3 tools/recipes/recipe_json_to_vllm_config.py
44
45
The script will:
46
28
-1. Search the Recipes model index.
47
+1. Search the Recipes model index, including promoted variant model IDs.
48
2. Ask which model to use.
49
3. Show the hardware configurations available for that model.
31
-4. Resolve the correct hardware-specific JSON endpoint.
32
-5. Generate `config.yml` and `env.sh`.
50
+4. Show the recommended strategy and generated alternative strategies.
51
+5. Resolve the exact Recipes JSON endpoint.
52
+6. Generate `config.yml` and `env.sh`.
53
54
Example interaction:
55
56
```text
57
No recipe JSON supplied; starting Recipes API discovery.
38
-Model search (for example: llama 3.1): llama 3.1
58
+Model search (for example: llama 3.1): llama 3.1 fp8
59
60
Matching models:
41
- [1] meta-llama/Llama-3.1-8B-Instruct — Llama-3.1-8B-Instruct [Meta]
61
+ [1] nvidia/Llama-3.1-8B-Instruct-FP8 — Llama-3.1-8B-Instruct [Meta]
62
Select model: 1
63
64
Available hardware:
45
- [1] h100
46
- [2] h200
47
- [3] trillium
48
- [4] xeon6
49
-Select hardware: 4
65
+ [1] arc_pro_b70
66
+ [2] b200
67
+ [3] h100
68
+Select hardware: 1
69
+
70
+Available strategies:
71
+ [1] single_node_tp (recommended)
72
+ [2] multi_node_tp
73
+Select strategy: 1
74
75
Resolved recipe:
52
- Model: meta-llama/Llama-3.1-8B-Instruct
53
- Hardware: xeon6
54
- JSON: https://recipes.vllm.ai/meta-llama/Llama-3.1-8B-Instruct/hw/xeon6.json
76
+ Model: nvidia/Llama-3.1-8B-Instruct-FP8
77
+ Hardware: arc_pro_b70
78
+ Strategy: single_node_tp
79
+ JSON: https://recipes.vllm.ai/nvidia/Llama-3.1-8B-Instruct-FP8/hw/arc_pro_b70.json
80
```
81
82
## Non-Interactive Recipe Discovery
@@ -64,7 +89,18 @@ python3 tools/recipes/recipe_json_to_vllm_config.py \
89
--hardware xeon6
90
```
91
67
-The script verifies that the requested hardware rendering exists before using it.
92
+When `--strategy` is omitted with both `--model` and `--hardware`, the converter uses the Recipes-recommended strategy from the per-hardware JSON.
93
+
94
+To request a generated alternative strategy explicitly:
95
+
96
+```bash
97
+python3 tools/recipes/recipe_json_to_vllm_config.py \
98
+ --model nvidia/Llama-3.1-8B-Instruct-FP8 \
99
+ --hardware arc_pro_b70 \
100
+ --strategy multi_node_tp
101
+```
102
+
103
+The converter does not synthesize a URL such as `.../strategies/multi_node_tp.json`. It reads the selected hardware JSON and follows the exact `alternatives["multi_node_tp"]` link published by the Recipes API.
104
105
## Direct Recipe JSON Input
106
@@ -81,6 +117,8 @@ A local JSON file is also supported:
117
python3 tools/recipes/recipe_json_to_vllm_config.py recipe.json
118
```
119
120
+When a positional JSON source is supplied, do not combine it with `--model`, `--hardware`, or `--strategy`.
121
+
122
## Start vLLM
123
124
After generating the files:
@@ -109,8 +147,10 @@ python3 tools/recipes/recipe_json_to_vllm_config.py \
147
--env-out llama31-xeon6-env.sh
148
```
149
112
-## Scope
150
+## Strategy and Deployment Scope
151
+
152
+Strategy discovery and config conversion are separate concerns.
153
114
-The converter currently targets recipes that resolve to a single `vllm serve` process.
154
+The converter can resolve any generated strategy JSON exposed by the Recipes API. It currently emits one `config.yml`, so the selected rendering must contain a single `vllm serve` `argv`.
155
116
-Multi-node, disaggregated prefill/decode, or other multi-process deployment recipes require multiple commands and cannot be represented by a single `config.yml`.
156
+Single-process renderings such as `single_node_tp` can be converted directly. Multi-node, disaggregated prefill/decode, or other multi-process renderings expose fields such as `head_argv`, `worker_argv`, `prefill`, or `decode`; the converter intentionally exits instead of generating an incomplete single-process config.
tools/recipes/recipe_json_to_vllm_config.py
+110
-8
@@ -16,14 +16,25 @@ Examples:
16
python3 recipe_json_to_vllm_config.py \
17
https://recipes.vllm.ai/meta-llama/Llama-3.1-8B-Instruct/hw/xeon6.json
18
19
- # Non-interactive discovery
19
+ # Non-interactive discovery (recommended strategy)
20
python3 recipe_json_to_vllm_config.py \
21
--model meta-llama/Llama-3.1-8B-Instruct --hardware xeon6
22
23
+ # Promoted variant + explicit strategy
24
+ python3 recipe_json_to_vllm_config.py \
25
+ --model nvidia/Llama-3.1-8B-Instruct-FP8 \
26
+ --hardware arc_pro_b70 \
27
+ --strategy single_node_tp
28
+
29
Then:
30
source env.sh
31
vllm serve --config config.yml
32
33
+Variants with a distinct Hugging Face model ID are promoted by the Recipes API
34
+and selected through --model; this converter does not need a separate --variant
35
+argument. Strategy selection follows the exact `alternatives` JSON links emitted
36
+by the Recipes API instead of synthesizing strategy URLs locally.
37
+
38
The converter intentionally targets a single `vllm serve` process. If the
39
recipe rendering is multi-node, PD-disaggregated, or another multi-process
40
deployment, it exits instead of silently generating an incomplete config.
@@ -71,13 +82,25 @@ def parse_args() -> argparse.Namespace:
82
p.add_argument(
83
"--model",
84
help=(
74
- "Model hf_id or search text for recipe discovery. "
75
- "Use with --hardware for non-interactive operation."
85
+ "Model hf_id or search text for recipe discovery. Promoted recipe "
86
+ "variants are selected by their own hf_id. Use with --hardware for "
87
+ "non-interactive operation."
88
),
89
)
90
p.add_argument(
91
"--hardware",
80
- help="Hardware ID for recipe discovery, for example xeon6 or h100.",
92
+ help=(
93
+ "Hardware ID for recipe discovery, for example xeon6, b200, or arc_pro_b70."
94
+ ),
95
+ )
96
+ p.add_argument(
97
+ "--strategy",
98
+ help=(
99
+ "Serving strategy for recipe discovery, for example single_node_tp. "
100
+ "When omitted with --model and --hardware, use the Recipes-recommended "
101
+ "strategy. Interactive discovery offers the recommended strategy and "
102
+ "all generated alternatives."
103
+ ),
104
)
105
p.add_argument(
106
"--api-base",
@@ -238,10 +261,74 @@ def select_hardware(
261
return selected, by_hardware[selected]
262
263
264
+def strategy_sources(
265
+ api_base: str, hardware_json_url: str, recipe: dict[str, Any]
266
+) -> tuple[str, dict[str, str]]:
267
+ recommended = recipe.get("strategy")
268
+ if not isinstance(recommended, str) or not recommended:
269
+ raise ValueError(
270
+ "Hardware recipe JSON does not contain a usable `strategy` field."
271
+ )
272
+
273
+ sources = {recommended: hardware_json_url}
274
+ raw_alternatives = recipe.get("alternatives") or {}
275
+ if not isinstance(raw_alternatives, dict):
276
+ raise ValueError(
277
+ "Hardware recipe JSON `alternatives` must be an object when present."
278
+ )
279
+
280
+ for strategy, path in raw_alternatives.items():
281
+ if isinstance(strategy, str) and strategy and isinstance(path, str) and path:
282
+ sources[strategy] = api_url(api_base, path)
283
+
284
+ return recommended, sources
285
+
286
+
287
+def select_strategy(
288
+ api_base: str,
289
+ hardware_json_url: str,
290
+ recipe: dict[str, Any],
291
+ requested: str | None,
292
+ interactive: bool,
293
+) -> tuple[str, str]:
294
+ recommended, sources = strategy_sources(api_base, hardware_json_url, recipe)
295
+ strategies = [recommended, *sorted(s for s in sources if s != recommended)]
296
+
297
+ if requested:
298
+ selected = next(
299
+ (
300
+ strategy
301
+ for strategy in strategies
302
+ if strategy.lower() == requested.lower()
303
+ ),
304
+ None,
305
+ )
306
+ if selected is None:
307
+ raise ValueError(
308
+ f"Strategy {requested!r} is not available for this model/hardware. "
309
+ f"Available: {', '.join(strategies)}"
310
+ )
311
+ return selected, sources[selected]
312
+
313
+ if not interactive:
314
+ return recommended, hardware_json_url
315
+
316
+ print("\nAvailable strategies:")
317
+
318
+ def strategy_label(strategy: str) -> str:
319
+ if strategy == recommended:
320
+ return f"{strategy} (recommended)"
321
+ return strategy
322
+
323
+ selected = choose_from_menu(strategies, strategy_label, "Select strategy: ")
324
+ return selected, sources[selected]
325
+
326
+
327
def discover_recipe_source(
328
api_base: str,
329
requested_model: str | None,
330
requested_hardware: str | None,
331
+ requested_strategy: str | None,
332
) -> str:
333
print("No recipe JSON supplied; starting Recipes API discovery.")
334
@@ -287,11 +374,24 @@ def discover_recipe_source(
374
)
375
376
hardware, path = select_hardware(by_hardware, requested_hardware)
290
- resolved = api_url(api_base, path)
377
+ hardware_json_url = api_url(api_base, path)
378
+ hardware_data = load_json(hardware_json_url)
379
+ if not isinstance(hardware_data, dict):
380
+ raise ValueError(f"{hardware_json_url} did not return a JSON object.")
381
+
382
+ interactive = requested_model is None or requested_hardware is None
383
+ strategy, resolved = select_strategy(
384
+ api_base,
385
+ hardware_json_url,
386
+ hardware_data,
387
+ requested_strategy,
388
+ interactive,
389
+ )
390
391
print("\nResolved recipe:")
392
print(f" Model: {model.get('hf_id')}")
393
print(f" Hardware: {hardware}")
394
+ print(f" Strategy: {strategy}")
395
print(f" JSON: {resolved}")
396
print()
397
@@ -527,11 +627,13 @@ def main() -> int:
627
try:
628
source = args.source
629
if source is None:
530
- source = discover_recipe_source(args.api_base, args.model, args.hardware)
531
- elif args.model or args.hardware:
630
+ source = discover_recipe_source(
631
+ args.api_base, args.model, args.hardware, args.strategy
632
+ )
633
+ elif args.model or args.hardware or args.strategy:
634
raise ValueError(
635
"Do not combine a positional recipe JSON source with "
534
- "--model/--hardware discovery options."
636
+ "--model/--hardware/--strategy discovery options."
637
)
638
639
recipe = load_json(source)