[CPU] Include vLLM Recipes tooling in release image to deploy models using vLLM Recipes (#58796)
Signed-off-by: louie-tsai <louie.tsai@intel.com>
Louie Tsai committed
Sep 28, 2026 at 01:46 UTC
ccfd1cea75eff8956c5d08b15a7eaec8891642e9
4 files changed
+64
-11
docker/Dockerfile.cpu
+1
@@ -482,5 +482,6 @@ LABEL ai.vllm.build.python-version="${PYTHON_VERSION:-3.12}"
482
# stage above adds examples/ for testing only, so without this the published
483
# vllm-openai-cpu image would not ship examples/*.jinja.
484
COPY examples examples
485
+COPY tools/recipes tools/recipes
486
487
ENTRYPOINT ["vllm", "serve"]
docs/getting_started/installation/cpu.x86.inc.md
+34
@@ -183,6 +183,40 @@ docker run \
183
vllm/vllm-openai-cpu:latest-x86_64 <args...>
184
```
185
186
+### Serve with vLLM Recipes
187
+
188
+The CPU image includes the [vLLM Recipes](https://recipes.vllm.ai/) deployment
189
+tool. Recipe-assisted serving queries the site at runtime, so the image can use
190
+the latest published recipe for the selected model and hardware.
191
+
192
+To find the value for `--hardware`, open vLLM Recipes, select the model, and
193
+choose the target in the **Hardware** picker. The selected page URL also
194
+contains the hardware key; for example, `?hardware=xeon6` maps to
195
+`--hardware xeon6`.
196
+
197
+Keep the normal image entrypoint for regular serving. For recipe-assisted
198
+serving, override it with `serve_with_recipe.sh`:
199
+
200
+```bash
201
+docker run --rm \
202
+ --shm-size=4g \
203
+ -v ~/.cache/huggingface:/root/.cache/huggingface \
204
+ -p 8000:8000 \
205
+ --env "HF_TOKEN=<secret>" \
206
+ --entrypoint tools/recipes/serve_with_recipe.sh \
207
+ vllm/vllm-openai-cpu:latest-x86_64 \
208
+ --model meta-llama/Llama-3.1-8B-Instruct \
209
+ --hardware xeon6
210
+```
211
+
212
+For Xeon 6, the script enables hardware detection automatically, generates
213
+`config.yml` and `env.sh`, and starts `vllm serve`. The image keeps
214
+`/vllm-workspace` as its working directory so relative recipe assets under
215
+`examples/` continue to resolve.
216
+
217
+See the [Recipes tool documentation](../../../tools/recipes/README.md) for
218
+interactive discovery and additional options.
219
+
220
--8<-- [end:pre-built-images]
221
--8<-- [start:build-image-from-source]
222
docs/models/hardware_supported_models/cpu.md
+9
-9
@@ -12,17 +12,17 @@
12
13
## Deploy from a vLLM Recipe
14
15
-The **Recipe** column below links to validated Xeon 6 configurations when
16
-available. Use the Recipe conversion tool to generate `config.yaml` and
17
-`env.sh`; see the
18
-[Recipes conversion tool README](../../../tools/recipes/README.md) for usage.
15
+The **Recipe** column below links to published Xeon 6 configurations when
16
+available. Open a recipe to review the latest deployment configuration for the
17
+model and hardware.
18
20
-Load the generated environment before starting vLLM:
19
+For one-step deployment with the pre-built CPU image, see
20
+[Serve with vLLM Recipes](../../getting_started/installation/cpu.md#serve-with-vllm-recipes).
21
+The image includes the Recipes tool, which can retrieve the latest published
22
+recipe, apply Xeon hardware detection, and start `vllm serve`.
23
22
-```bash
23
-source env.sh
24
-vllm serve --config config.yaml
25
-```
24
+For converter usage and interactive model/hardware discovery, see the
25
+[Recipes tool documentation](../../../tools/recipes/README.md).
26
27
## Recommended Models
28
tools/recipes/README.md
+20
-2
@@ -23,8 +23,9 @@ refine the initial configuration. Sweep tuning is an optional validation step.
23
24
## Getting Started
25
26
-Use `serve_with_recipe.sh` to generate the recipe configuration and start vLLM
27
-in one step:
26
+Use `serve_with_recipe.sh` to fetch the selected deployment from
27
+[vLLM Recipes](https://recipes.vllm.ai/), generate the configuration, and start
28
+vLLM in one step:
29
30
```bash
31
tools/recipes/serve_with_recipe.sh \
@@ -32,6 +33,23 @@ tools/recipes/serve_with_recipe.sh \
33
--hardware xeon6
34
```
35
36
+The converter queries `recipes.vllm.ai` at runtime, so it can use the latest
37
+published recipe for the requested model and hardware without rebuilding the
38
+vLLM image.
39
+
40
+### Find the hardware name
41
+
42
+Open [vLLM Recipes](https://recipes.vllm.ai/), select the model, and choose the
43
+target in the **Hardware** picker. The selected page URL contains the hardware
44
+key. For example, `?hardware=xeon6` maps to `--hardware xeon6`.
45
+
46
+You can also run the converter without arguments for interactive model,
47
+hardware, and strategy discovery:
48
+
49
+```bash
50
+python3 tools/recipes/recipe_json_to_vllm_config.py
51
+```
52
+
53
For `xeon6`, the script enables hardware detection automatically. Generated
54
`config.yml` and `env.sh` files remain in the directory where the script was
55
invoked.