| 1 | # LLM Smoke Test |
| 2 | |
| 3 | Start headless `vllm`: |
| 4 | |
| 5 | ```bash |
| 6 | source ../vllm/.venv/bin/activate |
| 7 | HF_HUB_OFFLINE=1 \ |
| 8 | VLLM_LOGGING_LEVEL=DEBUG \ |
| 9 | VLLM_CPU_KVCACHE_SPACE=2 \ |
| 10 | VLLM_HOST_IP=127.0.0.1 \ |
| 11 | VLLM_LOOPBACK_IP=127.0.0.1 \ |
| 12 | python3 -m vllm.entrypoints.cli.main serve Qwen/Qwen3-0.6B \ |
| 13 | --headless \ |
| 14 | --data-parallel-address 127.0.0.1 \ |
| 15 | --data-parallel-rpc-port 62100 \ |
| 16 | --data-parallel-size-local 1 \ |
| 17 | --max-model-len 512 \ |
| 18 | --dtype float16 |
| 19 | ``` |
| 20 | |
| 21 | Run the Rust smoke test through the `vllm-llm` generate interface: |
| 22 | |
| 23 | ```bash |
| 24 | cargo run -p vllm-llm --example external_engine_smoke -- \ |
| 25 | --handshake-address tcp://127.0.0.1:62100 \ |
| 26 | --host 127.0.0.1 |
| 27 | ``` |
| 28 | |
| 29 | IMPORTANT: You must restart `vllm` each time you run the smoke test, as the vLLM engine cannot manage frontend closures and subsequent reconnects. In other words, do not reuse existing `vllm` instances, if any. |