| 1 | # Server Smoke Test |
| 2 | |
| 3 | Start a fresh headless `vllm` engine: |
| 4 | |
| 5 | ```bash |
| 6 | source ../vllm/.venv/bin/activate |
| 7 | HF_HUB_OFFLINE=1 \ |
| 8 | VLLM_LOGGING_LEVEL=DEBUG \ |
| 9 | VLLM_CPU_KVCACHE_SPACE=2 \ |
| 10 | VLLM_HOST_IP=127.0.0.1 \ |
| 11 | VLLM_LOOPBACK_IP=127.0.0.1 \ |
| 12 | python3 -m vllm.entrypoints.cli.main serve Qwen/Qwen3-0.6B \ |
| 13 | --headless \ |
| 14 | --data-parallel-address 127.0.0.1 \ |
| 15 | --data-parallel-rpc-port 62100 \ |
| 16 | --data-parallel-size-local 1 \ |
| 17 | --max-model-len 512 \ |
| 18 | --dtype float16 |
| 19 | ``` |
| 20 | |
| 21 | Run the Rust server smoke test: |
| 22 | |
| 23 | ```bash |
| 24 | cargo run -p vllm-server --example external_engine_openai_qwen -- \ |
| 25 | --handshake-address tcp://127.0.0.1:62100 |
| 26 | ``` |
| 27 | |
| 28 | The example starts the Rust OpenAI-compatible server on an ephemeral local port, |
| 29 | connects to it via the `async-openai` Rust client, lists models, and then checks |
| 30 | that a streamed chat completion yields the assistant role chunk, final-answer |
| 31 | content chunks, and a terminal finish chunk. This example intentionally uses |
| 32 | `async-openai`'s standard typed `create_stream` API instead of BYOT, so it does |
| 33 | not inspect the nonstandard `reasoning_content` field even though the Rust |
| 34 | server may emit it for reasoning-capable models such as Qwen3. For reasoning |
| 35 | behavior itself, use the `vllm-chat` smoke test or the `vllm-server` |
| 36 | route tests. |
| 37 | |
| 38 | IMPORTANT: Restart `vllm` each time you run the smoke test. The current headless |
| 39 | engine cannot safely handle frontend reconnects after the client shuts down. |