[Misc] Move the legacy api_server.py to the examples directory. (#46783)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
wang.yuqi committed
Jun 26, 2026 at 15:43 UTC
5e3dad04b10df208513d4941da9e72e6d9e77048
5 files changed
+6
-6
docs/design/arch_overview.md
+1
-1
@@ -178,7 +178,7 @@ incoming requests. The `AsyncLLMEngine` is designed for online serving, where it
178
can handle multiple concurrent requests and stream outputs to clients.
179
180
The OpenAI-compatible API server uses the `AsyncLLMEngine`. There is also a demo
181
-API server that serves as a simpler example in [vllm/entrypoints/api_server.py](../../vllm/entrypoints/api_server.py).
181
+API server that serves as a simpler example in [examples/applications/api_server/server.py](../../examples/applications/api_server/server.py).
182
183
The code for `AsyncLLMEngine` can be found in [vllm/engine/async_llm_engine.py](../../vllm/engine/async_llm_engine.py).
184
docs/examples/README.md
+1
-1
@@ -9,7 +9,7 @@ vLLM's examples are organized into the following categories:
9
- **[`features/`](../../examples/features)** – Demonstrations of individual vLLM features: automatic prefix caching, speculative decoding, LoRA, structured outputs, prompt embedding, pause/resume, batch invariance, KV events, data parallelism, and more.
10
- **[`reasoning/`](../../examples/reasoning)** – Examples for reasoning with vLLM.
11
- **[`tool_calling/`](../../examples/tool_calling)** – Examples for function/tool calling with vLLM.
12
-- **[`applications/`](../../examples/applications)** – Application examples such as chatbots and RAG (Retrieval-Augmented Generation).
12
+- **[`applications/`](../../examples/applications)** – Application examples such as simpler api server, chatbots and RAG (Retrieval-Augmented Generation).
13
- **[`rl/`](../../examples/rl)** – Reinforcement learning examples.
14
- **[`deployment/`](../../examples/deployment)** – Examples for deploying vLLM in production.
15
- **[`ray_serving/`](../../examples/ray_serving)** – Scalable serving using Ray.
examples/applications/api_server/client.py
renamed
+2
-2
@@ -1,8 +1,8 @@
1
# SPDX-License-Identifier: Apache-2.0
2
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3
-"""Example Python client for `vllm.entrypoints.api_server`
3
+"""Example Python client for `examples/applications/api_server/server.py`
4
Start the demo server:
5
- python -m vllm.entrypoints.api_server --model <model_name>
5
+ python examples/applications/api_server/server.py --model <model_name>
6
7
NOTE: The API server is used only for demonstration and simple performance
8
benchmarks. It is not intended for production use.
examples/applications/api_server/server.py
renamed
+1
-1
@@ -31,7 +31,7 @@ from vllm.utils.argparse_utils import FlexibleArgumentParser
31
from vllm.utils.system_utils import set_ulimit
32
from vllm.version import __version__ as VLLM_VERSION
33
34
-logger = init_logger("vllm.entrypoints.api_server")
34
+logger = init_logger("api_server")
35
36
app = FastAPI()
37
engine = None
examples/applications/chatbot/gradio_webserver.py
+1
-1
@@ -2,7 +2,7 @@
2
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3
"""Example for starting a Gradio Webserver
4
Start vLLM API server:
5
- python -m vllm.entrypoints.api_server \
5
+ python examples/applications/api_server/server.py \
6
--model meta-llama/Llama-2-7b-chat-hf
7
8
Start Webserver: