A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Docs] Have a try to improve frameworks/streamlit.md (#24841)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
Michael Yao committed
Sep 15, 2025 at 12:50 UTC
78818dd1b0b4b8ae5e90a8259030684f23f589f1
1 file changed
+18
-20
docs/deployment/frameworks/streamlit.md
+18
-20
@@ -6,35 +6,33 @@ It can be quickly integrated with vLLM as a backend API server, enabling powerfu
6
7
## Prerequisites
8
9
-- Setup vLLM environment
10
-
11
-## Deploy
12
-
13
-- Start the vLLM server with the supported chat completion model, e.g.
9
+Set up the vLLM environment by installing all required packages:
10
11
```bash
16
-vllm serve qwen/Qwen1.5-0.5B-Chat
12
+pip install vllm streamlit openai
13
```
14
19
-- Install streamlit and openai:
15
+## Deploy
16
21
-```bash
22
-pip install streamlit openai
23
-```
17
+1. Start the vLLM server with a supported chat completion model, e.g.
18
25
-- Use the script: <gh-file:examples/online_serving/streamlit_openai_chatbot_webserver.py>
19
+ ```bash
20
+ vllm serve Qwen/Qwen1.5-0.5B-Chat
21
+ ```
22
27
-- Start the streamlit web UI and start to chat:
23
+1. Use the script: <gh-file:examples/online_serving/streamlit_openai_chatbot_webserver.py>
24
29
-```bash
30
-streamlit run streamlit_openai_chatbot_webserver.py
25
+1. Start the streamlit web UI and start to chat:
26
32
-# or specify the VLLM_API_BASE or VLLM_API_KEY
33
-VLLM_API_BASE="http://vllm-server-host:vllm-server-port/v1" \
27
+ ```bash
28
streamlit run streamlit_openai_chatbot_webserver.py
29
36
-# start with debug mode to view more details
37
-streamlit run streamlit_openai_chatbot_webserver.py --logger.level=debug
38
-```
30
+ # or specify the VLLM_API_BASE or VLLM_API_KEY
31
+ VLLM_API_BASE="http://vllm-server-host:vllm-server-port/v1" \
32
+ streamlit run streamlit_openai_chatbot_webserver.py
33
+
34
+ # start with debug mode to view more details
35
+ streamlit run streamlit_openai_chatbot_webserver.py --logger.level=debug
36
+ ```
37
40
-
38
+ 