A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[docs] standardize Hugging Face env var to `HF_TOKEN` (deprecates `HUGGING_FACE_HUB_TOKEN`) (#27020)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
Kay Yan committed
Oct 16, 2025 at 22:31 UTC
02d709a6f1e442ab8f1129170eac338a6b3a4b1f
3 files changed
+8
-8
docs/deployment/docker.md
+3
-3
@@ -10,7 +10,7 @@ The image can be used to run OpenAI compatible server and is available on Docker
10
```bash
11
docker run --runtime nvidia --gpus all \
12
-v ~/.cache/huggingface:/root/.cache/huggingface \
13
- --env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
13
+ --env "HF_TOKEN=$HF_TOKEN" \
14
-p 8000:8000 \
15
--ipc=host \
16
vllm/vllm-openai:latest \
@@ -22,7 +22,7 @@ This image can also be used with other container engines such as [Podman](https:
22
```bash
23
podman run --device nvidia.com/gpu=all \
24
-v ~/.cache/huggingface:/root/.cache/huggingface \
25
- --env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
25
+ --env "HF_TOKEN=$HF_TOKEN" \
26
-p 8000:8000 \
27
--ipc=host \
28
docker.io/vllm/vllm-openai:latest \
@@ -128,7 +128,7 @@ To run vLLM with the custom-built Docker image:
128
docker run --runtime nvidia --gpus all \
129
-v ~/.cache/huggingface:/root/.cache/huggingface \
130
-p 8000:8000 \
131
- --env "HUGGING_FACE_HUB_TOKEN=<secret>" \
131
+ --env "HF_TOKEN=<secret>" \
132
vllm/vllm-openai <args...>
133
```
134
docs/deployment/frameworks/lws.md
+2
-2
@@ -35,7 +35,7 @@ Deploy the following yaml file `lws.yaml`
35
- name: vllm-leader
36
image: docker.io/vllm/vllm-openai:latest
37
env:
38
- - name: HUGGING_FACE_HUB_TOKEN
38
+ - name: HF_TOKEN
39
value: <your-hf-token>
40
command:
41
- sh
@@ -83,7 +83,7 @@ Deploy the following yaml file `lws.yaml`
83
ephemeral-storage: 800Gi
84
cpu: 125
85
env:
86
- - name: HUGGING_FACE_HUB_TOKEN
86
+ - name: HF_TOKEN
87
value: <your-hf-token>
88
volumeMounts:
89
- mountPath: /dev/shm
docs/deployment/k8s.md
+3
-3
@@ -82,7 +82,7 @@ Next, start the vLLM server as a Kubernetes Deployment and Service:
82
"vllm serve meta-llama/Llama-3.2-1B-Instruct"
83
]
84
env:
85
- - name: HUGGING_FACE_HUB_TOKEN
85
+ - name: HF_TOKEN
86
valueFrom:
87
secretKeyRef:
88
name: hf-token-secret
@@ -209,7 +209,7 @@ INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
209
"vllm serve mistralai/Mistral-7B-Instruct-v0.3 --trust-remote-code --enable-chunked-prefill --max_num_batched_tokens 1024"
210
]
211
env:
212
- - name: HUGGING_FACE_HUB_TOKEN
212
+ - name: HF_TOKEN
213
valueFrom:
214
secretKeyRef:
215
name: hf-token-secret
@@ -298,7 +298,7 @@ INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
298
"vllm serve mistralai/Mistral-7B-v0.3 --port 8000 --trust-remote-code --enable-chunked-prefill --max_num_batched_tokens 1024"
299
]
300
env:
301
- - name: HUGGING_FACE_HUB_TOKEN
301
+ - name: HF_TOKEN
302
valueFrom:
303
secretKeyRef:
304
name: hf-token-secret