A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Doc] Fix issues in integrations/llamastack.md (#24428)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
Michael Yao committed
Sep 8, 2025 at 17:28 UTC
c2a8b08fcdc48579871cd568b0799037a1214ea1
1 file changed
+7
-7
docs/deployment/integrations/llamastack.md
+7
-7
@@ -1,6 +1,6 @@
1
# Llama Stack
2
3
-vLLM is also available via [Llama Stack](https://github.com/meta-llama/llama-stack) .
3
+vLLM is also available via [Llama Stack](https://github.com/llamastack/llama-stack).
4
5
To install Llama Stack, run
6
@@ -8,9 +8,9 @@ To install Llama Stack, run
8
pip install llama-stack -q
9
```
10
11
-## Inference using OpenAI Compatible API
11
+## Inference using OpenAI-Compatible API
12
13
-Then start Llama Stack server pointing to your vLLM server with the following configuration:
13
+Then start the Llama Stack server and configure it to point to your vLLM server with the following settings:
14
15
```yaml
16
inference:
@@ -20,15 +20,15 @@ inference:
20
url: http://127.0.0.1:8000
21
```
22
23
-Please refer to [this guide](https://llama-stack.readthedocs.io/en/latest/distributions/self_hosted_distro/remote-vllm.html) for more details on this remote vLLM provider.
23
+Please refer to [this guide](https://llama-stack.readthedocs.io/en/latest/providers/inference/remote_vllm.html) for more details on this remote vLLM provider.
24
25
-## Inference via Embedded vLLM
25
+## Inference using Embedded vLLM
26
27
-An [inline vLLM provider](https://github.com/meta-llama/llama-stack/tree/main/llama_stack/providers/inline/inference/vllm)
27
+An [inline provider](https://github.com/llamastack/llama-stack/tree/main/llama_stack/providers/inline/inference)
28
is also available. This is a sample of configuration using that method:
29
30
```yaml
31
-inference
31
+inference:
32
- provider_type: vllm
33
config:
34
model: Llama3.1-8B-Instruct