A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 11:00 UTC
[doc] update lora doc (#17936)
Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>
Reid committed
May 11, 2025 at 15:56 UTC
d1110f5b5a241340e7c1e3970c95fe6dd6a14769
1 file changed
+4
-2
docs/source/features/lora.md
+4
-2
@@ -66,7 +66,7 @@ The commit ID `0dfa347e8877a4d4ed19ee56c140fa518470028c` may change over time. P
66
67
The server entrypoint accepts all other LoRA configuration parameters (`max_loras`, `max_lora_rank`, `max_cpu_loras`,
68
etc.), which will apply to all forthcoming requests. Upon querying the `/models` endpoint, we should see our LoRA along
69
-with its base model:
69
+with its base model (if `jq` is not installed, you can follow [this guide](https://jqlang.org/download/) to install it.):
70
71
```bash
72
curl localhost:8000/v1/models | jq .
@@ -134,7 +134,7 @@ curl -X POST http://localhost:8000/v1/load_lora_adapter \
134
}'
135
```
136
137
-Upon a successful request, the API will respond with a 200 OK status code. If an error occurs, such as if the adapter
137
+Upon a successful request, the API will respond with a `200 OK` status code from `vllm serve`, and `curl` returns the response body: `Success: LoRA adapter 'sql_adapter' added successfully`. If an error occurs, such as if the adapter
138
cannot be found or loaded, an appropriate error message will be returned.
139
140
Unloading a LoRA Adapter:
@@ -142,6 +142,8 @@ Unloading a LoRA Adapter:
142
To unload a LoRA adapter that has been previously loaded, send a POST request to the `/v1/unload_lora_adapter` endpoint
143
with the name or ID of the adapter to be unloaded.
144
145
+Upon a successful request, the API responds with a `200 OK` status code from `vllm serve`, and `curl` returns the response body: `Success: LoRA adapter 'sql_adapter' removed successfully`.
146
+
147
Example request to unload a LoRA adapter:
148
149
```bash