A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 09:45 UTC
[Doc] Fix typo (#18355)
Elad Segal committed
May 19, 2025 at 19:05 UTC
84ab4feb7e994ee6c692957e6d80a528af072e49
1 file changed
+1
-1
docs/source/models/supported_models.md
+1
-1
@@ -54,7 +54,7 @@ For a model to be compatible with the Transformers backend for vLLM it must:
54
55
If the compatible model is:
56
57
-- on the Hugging Face Model Hub, simply set `trust_remote_code=True` for <project:#offline-inference> or `--trust-remode-code` for the <project:#openai-compatible-server>.
57
+- on the Hugging Face Model Hub, simply set `trust_remote_code=True` for <project:#offline-inference> or `--trust-remote-code` for the <project:#openai-compatible-server>.
58
- in a local directory, simply pass directory path to `model=<MODEL_DIR>` for <project:#offline-inference> or `vllm serve <MODEL_DIR>` for the <project:#openai-compatible-server>.
59
60
This means that, with the Transformers backend for vLLM, new models can be used before they are officially supported in Transformers or vLLM!