@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 7f2593b164
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 03:00 UTC

[Doc]: Update the doc of adding new models (#4236)

xiaoji committed Apr 22, 2024 at 00:57 UTC 7f2593b164c2ff115ba4fb9ce95fe63bdd824b85
1 file changed +1 -1
docs/source/models/adding_model.rst
+1 -1
@@ -95,7 +95,7 @@ This method should load the weights from the HuggingFace's checkpoint file and a
95 5. Register your model
96 ----------------------
97
98 -Finally, include your :code:`*ForCausalLM` class in `vllm/model_executor/models/__init__.py <https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/__init__.py>`_ and register it to the :code:`_MODEL_REGISTRY` in `vllm/model_executor/model_loader.py <https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/model_loader.py>`_.
98 +Finally, register your :code:`*ForCausalLM` class to the :code:`_MODELS` in `vllm/model_executor/models/__init__.py <https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/__init__.py>`_.
99
100 6. Out-of-Tree Model Integration
101 --------------------------------------------