| 1 | # Registering a Model |
| 2 | |
| 3 | vLLM relies on a model registry to determine how to run each model. |
| 4 | A list of pre-registered architectures can be found [here](../../models/supported_models.md). |
| 5 | |
| 6 | If your model is not on this list, you must register it to vLLM. |
| 7 | This page provides detailed instructions on how to do so. |
| 8 | |
| 9 | ## Built-in models |
| 10 | |
| 11 | To add a model directly to the vLLM library, start by forking our [GitHub repository](https://github.com/vllm-project/vllm) and then [build it from source](../../getting_started/installation/gpu.md#build-wheel-from-source). |
| 12 | This gives you the ability to modify the codebase and test your model. |
| 13 | |
| 14 | After you have implemented your model (see [tutorial](basic.md)), put it into the [vllm/model_executor/models](../../../vllm/model_executor/models) directory. |
| 15 | Then, add your model class to `_VLLM_MODELS` in [vllm/model_executor/models/registry.py](../../../vllm/model_executor/models/registry.py) so that it is automatically registered upon importing vLLM. |
| 16 | Finally, update our [list of supported models](../../models/supported_models.md) to promote your model! |
| 17 | |
| 18 | !!! important |
| 19 | The list of models in each section should be maintained in alphabetical order. |
| 20 | |
| 21 | ## Out-of-tree models |
| 22 | |
| 23 | You can load an external model [using a plugin](../../design/plugin_system.md) without modifying the vLLM codebase. |
| 24 | |
| 25 | To register the model, use the following code: |
| 26 | |
| 27 | ```python |
| 28 | # The entrypoint of your plugin |
| 29 | def register(): |
| 30 | from vllm import ModelRegistry |
| 31 | from your_code import YourModelForCausalLM |
| 32 | |
| 33 | ModelRegistry.register_model("YourModelForCausalLM", YourModelForCausalLM) |
| 34 | ``` |
| 35 | |
| 36 | If your model imports modules that initialize CUDA, consider lazy-importing it to avoid errors like `RuntimeError: Cannot re-initialize CUDA in forked subprocess`: |
| 37 | |
| 38 | ```python |
| 39 | # The entrypoint of your plugin |
| 40 | def register(): |
| 41 | from vllm import ModelRegistry |
| 42 | |
| 43 | ModelRegistry.register_model( |
| 44 | "YourModelForCausalLM", |
| 45 | "your_code:YourModelForCausalLM", |
| 46 | ) |
| 47 | ``` |
| 48 | |
| 49 | !!! important |
| 50 | If your model is a multimodal model, ensure the model class implements the [SupportsMultiModal][vllm.model_executor.models.interfaces.SupportsMultiModal] interface. |
| 51 | Read more about that [here](multimodal.md). |