| 1 | # Model Resolution |
| 2 | |
| 3 | vLLM loads HuggingFace-compatible models by inspecting the `architectures` field in `config.json` of the model repository |
| 4 | and finding the corresponding implementation that is registered to vLLM. |
| 5 | Nevertheless, our model resolution may fail for the following reasons: |
| 6 | |
| 7 | - The `config.json` of the model repository lacks the `architectures` field. |
| 8 | - Unofficial repositories refer to a model using alternative names which are not recorded in vLLM. |
| 9 | - The same architecture name is used for multiple models, creating ambiguity as to which model should be loaded. |
| 10 | |
| 11 | To fix this, explicitly specify the model architecture by passing `config.json` overrides to the `hf_overrides` option. |
| 12 | For example: |
| 13 | |
| 14 | ```python |
| 15 | from vllm import LLM |
| 16 | |
| 17 | llm = LLM( |
| 18 | model="cerebras/Cerebras-GPT-1.3B", |
| 19 | hf_overrides={"architectures": ["GPT2LMHeadModel"]}, # GPT-2 |
| 20 | ) |
| 21 | ``` |
| 22 | |
| 23 | Our [list of supported models](../models/supported_models.md) shows the model architectures that are recognized by vLLM. |