| 1 | # Server Arguments |
| 2 | |
| 3 | The `vllm serve` command is used to launch the OpenAI-compatible server. |
| 4 | |
| 5 | ## CLI Arguments |
| 6 | |
| 7 | The `vllm serve` command is used to launch the OpenAI-compatible server. |
| 8 | To see the available options, take a look at the [CLI Reference](../cli/README.md)! |
| 9 | |
| 10 | ## Configuration file |
| 11 | |
| 12 | You can load CLI arguments via a [YAML](https://yaml.org/) config file. |
| 13 | The argument names must be the long form of those outlined [above](serve_args.md). |
| 14 | |
| 15 | For example: |
| 16 | |
| 17 | ```yaml |
| 18 | # config.yaml |
| 19 | |
| 20 | model: meta-llama/Llama-3.1-8B-Instruct |
| 21 | host: "127.0.0.1" |
| 22 | port: 6379 |
| 23 | uvicorn-log-level: "info" |
| 24 | ``` |
| 25 | |
| 26 | To use the above config file: |
| 27 | |
| 28 | ```bash |
| 29 | vllm serve --config config.yaml |
| 30 | ``` |
| 31 | |
| 32 | ### Generate a configuration from vLLM Recipes |
| 33 | |
| 34 | [vLLM Recipes](https://recipes.vllm.ai/) can be converted into `config.yaml` |
| 35 | and `env.sh`. See the |
| 36 | [Recipes conversion tool README](../../tools/recipes/README.md) for usage. |
| 37 | |
| 38 | Source the generated environment before starting vLLM: |
| 39 | |
| 40 | ```bash |
| 41 | source env.sh |
| 42 | vllm serve --config config.yaml |
| 43 | ``` |
| 44 | |
| 45 | !!! note |
| 46 | In case an argument is supplied simultaneously using command line and the config file, the value from the command line will take precedence. |
| 47 | The order of priorities is `command line > config file values > defaults`. |
| 48 | e.g. `vllm serve SOME_MODEL --config config.yaml`, SOME_MODEL takes precedence over `model` in config file. |