| 1 | # Chatbox |
| 2 | |
| 3 | [Chatbox](https://github.com/chatboxai/chatbox) is a desktop client for LLMs, available on Windows, Mac, Linux. |
| 4 | |
| 5 | It allows you to deploy a large language model (LLM) server with vLLM as the backend, which exposes OpenAI-compatible endpoints. |
| 6 | |
| 7 | ## Prerequisites |
| 8 | |
| 9 | Set up the vLLM environment: |
| 10 | |
| 11 | ```bash |
| 12 | pip install vllm |
| 13 | ``` |
| 14 | |
| 15 | ## Deploy |
| 16 | |
| 17 | 1. Start the vLLM server with the supported chat completion model, e.g. |
| 18 | |
| 19 | ```bash |
| 20 | vllm serve qwen/Qwen1.5-0.5B-Chat |
| 21 | ``` |
| 22 | |
| 23 | 1. Download and install [Chatbox desktop](https://chatboxai.app/en#download). |
| 24 | |
| 25 | 1. On the bottom left of settings, Add Custom Provider |
| 26 | - API Mode: `OpenAI API Compatible` |
| 27 | - Name: vllm |
| 28 | - API Host: `http://{vllm server host}:{vllm server port}/v1` |
| 29 | - API Path: `/chat/completions` |
| 30 | - Model: `qwen/Qwen1.5-0.5B-Chat` |
| 31 | |
| 32 |  |
| 33 | |
| 34 | 1. Go to `Just chat`, and start to chat: |
| 35 | |
| 36 |  |