A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 04:45 UTC
[Doc] Update docs for New Model Implementation (#20115)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Cyrus Leung committed
Jun 26, 2025 at 15:47 UTC
1d7c29f5fecab930fbb28bf59f1bc4510abe335b
2 files changed
+19
-12
docs/.nav.yml
+6
-1
@@ -48,7 +48,12 @@ nav:
48
- General:
49
- glob: contributing/*
50
flatten_single_child_sections: true
51
- - Model Implementation: contributing/model
51
+ - Model Implementation:
52
+ - contributing/model/README.md
53
+ - contributing/model/basic.md
54
+ - contributing/model/registration.md
55
+ - contributing/model/tests.md
56
+ - contributing/model/multimodal.md
57
- Design Documents:
58
- V0: design
59
- V1: design/v1
docs/contributing/model/README.md
+13
-11
@@ -1,21 +1,23 @@
1
---
2
-title: Adding a New Model
2
+title: Summary
3
---
4
[](){ #new-model }
5
6
-This section provides more information on how to integrate a [PyTorch](https://pytorch.org/) model into vLLM.
6
+!!! important
7
+ Many decoder language models can now be automatically loaded using the [Transformers backend][transformers-backend] without having to implement them in vLLM. See if `vllm serve <model>` works first!
8
8
-Contents:
9
+vLLM models are specialized [PyTorch](https://pytorch.org/) models that take advantage of various [features][compatibility-matrix] to optimize their performance.
10
10
-- [Basic](basic.md)
11
-- [Registration](registration.md)
12
-- [Tests](tests.md)
13
-- [Multimodal](multimodal.md)
11
+The complexity of integrating a model into vLLM depends heavily on the model's architecture.
12
+The process is considerably straightforward if the model shares a similar architecture with an existing model in vLLM.
13
+However, this can be more complex for models that include new operators (e.g., a new attention mechanism).
14
15
-!!! note
16
- The complexity of adding a new model depends heavily on the model's architecture.
17
- The process is considerably straightforward if the model shares a similar architecture with an existing model in vLLM.
18
- However, for models that include new operators (e.g., a new attention mechanism), the process can be a bit more complex.
15
+Read through these pages for a step-by-step guide:
16
+
17
+- [Implementing a Basic Model](basic.md)
18
+- [Registering a Model to vLLM](registration.md)
19
+- [Writing Unit Tests](tests.md)
20
+- [Multi-Modal Support](multimodal.md)
21
22
!!! tip
23
If you are encountering issues while integrating your model into vLLM, feel free to open a [GitHub issue](https://github.com/vllm-project/vllm/issues)