@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 77072e93b3
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 11, 21:15 UTC

[docs] governance documents (#24801)

Signed-off-by: simon-mo <simon.mo@hey.com> Signed-off-by: Michael Goin <mgoin64@gmail.com> Signed-off-by: Simon Mo <simon.mo@hey.com> Co-authored-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: Russell Bryant <rbryant@redhat.com> Co-authored-by: Michael Goin <mgoin64@gmail.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Simon Mo committed Dec 8, 2025 at 03:06 UTC 77072e93b327fd391482f5facadf93a804ced0cc
4 files changed +352
docs/.nav.yml
+1
@@ -59,6 +59,7 @@ nav:
59 - CLI Reference: cli
60 - Community:
61 - community/*
62 + - Governance: governance
63 - Blog: https://blog.vllm.ai
64 - Forum: https://discuss.vllm.ai
65 - Slack: https://slack.vllm.ai
docs/governance/collaboration.md new
+43
@@ -0,0 +1,43 @@
1 +# Collaboration Policy
2 +
3 +This page outlines how vLLM collaborates with model providers, hardware vendors, and other stakeholders.
4 +
5 +## Adding New Major Features
6 +
7 +Anyone can contribute to vLLM. For major features, submit an RFC (request for comments) first. To submit an RFC, create an [issue](https://github.com/vllm-project/vllm/issues/new/choose) and select the `RFC` template.
8 +RFCs are similar to design docs that discuss the motivation, problem solved, alternatives considered, and proposed change.
9 +
10 +Once you submit the RFC, please post it in the #contributors channel in vLLM Slack, and loop in area owners and committers for feedback.
11 +For high-interest features, the committers nominate a person to help with the RFC process and PR review. This makes sure someone is guiding you through the process. It is reflected as the "assignee" field in the RFC issue.
12 +If the assignee and lead maintainers find the feature to be contentious, the maintainer team aims to make decisions quickly after learning the details from everyone. This involves assigning a committer as the DRI (Directly Responsible Individual) to make the decision and shepherd the code contribution process.
13 +
14 +For features that you intend to maintain, please feel free to add yourself in [`mergify.yml`](https://github.com/vllm-project/vllm/blob/main/.github/mergify.yml) to receive notifications and auto-assignment when the PRs touching the feature you are maintaining. Over time, the ownership will be evaluated and updated through the committers nomination and voting process.
15 +
16 +## Adding New Models
17 +
18 +If you use vLLM, we recommend you making the model work with vLLM by following the [model registration](../contributing/model/registration.md) process before you release it publicly.
19 +
20 +The vLLM team helps with new model architectures not supported by vLLM, especially models pushing architectural frontiers.
21 +Here's how the vLLM team works with model providers. The vLLM team includes all [committers](./committers.md) of the project. model providers can exclude certain members but shouldn't, as this may harm release timelines due to missing expertise. Contact [project leads](./process.md) if you want to collaborate.
22 +
23 +Once we establish the connection between the vLLM team and model provider:
24 +
25 +- The vLLM team learns the model architecture and relevant changes, then plans which area owners to involve and what features to include.
26 +- The vLLM team creates a private communication channel (currently a Slack channel in the vLLM workspace) and a private fork within the vllm-project organization. The model provider team can invite others to the channel and repo.
27 +- Third parties like compute providers, hosted inference providers, hardware vendors, and other organizations often work with both the model provider and vLLM on model releases. We establish direct communication (with permission) or three-way communication as needed.
28 +
29 +The vLLM team works with model providers on features, integrations, and release timelines. We work to meet release timelines, but engineering challenges like feature development, model accuracy alignment, and optimizations can cause delays.
30 +
31 +The vLLM maintainers will not publicly share details about model architecture, release timelines, or upcoming releases. We maintain model weights on secure servers with security measures (though we can work with security reviews and testing without certification). We delete pre-release weights or artifacts upon request.
32 +
33 +The vLLM team collaborates on marketing and promotional efforts for model releases. model providers can use vLLM's trademark and logo in publications and materials.
34 +
35 +## Adding New Hardware
36 +
37 +vLLM is designed as a platform for frontier model architectures and high-performance accelerators.
38 +For new hardware, follow the [hardware plugin](../design/plugin_system.md) system to add support.
39 +Use the platform plugin system to add hardware support.
40 +As hardware gains popularity, we help endorse it in our documentation and marketing materials.
41 +The vLLM GitHub organization can host hardware plugin repositories, especially for collaborative efforts among companies.
42 +
43 +We rarely add new hardware to vLLM directly. Instead, we make existing hardware platforms modular to keep the vLLM core hardware-agnostic.
docs/governance/committers.md new
+183
@@ -0,0 +1,183 @@
1 +# Committers
2 +
3 +This document lists the current committers of the vLLM project and the core areas they maintain.
4 +Committers have write access to the vLLM repository and are responsible for reviewing and merging PRs.
5 +You can also refer to the [CODEOWNERS](https://github.com/vllm-project/vllm/blob/main/.github/CODEOWNERS) file for concrete file-level ownership and reviewers. Both this documents and the CODEOWNERS file are living documents and they complement each other.
6 +
7 +## Active Committers
8 +
9 +We try to summarize each committer's role in vLLM in a few words. In general, vLLM committers cover a wide range of areas and help each other in the maintenance process.
10 +Please refer to the later section about Area Owners for exact component ownership details.
11 +Sorted alphabetically by GitHub handle:
12 +
13 +- [@22quinn](https://github.com/22quinn): RL API
14 +- [@aarnphm](https://github.com/aarnphm): Structured output
15 +- [@alexm-redhat](https://github.com/alexm-redhat): Performance
16 +- [@ApostaC](https://github.com/ApostaC): Connectors, offloading
17 +- [@benchislett](https://github.com/benchislett): Engine core and spec decode
18 +- [@bigPYJ1151](https://github.com/bigPYJ1151): Intel CPU/XPU integration
19 +- [@chaunceyjiang](https://github.com/chaunceyjiang): Tool use and reasoning parser
20 +- [@DarkLight1337](https://github.com/DarkLight1337): Multimodality, API server
21 +- [@esmeetu](https://github.com/esmeetu): developer marketing, community
22 +- [@gshtras](https://github.com/gshtras): AMD integration
23 +- [@heheda12345](https://github.com/heheda12345): Hybrid memory allocator
24 +- [@hmellor](https://github.com/hmellor): Hugging Face integration, documentation
25 +- [@houseroad](https://github.com/houseroad): Engine core and Llama models
26 +- [@Isotr0py](https://github.com/Isotr0py): Multimodality, new model support
27 +- [@jeejeelee](https://github.com/jeejeelee): LoRA, new model support
28 +- [@jikunshang](https://github.com/jikunshang): Intel CPU/XPU integration
29 +- [@khluu](https://github.com/khluu): CI infrastructure
30 +- [@KuntaiDu](https://github.com/KuntaiDu): KV Connector
31 +- [@LucasWilkinson](https://github.com/LucasWilkinson): Kernels and performance
32 +- [@luccafong](https://github.com/luccafong): Llama models, speculative decoding, distributed
33 +- [@markmc](https://github.com/markmc): Observability
34 +- [@mgoin](https://github.com/mgoin): Quantization and performance
35 +- [@NickLucche](https://github.com/NickLucche): KV connector
36 +- [@njhill](https://github.com/njhill): Distributed, API server, engine core
37 +- [@noooop](https://github.com/noooop): Pooling models
38 +- [@patrickvonplaten](https://github.com/patrickvonplaten): Mistral models, new model support
39 +- [@pavanimajety](https://github.com/pavanimajety): NVIDIA GPU integration
40 +- [@ProExpertProg](https://github.com/ProExpertProg): Compilation, startup UX
41 +- [@robertgshaw2-redhat](https://github.com/robertgshaw2-redhat): Core, distributed, disagg
42 +- [@ruisearch42](https://github.com/ruisearch42): Pipeline parallelism, Ray Support
43 +- [@russellb](https://github.com/russellb): Structured output, engine core, security
44 +- [@sighingnow](https://github.com/sighingnow): Qwen models, new model support
45 +- [@simon-mo](https://github.com/simon-mo): Project lead, API entrypoints, community
46 +- [@tdoublep](https://github.com/tdoublep): State space models
47 +- [@tjtanaa](https://github.com/tjtanaa): AMD GPU integration
48 +- [@tlrmchlsmth](https://github.com/tlrmchlsmth): Kernels and performance, distributed, disagg
49 +- [@WoosukKwon](https://github.com/WoosukKwon): Project lead, engine core
50 +- [@yaochengji](https://github.com/yaochengji): TPU integration
51 +- [@yeqcharlotte](https://github.com/yeqcharlotte): Benchmark, Llama models
52 +- [@yewentao256](https://github.com/yewentao256): Kernels and performance
53 +- [@Yikun](https://github.com/Yikun): Pluggable hardware interface
54 +- [@youkaichao](https://github.com/youkaichao): Project lead, distributed, compile, community
55 +- [@ywang96](https://github.com/ywang96): Multimodality, benchmarks
56 +- [@zhuohan123](https://github.com/zhuohan123): Project lead, RL integration, numerics
57 +- [@zou3519](https://github.com/zou3519): Compilation
58 +
59 +### Emeritus Committers
60 +
61 +Committers who have contributed to vLLM significantly in the past (thank you!) but no longer active:
62 +
63 +- [@andoorve](https://github.com/andoorve): Pipeline parallelism
64 +- [@cadedaniel](https://github.com/cadedaniel): Speculative decoding
65 +- [@comaniac](https://github.com/comaniac): KV cache management, pipeline parallelism
66 +- [@LiuXiaoxuanPKU](https://github.com/LiuXiaoxuanPKU): Speculative decoding
67 +- [@pcmoritz](https://github.com/pcmoritz): MoE
68 +- [@rkooo567](https://github.com/rkooo567): Chunked prefill
69 +- [@sroy745](https://github.com/sroy745): Speculative decoding
70 +- [@Yard1](https://github.com/Yard1): kernels and performance
71 +- [@zhisbug](https://github.com/zhisbug): Arctic models, distributed
72 +
73 +## Area Owners
74 +
75 +This section breaks down the active committers by vLLM components and lists the area owners.
76 +If you have PRs touching the area, please feel free to ping the area owner for review.
77 +
78 +### Engine Core
79 +
80 +- Scheduler: the core vLLM engine loop scheduling requests to next batch
81 + - @WoosukKwon, @robertgshaw2-redhat, @njhill, @heheda12345
82 +- KV Cache Manager: memory management layer within scheduler maintaining KV cache logical block data
83 + - @heheda12345, @WoosukKwon
84 +- AsyncLLM: the zmq based protocol hosting engine core and making it accessible for entrypoints
85 + - @robertgshaw2-redhat, @njhill, @russellb
86 +- ModelRunner, Executor, Worker: the abstractions for engine wrapping model implementation
87 + - @WoosukKwon, @tlrmchlsmth, @heheda12345, @LucasWilkinson, @ProExpertProg
88 +- KV Connector: Connector interface and implementation for KV cache offload and transfer
89 + - @robertgshaw2-redhat, @njhill, @KuntaiDu, @NickLucche, @ApostaC
90 +- Distributed, Parallelism, Process Management: Process launchers managing each worker, and assign them to the right DP/TP/PP/EP ranks
91 + - @youkaichao, @njhill, @WoosukKwon, @ruisearch42
92 +- Collectives: the usage of nccl and other communication libraries/kernels
93 + - @tlrmchlsmth, @youkaichao
94 +- Multimodality engine and memory management: core scheduling and memory management concerning vision, audio, and video inputs.
95 + - @ywang96, @DarkLight1337
96 +
97 +### Model Implementations
98 +
99 +- Model Interface: The `nn.Module` interface and implementation for various models
100 + - @zhuohan123, @mgoin, @simon-mo, @houseroad, @ywang96 (multimodality), @jeejeelee (lora)
101 +- Logits Processors / Sampler: The provided sampler class and pluggable logits processors
102 + - @njhill, @houseroad, @22quinn
103 +- Custom Layers: Utility layers in vLLM such as rotary embedding and rms norms
104 + - @ProExpertProg
105 +- Attention: Attention interface for paged attention
106 + - @WoosukKwon, @LucasWilkinson, @heheda12345
107 +- FusedMoE: FusedMoE kernel, Modular kernel framework, EPLB
108 + - @tlrmchlsmth
109 +- Quantization: Various quantization config, weight loading, and kernel.
110 + - @mgoin, @Isotr0py, @yewentao256
111 +- Custom quantized GEMM kernels (cutlass_scaled_mm, marlin, machete)
112 + - @tlrmchlsmth, @LucasWilkinson
113 +- Multi-modal Input Processing: Components that load and process image/video/audio data into feature tensors
114 + - @DarkLight1337, @ywang96, @Isotr0py
115 +- torch compile: The torch.compile integration in vLLM, custom passes & transformations
116 + - @ProExpertProg, @zou3519, @youkaichao
117 +- State space models: The state space models implementation in vLLM
118 + - @tdoublep, @tlrmchlsmth
119 +- Reasoning and tool calling parsers
120 + - @chaunceyjiang, @aarnphm
121 +
122 +### Entrypoints
123 +
124 +- LLM Class: The LLM class for offline inference
125 + - @DarkLight1337
126 +- API Server: The OpenAI-compatible API server
127 + - @DarkLight1337, @njhill, @aarnphm, @simon-mo, @heheda12345 (Responses API)
128 +- Batch Runner: The OpenAI-compatible batch runner
129 + - @simon-mo
130 +
131 +### Features
132 +
133 +- Spec Decode: Covers model definition, attention, sampler, and scheduler related to n-grams, EAGLE, and MTP.
134 + - @WoosukKwon, @benchislett, @luccafong
135 +- Structured Output: The structured output implementation
136 + - @russellb, @aarnphm
137 +- RL: The RL related features such as collective rpc, sleep mode, etc.
138 + - @youkaichao, @zhuohan123, @22quinn
139 +- LoRA: @jeejeelee
140 +- Observability: Metrics and Logging
141 + - @markmc, @robertgshaw2-redhat, @simon-mo
142 +
143 +### Code Base
144 +
145 +- Config: Configuration registration and parsing
146 + - @hmellor
147 +- Documentation: @hmellor, @DarkLight1337, @simon-mo
148 +- Benchmarks: @ywang96, @simon-mo
149 +- CI, Build, Release Process: @khluu, @njhill, @simon-mo
150 +- Security: @russellb
151 +
152 +### External Kernels Integration
153 +
154 +- FlashAttention: @LucasWilkinson
155 +- FlashInfer: @LucasWilkinson, @mgoin, @WoosukKwon
156 +- Blackwell Kernels: @mgoin, @yewentao256
157 +- DeepEP/DeepGEMM/pplx: @mgoin, @yewentao256
158 +
159 +### Integrations
160 +
161 +- Hugging Face: @hmellor, @Isotr0py
162 +- Ray: @ruisearch42
163 +- NIXL: @robertgshaw2-redhat, @NickLucche
164 +
165 +### Collaboration with Model Vendors
166 +
167 +- gpt-oss: @heheda12345, @simon-mo, @zhuohan123
168 +- Llama: @luccafong
169 +- Qwen: @sighingnow
170 +- Mistral: @patrickvonplaten
171 +
172 +### Hardware
173 +
174 +- Plugin Interface: @youkaichao, @Yikun
175 +- NVIDIA GPU: @pavanimajety
176 +- AMD GPU: @gshtras, @tjtanaa
177 +- Intel CPU/GPU: @jikunshang, @bigPYJ1151
178 +- Google TPU: @yaochengji
179 +
180 +### Ecosystem Projects
181 +
182 +- Ascend NPU: [@wangxiyuan](https://github.com/wangxiyuan) and [see more details](https://vllm-ascend.readthedocs.io/en/latest/community/contributors.html#maintainers)
183 +- Intel Gaudi HPU [@xuechendi](https://github.com/xuechendi) and [@kzawora-intel](https://github.com/kzawora-intel)
docs/governance/process.md new
+125
@@ -0,0 +1,125 @@
1 +# Governance Process
2 +
3 +vLLM's success comes from our strong open source community. We favor informal, meritocratic norms over formal policies. This document clarifies our governance philosophy and practices.
4 +
5 +## Values
6 +
7 +vLLM aims to be the fastest and easiest-to-use LLM inference and serving engine. We stay current with advances, enable innovation, and support diverse models, modalities, and hardware.
8 +
9 +### Design Values
10 +
11 +1. **Top performance**: System performance is our top priority. We monitor overheads, optimize kernels, and publish benchmarks. We never leave performance on the table.
12 +2. **Ease of use**: vLLM must be simple to install, configure, and operate. We provide clear documentation, fast startup, clean logs, helpful error messages, and monitoring guides. Many users fork our code or study it deeply, so we keep it readable and modular.
13 +3. **Wide coverage**: vLLM supports frontier models and high-performance accelerators. We make it easy to add new models and hardware. vLLM + PyTorch form a simple interface that avoids complexity.
14 +4. **Production ready**: vLLM runs 24/7 in production. It must be easy to operate and monitor for health issues.
15 +5. **Extensibility**: vLLM serves as fundamental LLM infrastructure. Our codebase cannot cover every use case, so we design for easy forking and customization.
16 +
17 +### Collaboration Values
18 +
19 +1. **Tightly Knit and Fast-Moving**: Our maintainer team is aligned on vision, philosophy, and roadmap. We work closely to unblock each other and move quickly.
20 +2. **Individual Merit**: No one buys their way into governance. Committer status belongs to individuals, not companies. We reward contribution, maintenance, and project stewardship.
21 +
22 +## Project Maintainers
23 +
24 +Maintainers form a hierarchy based on sustained, high-quality contributions and alignment with our design philosophy.
25 +
26 +### Core Maintainers
27 +
28 +Core Maintainers function like a project planning and decision making committee. In other convention, they might be called a Technical Steering Committee (TSC). In vLLM vocabulary, they are often known as "Project Leads". They meet weekly to coordinate roadmap priorities and allocate engineering resources. Current active leads: @WoosukKwon, @zhuohan123, @simon-mo, @youkaichao, @robertshaw2-redhat, @tlrmchlsmth, @mgoin, @njhill, @ywang96, @houseroad, @yeqcharlotte, @ApostaC
29 +
30 +The responsibilities of the core maintainers are:
31 +
32 +* Author quarterly roadmap and responsible for each development effort.
33 +* Making major changes to the technical direction or scope of vLLM and vLLM projects.
34 +* Defining the project's release strategy.
35 +* Work with model providers, hardware vendors, and key users of vLLM to ensure the project is on the right track.
36 +
37 +### Lead Maintainers
38 +
39 +While Core maintainers assume the day-to-day responsibilities of the project, Lead maintainers are responsible for the overall direction and strategy of the project. A committee of @WoosukKwon, @zhuohan123, @simon-mo, and @youkaichao currently shares this role with divided responsibilities.
40 +
41 +The responsibilities of the lead maintainers are:
42 +
43 +* Making decisions where consensus among core maintainers cannot be reached.
44 +* Adopting changes to the project's technical governance.
45 +* Organizing the voting process for new committers.
46 +
47 +### Committers and Area Owners
48 +
49 +Committers have write access and merge rights. They typically have deep expertise in specific areas and help the community.
50 +
51 +The responsibilities of the committers are:
52 +
53 +* Reviewing PRs and providing feedback.
54 +* Addressing issues and questions from the community.
55 +* Own specific areas of the codebase and development efforts: reviewing PRs, addressing issues, answering questions, improving documentation.
56 +
57 +Specially, committers are almost all area owners. They author subsystems, review PRs, refactor code, monitor tests, and ensure compatibility with other areas. All area owners are committers with deep expertise in that area, but not all committers own areas.
58 +
59 +For a full list of committers and their respective areas, see the [committers](./committers.md) page.
60 +
61 +#### Nomination Process
62 +
63 +Any committer can nominate candidates via our private mailing list:
64 +
65 +1. **Nominate**: Any committer may nominate a candidate by email to the private maintainers’ list, citing evidence mapped to the pre‑existing standards with links to PRs, reviews, RFCs, issues, benchmarks, and adoption evidence.
66 +2. **Vote**: The lead maintainers will group voices support or concerns. Shared concerns can stop the process. The vote typically last 3 working days. For concerns, committers group discuss the clear criteria for such person to be nominated again. The lead maintainers will make the final decision.
67 +3. **Confirm**: The lead maintainers send invitation, update CODEOWNERS, assign permissions, add to communications channels (mailing list and Slack).
68 +
69 +Committership is highly selective and merit based. The selection criteria requires:
70 +
71 +* **Area expertise**: leading design/implementation of core subsystems, material performance or reliability improvements adopted project‑wide, or accepted RFCs that shape technical direction.
72 +* **Sustained contributions**: high‑quality merged contributions and reviews across releases, responsiveness to feedback, and stewardship of code health.
73 +* **Community leadership**: mentoring contributors, triaging issues, improving docs, and elevating project standards.
74 +
75 +To further illustrate, a committer typically satisfies at least two of the following accomplishment patterns:
76 +
77 +* Author of an accepted RFC or design that materially shaped project direction
78 +* Measurable, widely adopted performance or reliability improvement in core paths
79 +* Long‑term ownership of a subsystem with demonstrable quality and stability gains
80 +* Significant cross‑project compatibility or ecosystem enablement work (models, hardware, tooling)
81 +
82 +While there isn't a quantitative bar, past committers have:
83 +
84 +* Submitted approximately 30+ PRs of substantial quality and scope
85 +* Provided high-quality reviews of approximately 10+ substantial external contributor PRs
86 +* Addressed multiple issues and questions from the community in issues/forums/Slack
87 +* Led concentrated efforts on RFCs and their implementation, or significant performance or reliability improvements adopted project‑wide
88 +
89 +### Working Groups
90 +
91 +vLLM runs informal working groups such as CI, CI infrastructure, torch compile, and startup UX. These can be loosely tracked via `#sig-` (or `#feat-`) channels in vLLM Slack. Some groups have regular sync meetings.
92 +
93 +### Advisory Board
94 +
95 +vLLM project leads consult with an informal advisory board that is composed of model providers, hardware vendors, and ecosystem partners. This manifests as a collaboration channel in Slack and frequent communications.
96 +
97 +## Process
98 +
99 +### Project Roadmap
100 +
101 +Project Leads publish quarterly roadmaps as GitHub issues. These clarify current priorities. Unlisted topics aren't excluded but may get less review attention. See [https://roadmap.vllm.ai/](https://roadmap.vllm.ai/).
102 +
103 +### Decision Making
104 +
105 +We make technical decisions in Slack and GitHub using RFCs and design docs. Discussion may happen elsewhere, but we maintain public records of significant changes: problem statements, rationale, and alternatives considered.
106 +
107 +### Merging Code
108 +
109 +Contributors and maintainers often collaborate closely on code changes, especially within organizations or specific areas. Maintainers should give others appropriate review opportunities based on change significance.
110 +
111 +PRs requires at least one committer review and approval. If the code is covered by CODEOWNERS, the PR should be reviewed by the CODEOWNERS. There are cases where the code is trivial or hotfix, the PR can be merged by the lead maintainers directly.
112 +
113 +In case where CI didn't pass due to the failure is not related to the PR, the PR can be merged by the lead maintainers using "force merge" option that overrides the CI checks.
114 +
115 +### Slack
116 +
117 +Contributors are encouraged to join `#pr-reviews` and `#contributors` channels.
118 +
119 +There are `#sig-` and `#feat-` channels for discussion and coordination around specific topics.
120 +
121 +The project maintainer group also uses a private channel for high-bandwidth collaboration.
122 +
123 +### Meetings
124 +
125 +We hold weekly contributor syncs with standup-style updates on progress, blockers, and plans. You can refer to the notes [standup.vllm.ai](https://standup.vllm.ai) for joining instructions.