AGENTS MD: Add suggestion on how to incorporate tests (#47784)
Signed-off-by: Simon Mo <simon.mo@hey.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Simon Mo committed
Jul 7, 2026 at 08:16 UTC
21b396abe14bd35d821c346d10961a97de4f2d56
1 file changed
+30
-19
AGENTS.md
+30
-19
@@ -29,6 +29,7 @@ Do not open one-off PRs for tiny edits (single typo, isolated style change, one
29
- PR descriptions for AI-assisted work **must** include:
30
- Why this is not duplicating an existing PR.
31
- Test commands run and results.
32
+ - Model evaluation results when the change affects output, accuracy, or serving.
33
- Clear statement that AI assistance was used.
34
35
### Fail-closed behavior
@@ -66,23 +67,38 @@ VLLM_USE_PRECOMPILED=1 uv pip install -e . --torch-backend=auto
67
uv pip install -e . --torch-backend=auto
68
```
69
69
-### Running tests
70
+### Tests
71
72
> Requires [Environment setup](#environment-setup) and [Installing dependencies](#installing-dependencies).
73
74
```bash
74
-# Install test dependencies.
75
-# requirements/test/cuda.txt is pinned to x86_64; on other platforms, use the
76
-# unpinned source file instead:
77
-uv pip install -r requirements/test/cuda.in # resolves for current platform
78
-# Or on x86_64:
79
-uv pip install -r requirements/test/cuda.txt
80
-
81
-# Run a specific test file (use .venv/bin/python directly;
82
-# `source activate` does not persist in non-interactive shells):
75
+# Install test dependencies (use cuda.in on non-x86_64):
76
+uv pip install -r requirements/test/cuda.in
77
+
78
+# Run a specific test file:
79
.venv/bin/python -m pytest tests/path/to/test_file.py -v
80
```
81
82
+When adding tests:
83
+
84
+- **Design before you write.** Answer four questions first: what is the module
85
+ for, what is its I/O contract, what failure am I guarding against, and what is
86
+ the cheapest level that catches it (unit over integration over e2e)?
87
+- **Reuse before create.** Extend existing test files, `conftest.py` fixtures, and
88
+ helpers; add a new file only when no nearby suite fits.
89
+- **Test behavior with intent.** Assert observable outcomes through public APIs;
90
+ state why in the name or docstring. Skip trivial wiring; flaky tests are worse
91
+ than no tests.
92
+- **Keep it minimal.** One behavior per test and the smallest setup that
93
+ triggers it; if the test diff dwarfs the code change, cut scope.
94
+- **No one-off kernel benchmarks in `tests/`.** Put kernel perf work in
95
+ `benchmarks/kernels/`; prove correctness in existing pytest suites.
96
+- **Run model evals for model-affecting changes.** Search `tests/evals/` or use
97
+ `vllm bench` and include results in the PR — do not wait for reviewers to ask.
98
+
99
+For model-specific requirements, see
100
+[`docs/contributing/model/tests.md`](docs/contributing/model/tests.md).
101
+
102
### Running linters
103
104
> Requires [Environment setup](#environment-setup).
@@ -107,23 +123,18 @@ Use [Google-style docstrings](https://google.github.io/styleguide/pyguide.html#3
123
124
### Coding style guidelines
125
110
-Follow these rules for all code changes in this repository:
111
-
112
-- Try to match existing code style.
113
-- Code should be self-documenting and self-explanatory.
114
-- Keep comments and docstrings minimal and concise.
126
+- Match existing code style
127
+- Minimize use of comments. Eliminate comments which are redundant, preferring legible and self-documenting code. When used, keep docstrings and comments brief and direct.
128
- Assume the reader is familiar with vLLM.
129
130
### Commit messages
131
119
-Add attribution using commit trailers such as `Co-authored-by:` (other projects use `Assisted-by:` or `Generated-by:`). For example:
132
+Add attribution using commit trailers such as `Co-authored-by:` (other projects use `Assisted-by:` or `Generated-by:`):
133
134
```text
135
Your commit message here
136
124
-Co-authored-by: GitHub Copilot
125
-Co-authored-by: Claude
126
-Co-authored-by: gemini-code-assist
137
+Co-authored-by: Agent Name Here
138
Signed-off-by: Your Name <your.email@example.com>
139
```
140