[Skills] Add kernel benchmark sanity references (#54995)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Woosuk Kwon committed
Sep 2, 2026 at 17:58 UTC
0e14198a63c03f899a10f3e782e88eca7f11265b
1 file changed
+19
.agents/skills/kernel-microbenchmark/SKILL.md
+19
@@ -42,6 +42,25 @@ description: Build, debug, and interpret vLLM GPU kernel microbenchmarks for CUD
42
- Keep metadata setup, plan construction, allocation, random input generation,
43
and logging outside the timed region unless that overhead is the experiment.
44
45
+## Sanity-Check Reference Numbers
46
+
47
+Use these as rough reference points for large, well-shaped workloads, not as
48
+gold standards, guaranteed peaks, or hard limits. Hardware SKU, clocks, shape,
49
+precision conventions, and the FLOP/byte accounting can move the result. A
50
+large gap is a prompt to investigate, not proof that a kernel is poor.
51
+
52
+| Kernel regime | Hardware | Rough reference |
53
+| --- | --- | ---: |
54
+| Memory-bound, large batch | Blackwell | 6 TB/s |
55
+| BF16 GEMM | Blackwell | 2 PFLOP/s |
56
+| BF16 attention | B200 | 1.6 PFLOP/s |
57
+| FP8 GEMM | Blackwell | 4 PFLOP/s |
58
+| FP8 attention | B300 | 2.8 PFLOP/s |
59
+
60
+The BF16 attention reference is approximately the 1613 TFLOP/s result reported
61
+by the FlashAttention-4 paper. Compare kernels only with matching workload and
62
+throughput conventions.
63
+
64
## Multi-GPU Benchmarks
65
66
- State whether the run is local or multi-node and report the GPU topology,