| 1 | # gRPC protocol |
| 2 | |
| 3 | This directory is the canonical source for vLLM's gRPC schema. |
| 4 | |
| 5 | See [context parallel deployment](../../docs/serving/context_parallel_deployment.md) |
| 6 | for effective attention block-size metadata in Python and `Control.GetServerInfo`. |
| 7 | |
| 8 | Schema updates are no longer published to the Buf Schema Registry. Rust consumers |
| 9 | should use `vllm-proto` from crates.io; consumers in other languages can generate |
| 10 | bindings from the `.proto` files in this directory. Buf still builds and lints |
| 11 | the schemas on pull requests, including on forks. |
| 12 | |
| 13 | ## Rust crate |
| 14 | |
| 15 | `vllm-proto` exposes the Prost message types and Tonic client and server modules for both APIs: |
| 16 | |
| 17 | ```toml |
| 18 | [dependencies] |
| 19 | vllm-proto = "0.4" |
| 20 | ``` |
| 21 | |
| 22 | For example, import `vllm_proto::inference_client::InferenceClient` or |
| 23 | `vllm_proto::control_client::ControlClient`. The vLLM frontend uses this same crate. |
| 24 | The package includes the canonical `.proto` files and generates bindings during |
| 25 | compilation using `protox` and `tonic-prost-build`. Consumers do not need Buf |
| 26 | credentials, the vLLM checkout, or a separately installed `protoc`. |
| 27 | |
| 28 | ### Versioning and releases |
| 29 | |
| 30 | The crate has its own version in `Cargo.toml`, independent of the vLLM release |
| 31 | number and the other Rust workspace crates. Release it when changes to the |
| 32 | schema, generated Rust API, or dependencies require a new version. Review both |
| 33 | Rust API and protobuf wire compatibility when choosing the version bump; |
| 34 | a wire-compatible schema addition can still break Rust callers. |
| 35 | |
| 36 | On pull requests and releases, `cargo-semver-checks` compares the crate with its |
| 37 | latest published version. Include any required version bump in the protocol |
| 38 | change PR. This check becomes available after the first manual publication. |
| 39 | |
| 40 | 1. Update the crate version in `rust/proto/Cargo.toml` and update `rust/Cargo.lock`. |
| 41 | 2. Run `cargo publish --manifest-path rust/proto/Cargo.toml --locked --dry-run` |
| 42 | and the frontend gRPC tests. Record the tested vLLM releases or revisions in |
| 43 | the release notes; matching crate versions alone do not establish runtime compatibility. |
| 44 | 3. After the change merges, create a `proto-v<version>` tag on that commit. |
| 45 | The `proto-crate.yml` workflow checks that the tag matches the crate version, |
| 46 | verifies the package, and publishes it to crates.io. |
| 47 | |
| 48 | There is no scheduled crate publication. Pull requests and manual workflow runs only |
| 49 | verify packaging, including on forks. Retry a failed release by rerunning its |
| 50 | tag workflow; crates.io versions cannot be overwritten. |
| 51 | |
| 52 | ### Publisher setup |
| 53 | |
| 54 | A vLLM maintainer must publish the first version and establish the crate's owners. |
| 55 | For subsequent releases, configure [crates.io Trusted Publishing](https://crates.io/docs/trusted-publishing) |
| 56 | for repository `vllm-project/vllm`, workflow `proto-crate.yml`, and environment |
| 57 | `crates-io`. Configure that GitHub environment to allow `proto-v*` release tags. |
| 58 | Only tag pushes in the upstream repository can publish; forks do not obtain |
| 59 | publishing credentials. |