A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Docs][TPU] Highlight TPU Software version selection (#22242)
Signed-off-by: NickLucche <nlucches@redhat.com>
Nicolò Lucchesi committed
Aug 5, 2025 at 11:33 UTC
d1bf1b97111df876737e3af3d9249c7ccc545f15
1 file changed
+4
-1
docs/getting_started/installation/google_tpu.md
+4
-1
@@ -85,7 +85,7 @@ gcloud alpha compute tpus queued-resources create QUEUED_RESOURCE_ID \
85
| PROJECT_ID | Your Google Cloud project |
86
| ZONE | The GCP zone where you want to create your Cloud TPU. The value you use depends on the version of TPUs you are using. For more information, see [TPU regions and zones] |
87
| ACCELERATOR_TYPE | The TPU version you want to use. Specify the TPU version, for example `v5litepod-4` specifies a v5e TPU with 4 cores, `v6e-1` specifies a v6e TPU with 1 core. For more information, see [TPU versions]. |
88
-| RUNTIME_VERSION | The TPU VM runtime version to use. For example, use `v2-alpha-tpuv6e` for a VM loaded with one or more v6e TPU(s). For more information see [TPU VM images]. |
88
+| RUNTIME_VERSION | The TPU VM runtime version to use. For example, use `v2-alpha-tpuv6e` for a VM loaded with one or more v6e TPU(s). |
89
| SERVICE_ACCOUNT | The email address for your service account. You can find it in the IAM Cloud Console under *Service Accounts*. For example: `tpu-service-account@<your_project_ID>.iam.gserviceaccount.com` |
90
91
Connect to your TPU VM using SSH:
@@ -94,6 +94,9 @@ Connect to your TPU VM using SSH:
94
gcloud compute tpus tpu-vm ssh TPU_NAME --project PROJECT_ID --zone ZONE
95
```
96
97
+!!! note
98
+ When configuring `RUNTIME_VERSION` ("TPU software version") on GCP, ensure it matches the TPU generation you've selected by referencing the [TPU VM images] compatibility matrix. Using an incompatible version may prevent vLLM from running correctly.
99
+
100
[TPU versions]: https://cloud.google.com/tpu/docs/runtimes
101
[TPU VM images]: https://cloud.google.com/tpu/docs/runtimes
102
[TPU regions and zones]: https://cloud.google.com/tpu/docs/regions-zones