A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 13:30 UTC
[Doc] Minor fix for the vLLM TPU setup page (#17206)
Signed-off-by: Yarong Mu <ymu@google.com>
yarongmu-google committed
Apr 25, 2025 at 21:39 UTC
7bd0c7745c162496fb2b78e2326967332c893c97
1 file changed
+4
-4
docs/source/getting_started/installation/ai_accelerator/tpu.inc.md
+4
-4
@@ -44,7 +44,7 @@ There are no pre-built wheels for this device, so you must either use the pre-bu
44
45
You can provision Cloud TPUs using the [Cloud TPU API](https://cloud.google.com/tpu/docs/reference/rest)
46
or the [queued resources](https://cloud.google.com/tpu/docs/queued-resources)
47
-API. This section shows how to create TPUs using the queued resource API. For
47
+API (preferred). This section shows how to create TPUs using the queued resource API. For
48
more information about using the Cloud TPU API, see [Create a Cloud TPU using the Create Node API](https://cloud.google.com/tpu/docs/managing-tpus-tpu-vm#create-node-api).
49
Queued resources enable you to request Cloud TPU resources in a queued manner.
50
When you request queued resources, the request is added to a queue maintained by
@@ -97,10 +97,10 @@ gcloud alpha compute tpus queued-resources create QUEUED_RESOURCE_ID \
97
`TPU regions and zones <https://cloud.google.com/tpu/docs/regions-zones>`_
98
- * ACCELERATOR_TYPE
99
* The TPU version you want to use. Specify the TPU version, for example
100
- `v5litepod-4` specifies a v5e TPU with 4 cores. For more information,
101
- see `TPU versions <https://cloud.devsite.corp.google.com/tpu/docs/system-architecture-tpu-vm#versions>`_.
100
+ `v5litepod-4` specifies a v5e TPU with 4 cores, `v6e-1` specifies a v6e TPU with 1 core. For more information,
101
+ see [TPU versions](https://cloud.devsite.corp.google.com/tpu/docs/system-architecture-tpu-vm#versions).
102
- * RUNTIME_VERSION
103
- * The TPU VM runtime version to use. For more information see `TPU VM images <https://cloud.google.com/tpu/docs/runtimes>`_.
103
+ * The TPU VM runtime version to use. For example, use `v2-alpha-tpuv6e` for a VM loaded with one or more v6e TPU(s). For more information see [TPU VM images](https://cloud.google.com/tpu/docs/runtimes).
104
- * SERVICE_ACCOUNT
105
* The email address for your service account. You can find it in the IAM
106
Cloud Console under *Service Accounts*. For example: