A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[TPU] update TPU benchmark threshold (#25713)
Signed-off-by: Johnny Yang <johnnyyang@google.com>
Johnny Yang committed
Oct 7, 2025 at 13:53 UTC
59012df99bc716510951a1e2c64425da32372c07
1 file changed
+1
-1
.buildkite/scripts/tpu/quantized_v6e_1.env
+1
-1
@@ -9,6 +9,6 @@ MAX_NUM_BATCHED_TOKENS=1024
9
TENSOR_PARALLEL_SIZE=1
10
MAX_MODEL_LEN=2048
11
DOWNLOAD_DIR=/mnt/disks/persist
12
-EXPECTED_THROUGHPUT=10.0
12
+EXPECTED_THROUGHPUT=8.7
13
INPUT_LEN=1800
14
OUTPUT_LEN=128