A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 14:45 UTC
[CI][ROCm] Add Qwen3.5-35B-A3B-MXFP4 model eval into CI (#38664)
Signed-off-by: Bowen Bao <bowenbao@amd.com>
Bowen Bao committed
Apr 2, 2026 at 21:05 UTC
201d2ea5bfb932bb79e0a7c5dba0abbd9cb0570d
2 files changed
+9
tests/evals/gsm8k/configs/Qwen3.5-35B-A3B-MXFP4-TP2.yaml
new
+8
@@ -0,0 +1,8 @@
1
+model_name: "amd/Qwen3.5-35B-A3B-MXFP4"
2
+accuracy_threshold: 0.82
3
+tolerance: 0.03
4
+num_questions: 1319
5
+num_fewshot: 5
6
+server_args: >-
7
+ --max-model-len 4096
8
+ --tensor-parallel-size 2
tests/evals/gsm8k/configs/models-qwen35-mi355.txt
+1
@@ -1 +1,2 @@
1
Qwen3.5-35B-A3B-DEP2.yaml
2
+Qwen3.5-35B-A3B-MXFP4-TP2.yaml