A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[CI] Small Accuracy Eval Test for Deepseek Model (#24259)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Wentao Ye committed
Sep 15, 2025 at 22:14 UTC
3c96e7b8a1ffac31b947c753f4fe1d0cd14abf87
2 files changed
+7
tests/evals/gsm8k/configs/DeepSeek-V2-Lite-Instruct-FP8.yaml
new
+6
@@ -0,0 +1,6 @@
1
+model_name: "RedHatAI/DeepSeek-Coder-V2-Lite-Instruct-FP8"
2
+accuracy_threshold: 0.72
3
+num_questions: 1319
4
+num_fewshot: 5
5
+max_model_len: 4096
6
+
tests/evals/gsm8k/configs/models-small.txt
+1
@@ -3,3 +3,4 @@ Llama-3.2-1B-Instruct-INT8-CT.yaml
3
Llama-3-8B-Instruct-nonuniform-CT.yaml
4
Qwen2.5-VL-3B-Instruct-FP8-dynamic.yaml
5
Qwen1.5-MoE-W4A16-CT.yaml
6
+DeepSeek-V2-Lite-Instruct-FP8.yaml