@tduncan / Nvidia-Optimization-Stack-VLLM
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
yaml 6 lines 211 Bytes
Raw
1 model_name: "nm-testing/Qwen1.5-MoE-A2.7B-Chat-quantized.w4a16"
2 accuracy_threshold: 0.45
3 num_questions: 1319
4 num_fewshot: 5
5 rocm_request_timeout_seconds: 1800
6 server_args: "--enforce-eager --max-model-len 4096"