@tduncan / Nvidia-Optimization-Stack-VLLM
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
yaml 10 lines 233 Bytes
Raw
1 model_name: "nvidia/Qwen3.5-397B-A17B-NVFP4"
2 accuracy_threshold: 0.88
3 tolerance: 0.03
4 num_questions: 1319
5 num_fewshot: 5
6 server_args: >-
7 --max-model-len 4096
8 --data-parallel-size 2
9 --enable-expert-parallel
10 --max-num-seqs 256