@tduncan / Nvidia-Optimization-Stack-VLLM
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
yaml 7 lines 268 Bytes
Raw
1 model_name: "nvidia/Llama-4-Scout-17B-16E-Instruct-FP8"
2 accuracy_threshold: 0.92
3 num_questions: 1319
4 num_fewshot: 5
5 server_args: "--enforce-eager --max-model-len 8192 --data-parallel-size 2 --enable-expert-parallel --moe-backend=triton"
6 env:
7 VLLM_USE_DEEP_GEMM: "0"