@tduncan / Nvidia-Optimization-Stack-VLLM
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
yaml 11 lines 276 Bytes
Raw
1 model_name: "Qwen/Qwen3-30B-A3B-GPTQ-Int4"
2 accuracy_threshold: 0.86
3 num_questions: 1319
4 num_fewshot: 5
5 server_args: >-
6 --enforce-eager
7 --max-model-len 8192
8 --moe-backend humming
9 --linear-backend humming
10 env:
11 VLLM_HUMMING_INPUT_QUANT_CONFIG: '{"dtype":"float8e4m3"}'