@tduncan / Nvidia-Optimization-Stack-VLLM
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
yaml 5 lines 189 Bytes
Raw
1 model_name: "amd/Mixtral-8x7B-Instruct-v0.1-FP8-KV"
2 accuracy_threshold: 0.62
3 num_questions: 1319
4 num_fewshot: 5
5 server_args: "--enforce-eager --max-model-len 8192 --tensor-parallel-size 2"