@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / ca5f54a9b9
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 04:30 UTC

[Bugfix] fix minicpmv test (#11304)

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>

Joe Runde committed Dec 18, 2024 at 10:34 UTC ca5f54a9b93a9d044458b8103ed8c9dcc62e6611
1 file changed -1
tests/lora/test_minicpmv.py
-1
@@ -67,7 +67,6 @@ def test_minicpmv_lora(minicpmv_lora_files):
67 max_loras=4,
68 max_lora_rank=64,
69 trust_remote_code=True,
70 - gpu_memory_utilization=0.97, # This model is pretty big for CI gpus
70 enable_chunked_prefill=True,
71 )
72 output1 = do_sample(llm, minicpmv_lora_files, lora_id=1)