@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 18572e3384
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 02:00 UTC

[Bugfix] Fix `HfExampleModels.find_hf_info` (#12223)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Cyrus Leung committed Jan 20, 2025 at 23:35 UTC 18572e3384a6f55a7589dd81e1f3f70f7dd73e3a
1 file changed +5
tests/models/registry.py
+5
@@ -302,6 +302,11 @@ class HfExampleModels:
302 if info.default == model_id:
303 return info
304
305 + # Fallback to extras
306 + for info in self.hf_models.values():
307 + if any(extra == model_id for extra in info.extras.values()):
308 + return info
309 +
310 raise ValueError(f"No example model defined for {model_id}")
311
312