@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 4f5d8446ed
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 17:00 UTC

[Bugfix] Fix ModelScope models in v0.5.5 (#8037)

Nicolò Lucchesi committed Aug 31, 2024 at 09:27 UTC 4f5d8446ede9f85182126804c6b07a56e06fd3d1
1 file changed +3
vllm/transformers_utils/config.py
+3
@@ -108,6 +108,9 @@ def get_hf_image_processor_config(
108 revision: Optional[str] = None,
109 **kwargs,
110 ) -> Dict[str, Any]:
111 + # ModelScope does not provide an interface for image_processor
112 + if VLLM_USE_MODELSCOPE:
113 + return dict()
114 # Separate model folder from file path for GGUF models
115 if Path(model).is_file() and Path(model).suffix == ".gguf":
116 model = Path(model).parent