@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / af69a6aded
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 02:00 UTC

fix: update platform detection for M-series arm based MacBook processors (#12227)

Signed-off-by: isikhi <huseyin.isik000@gmail.com>

Işık committed Jan 20, 2025 at 22:23 UTC af69a6aded526343db8d4a199cdfd1bb84134201
1 file changed +4
vllm/platforms/__init__.py
+4
@@ -101,6 +101,10 @@ def cpu_platform_plugin() -> Optional[str]:
101 try:
102 from importlib.metadata import version
103 is_cpu = "cpu" in version("vllm")
104 + if is_cpu == False:
105 + import platform
106 + is_cpu = platform.machine().lower().startswith("arm")
107 +
108 except Exception:
109 pass
110