@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 01843c89b8
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[Misc] log when using default MoE config (#8971)

Divakar Verma committed Oct 2, 2024 at 23:31 UTC 01843c89b8ddae00d4a0f0f56b8aa7fbaa3efc42
1 file changed +3
vllm/model_executor/layers/fused_moe/fused_moe.py
+3
@@ -320,6 +320,9 @@ def get_moe_configs(E: int, N: int,
320
321 # If no optimized configuration is available, we will use the default
322 # configuration
323 + logger.warning(
324 + ("Using default MoE config. Performance might be sub-optimal! "
325 + "Config file not found at %s"), config_file_path)
326 return None
327
328