A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 7, 01:30 UTC
[Bugfix] Skip out-of-stage layers in get_layers_from_vllm_config for pipeline parallel (#36243)
Signed-off-by: Tushar Shetty <tushar.shetty@abbyy.com> Signed-off-by: Tushar Shetty <54362365+tusharshetty61@users.noreply.github.com>
Tushar Shetty committed
Mar 9, 2026 at 09:10 UTC
c4d859c274960d62f0b2ff6e7ac96be452994b55
1 file changed
+2
-1
vllm/config/vllm.py
+2
-1
@@ -1795,5 +1795,6 @@ def get_layers_from_vllm_config(
1795
return {
1796
layer_name: forward_context[layer_name]
1797
for layer_name in layer_names
1798
- if isinstance(forward_context[layer_name], layer_type)
1798
+ if layer_name in forward_context
1799
+ and isinstance(forward_context[layer_name], layer_type)
1800
}