@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 3724d5f6b5
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 17:00 UTC

[Bugfix][Model] Fix Python 3.8 compatibility in Pixtral model by updating type annotations (#8490)

Chris committed Sep 15, 2024 at 06:20 UTC 3724d5f6b59d9859e5b47c047535bb8edc124eab
1 file changed +1 -1
vllm/model_executor/models/pixtral.py
+1 -1
@@ -454,7 +454,7 @@ class Transformer(nn.Module):
454 return x
455
456
457 -def position_meshgrid(patch_embeds_list: list[torch.Tensor], ) -> torch.Tensor:
457 +def position_meshgrid(patch_embeds_list: List[torch.Tensor], ) -> torch.Tensor:
458 positions = torch.cat([
459 torch.stack(
460 torch.meshgrid(