@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 9c6459e4cb
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 05:45 UTC

[Neuron] Upgrade neuron to 2.20.2 (#11016)

Signed-off-by: Jerzy Zagorski <jzagorsk@amazon.com> Co-authored-by: Jerzy Zagorski <jzagorsk@amazon.com>

xendo committed Dec 9, 2024 at 22:53 UTC 9c6459e4cb020ec1ad9ea08cac9309b83d432fc8
2 files changed +3 -2
Dockerfile.neuron
+2 -1
@@ -1,5 +1,6 @@
1 # default base image
2 -ARG BASE_IMAGE="public.ecr.aws/neuron/pytorch-inference-neuronx:2.1.2-neuronx-py310-sdk2.20.0-ubuntu20.04"
2 +# https://gallery.ecr.aws/neuron/pytorch-inference-neuronx
3 +ARG BASE_IMAGE="public.ecr.aws/neuron/pytorch-inference-neuronx:2.1.2-neuronx-py310-sdk2.20.2-ubuntu20.04"
4
5 FROM $BASE_IMAGE
6
vllm/utils.py
+1 -1
@@ -1628,7 +1628,7 @@ def direct_register_custom_op(
1628 library object. If you want to bind the operator to a different library,
1629 make sure the library object is alive when the operator is used.
1630 """
1631 - if is_in_doc_build():
1631 + if is_in_doc_build() or not supports_custom_op():
1632 return
1633 import torch.library
1634 if hasattr(torch.library, "infer_schema"):