@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 4f074fbf53
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 04:30 UTC

[Misc]Suppress irrelevant exception stack trace information when CUDA… (#11438)

Co-authored-by: shiquan <shiquan>

dpxa committed Dec 24, 2024 at 16:43 UTC 4f074fbf53f9e11a57a4c3d8b084796d155a270a
1 file changed +2 -1
vllm/entrypoints/llm.py
+2 -1
@@ -233,7 +233,8 @@ class LLM:
233 self.request_counter = Counter()
234
235 def __del__(self):
236 - if self.llm_engine and hasattr(self.llm_engine, "shutdown"):
236 + if hasattr(self, 'llm_engine') and self.llm_engine and hasattr(
237 + self.llm_engine, "shutdown"):
238 self.llm_engine.shutdown()
239
240 @staticmethod