@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 25f9c78961
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 07:00 UTC

[misc][plugin] improve plugin loading (#10443)

Signed-off-by: youkaichao <youkaichao@gmail.com>

youkaichao committed Nov 19, 2024 at 02:43 UTC 25f9c78961daae10b9084d78901d71bc56691aa1
1 file changed +7
vllm/plugins/__init__.py
+7
@@ -9,12 +9,19 @@ if TYPE_CHECKING:
9
10 logger = logging.getLogger(__name__)
11
12 +# make sure one process only loads plugins once
13 +plugins_loaded = False
14 +
15
16 def load_general_plugins():
17 """WARNING: plugins can be loaded for multiple times in different
18 processes. They should be designed in a way that they can be loaded
19 multiple times without causing issues.
20 """
21 + global plugins_loaded
22 + if plugins_loaded:
23 + return
24 + plugins_loaded = True
25 import sys
26 if sys.version_info < (3, 10):
27 from importlib_metadata import entry_points