A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 05:15 UTC
[Misc] Add In-Container restart capability through supervisord for sagemaker entrypoint (#28502)
Signed-off-by: Shen Teng <sheteng@amazon.com> Signed-off-by: HappyAmazonian <91216626+HappyAmazonian@users.noreply.github.com>
HappyAmazonian committed
Jan 13, 2026 at 13:06 UTC
2f4a71daf200f4840d11435d932c676e943f2de3
2 files changed
+2
-2
examples/online_serving/sagemaker-entrypoint.sh
+1
-1
@@ -21,4 +21,4 @@ while IFS='=' read -r key value; do
21
done < <(env | grep "^${PREFIX}")
22
23
# Pass the collected arguments to the main entrypoint
24
-exec vllm serve "${ARGS[@]}"
\ No newline at end of file
24
+exec standard-supervisor vllm serve "${ARGS[@]}"
\ No newline at end of file
requirements/common.txt
+1
-1
@@ -49,7 +49,7 @@ ijson # Required for mistral streaming tool parser
49
setproctitle # Used to set process names for better debugging and monitoring
50
openai-harmony >= 0.0.3 # Required for gpt-oss
51
anthropic == 0.71.0
52
-model-hosting-container-standards >= 0.1.10, < 1.0.0
52
+model-hosting-container-standards >= 0.1.13, < 1.0.0
53
mcp
54
grpcio>=1.76.0
55
grpcio-reflection>=1.76.0
\ No newline at end of file