@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / c354072828
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 00:30 UTC

[Minor] Fix the path typo in loader.py: save_sharded_states.py -> save_sharded_state.py (#5151)

Signed-off-by: Ye Cao <caoye.cao@alibaba-inc.com>

Ye Cao committed Jun 2, 2024 at 01:11 UTC c35407282878cb3a42860d584a4d9eb6aed82299
1 file changed +1 -1
vllm/model_executor/model_loader/loader.py
+1 -1
@@ -386,7 +386,7 @@ class ShardedStateLoader(BaseModelLoader):
386 Model loader that directly loads each worker's model state dict, which
387 enables a fast load path for large tensor-parallel models where each worker
388 only needs to read its own shard rather than the entire checkpoint. See
389 - `examples/save_sharded_states.py` for creating a sharded checkpoint.
389 + `examples/save_sharded_state.py` for creating a sharded checkpoint.
390 """
391
392 DEFAULT_PATTERN = "model-rank-{rank}-part-{part}.safetensors"