A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
docs: update fastsafetensors usage instructions (#22891)
Signed-off-by: Nir Levy <bhr166@gmail.com>
Nir committed
Aug 14, 2025 at 22:56 UTC
637093ae26953f20aef328424e4307826820efe4
1 file changed
+2
-1
docs/models/extensions/fastsafetensor.md
+2
-1
@@ -2,4 +2,5 @@ Loading Model weights with fastsafetensors
2
===================================================================
3
4
Using fastsafetensors library enables loading model weights to GPU memory by leveraging GPU direct storage. See [their GitHub repository](https://github.com/foundation-model-stack/fastsafetensors) for more details.
5
-For enabling this feature, set the environment variable ``USE_FASTSAFETENSOR`` to ``true``
5
+
6
+To enable this feature, use the ``--load-format fastsafetensors`` command-line argument