@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 03:00 UTC

main

[Core] latency optimization (#3890)

youkaichao committed over 2 years ago

[Core] enable out-of-tree model register (#3871)

youkaichao committed over 2 years ago

[Misc] Define common requirements (#3841)

Woosuk Kwon committed over 2 years ago

[Bugfix] Fixing requirements.txt (#3865)

Noam Gat committed over 2 years ago

[CI/Build] refactor dockerfile & fix pip cache

youkaichao committed over 2 years ago

[Core] improve robustness of pynccl (#3860)

youkaichao committed over 2 years ago

[Model] Cohere CommandR+ (#3829)

Saurabh Dash committed over 2 years ago

[Bugfix] Fix args in benchmark_serving (#3836)

Chang Su committed over 2 years ago

[Core] [Frontend] Make detokenization optional (#3749)

Matthias Gerstgrasser committed over 2 years ago

← Newer Page 1068 of 1120 Older →