@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

main

Update Ray version requirements (#2636)

Simon Mo committed almost 3 years ago

Small async_llm_engine refactor (#2618)

Murali Andoorveedu committed almost 3 years ago

Speed up Punica compilation (#2632)

Woosuk Kwon committed almost 3 years ago

[Minor] Fix warning on Ray dependencies (#2630)

Woosuk Kwon committed almost 3 years ago

Implement custom all reduce kernels (#2192)

Hanzhi Zhou committed almost 3 years ago

Use head_dim in config if exists (#2622)

Xiang Xu committed almost 3 years ago

AWQ: Up to 2.66x higher throughput (#2566)

Casper committed almost 3 years ago

Don't build punica kernels by default (#2605)

Philipp Moritz committed almost 3 years ago

Support for Stable LM 2 (#2598)

dakotamahan-stability committed almost 3 years ago

[ROCm] add support to ROCm 6.0 and MI300 (#2274)

Hongxia Yang committed almost 3 years ago

fix names and license for Qwen2 (#2589)

Junyang Lin committed almost 3 years ago

Support Batch Completion in Server (#2529)

Simon Mo committed almost 3 years ago

[Bugfix] fix crash if max_tokens=None (#2570)

Nikola Borisov committed almost 3 years ago

[Experimental] Add multi-LoRA support (#1804)

Antoni Baum committed almost 3 years ago

← Newer Page 1086 of 1120 Older →