@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 01:45 UTC

main

[Kernel] Use flash-attn for decoding (#3648)

Stephen Krider committed over 2 years ago

[Misc] Enhance attention selector (#4751)

Woosuk Kwon committed over 2 years ago

[CORE] Improvement in ranks code (#4718)

Swapnil Parekh committed over 2 years ago

[CI/Build] Tweak Marlin Nondeterminism Issues (#4713)

Robert Shaw committed over 2 years ago

[Speculative decoding] CUDA graph support (#4295)

heeju-kim2 committed over 2 years ago

[Misc] Apply a couple g++ cleanups (#4719)

Steve Grubb committed over 2 years ago

[Core]fix type annotation for `swap_blocks` (#4726)

Kunshang Ji committed over 2 years ago

chunked-prefill-doc-syntax (#4603)

Simon Mo committed over 2 years ago

← Newer Page 1054 of 1120 Older →