@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 20:45 UTC

main

[Misc] Add a wrapper for torch.inference_mode (#6618)

Woosuk Kwon committed over 2 years ago

[Model] Initial Support for Chameleon (#5770)

Roger Wang committed over 2 years ago

[Kernel][Core] Add AWQ support to the Marlin kernel (#6612)

Alexander Matveev committed over 2 years ago

[Misc] Remove abused noqa (#6619)

Woosuk Kwon committed over 2 years ago

[Frontend] Move chat utils (#6602)

Cyrus Leung committed over 2 years ago

[ Bugfix ] Fix AutoFP8 fp8 marlin (#6609)

Robert Shaw committed over 2 years ago

[Misc] Fix input_scale typing in w8a8_utils.py (#6579)

Michael Goin committed over 2 years ago

[ Misc ] `fbgemm` checkpoints (#6559)

Robert Shaw committed over 2 years ago

[Core] Allow specifying custom Executor (#6557)

Antoni Baum committed over 2 years ago

[ Kernel ] FP8 Dynamic Per Token Quant - Add scale_ub (#6593)

Varun Sundar Rabindranath committed over 2 years ago

[Docs] Update PP docs (#6598)

Murali Andoorveedu committed over 2 years ago

[ Kernel ] Enable Dynamic Per Token `fp8` (#6547)

Robert Shaw committed over 2 years ago

← Newer Page 1020 of 1120 Older →