A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 03:15 UTC
main
[CI] Turn on basic correctness tests for V1 (#10864)
Tyler Michael Smith committed almost 2 years ago
[Docs] Add Google Cloud Meetup (#11864)
Simon Mo committed almost 2 years ago
[Kernel][Triton][AMD] Use block size heuristic for avg 2.8x speedup for int8 models (#11698)
rasmith committed almost 2 years ago
[TPU][Quantization] TPU `W8A8` (#11785)
Robert Shaw committed almost 2 years ago
[Misc]add some explanations for BlockHashType (#11847)
WangErXiao committed almost 2 years ago
[Doc] Expand Multimodal API Reference (#11852)
Cyrus Leung committed almost 2 years ago
[Misc] Move some model utils into vision file (#11848)
Cyrus Leung committed almost 2 years ago
[Bugfix][XPU] fix silu_and_mul (#11823)
Yan Ma committed almost 2 years ago
[CI/Build][Bugfix] Fix CPU CI image clean up (#11836)
Li, Jiang committed almost 2 years ago
[Doc][4/N] Reorganize API Reference (#11843)
Cyrus Leung committed almost 2 years ago
[Doc] Move examples into categories (#11840)
Harry Mellor committed almost 2 years ago
[VLM] Reorganize profiling/processing-related code (#11812)
Cyrus Leung committed almost 2 years ago
[torch.compile] consider relevant code in compilation cache (#11614)
youkaichao committed almost 2 years ago
[Hardware][Apple] Native support for macOS Apple Silicon (#11696)
Wallas Henrique committed almost 2 years ago
[Docs] Update sponsor name: 'Novita' to 'Novita AI' (#11833)
Simon Mo committed almost 2 years ago
[doc] update wheels url (#11830)
youkaichao committed almost 2 years ago
[misc] improve memory profiling (#11809)
youkaichao committed almost 2 years ago
[Bug] Fix pickling of `ModelConfig` when RunAI Model Streamer is used (#11825)
Cyrus Leung committed almost 2 years ago
[Docs] reorganize sponsorship page (#11639)
Simon Mo committed almost 2 years ago
[Bugfix] Add checks for LoRA and CPU offload (#11810)
Jee Jee Li committed almost 2 years ago