A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 15:45 UTC
main
[doc] update doc on testing and debugging (#8514)
youkaichao committed about 2 years ago
[Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270)
Luka Govedič committed about 2 years ago
[Bugfix][Kernel] Fix build for sm_60 in GGUF kernel (#8506)
sasha0552 committed about 2 years ago
[BugFix] Fix clean shutdown issues (#8492)
Nick Hill committed about 2 years ago
[Frontend] Expose revision arg in OpenAI server (#8501)
lewtun committed about 2 years ago
[Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032)
ElizaWszola committed about 2 years ago
[Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357)
Isotr0py committed about 2 years ago
[Bugfix][Model] Fix Python 3.8 compatibility in Pixtral model by updating type annotations (#8490)
Chris committed about 2 years ago
[TPU] Implement multi-step scheduling (#8489)
Woosuk Kwon committed about 2 years ago
[torch.compile] add a flag to disable custom op (#8488)
youkaichao committed about 2 years ago
[torch.compile] fix functionalization (#8480)
youkaichao committed about 2 years ago
[Model] support minicpm3 (#8297)
ywfang committed about 2 years ago
[Kernel][Hardware][Amd]Custom paged attention kernel for rocm (#8310)
Charlie Fu committed about 2 years ago
[Hardware][intel GPU] bump up ipex version to 2.3 (#8365)
Kunshang Ji committed about 2 years ago
bump version to v0.6.1.post2 (#8473)
Simon Mo committed about 2 years ago
[HotFix] Fix final output truncation with stop string + streaming (#8468)
Nick Hill committed about 2 years ago
[Doc] Add oneDNN installation to CPU backend documentation (#8467)
Isotr0py committed about 2 years ago
[CI/Build] Reorganize models tests (#7820)
Cyrus Leung committed about 2 years ago
[plugin][torch.compile] allow to add custom compile backend (#8445)
youkaichao committed about 2 years ago
[Installation] Gate FastAPI version for Python 3.8 (#8456)
Cyrus Leung committed about 2 years ago