A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
main
add gptq_marlin test for bug report https://github.com/vllm-project/vllm/issues/5088 (#5145)
Alexander Matveev committed over 2 years ago
[misc] Do not allow to use lora with chunked prefill. (#5538)
SangBin Cho committed over 2 years ago
[CI/Build] Test both text and token IDs in batched OpenAI Completions API (#5568)
Cyrus Leung committed over 2 years ago
[mypy] Enable type checking for test directory (#5017)
Cyrus Leung committed over 2 years ago
[Core][Bugfix]: fix prefix caching for blockv2 (#5364)
leiwen83 committed over 2 years ago
Simon Mo committed over 2 years ago
[Core][Distributed] improve p2p cache generation (#5528)
youkaichao committed over 2 years ago
[Bugfix] Fix typo in Pallas backend (#5558)
Woosuk Kwon committed over 2 years ago
[Bugfix] Enable loading FP8 checkpoints for gpt_bigcode models (#5460)
Thomas Parnell committed over 2 years ago
[Doc] Update documentation on Tensorizer (#5471)
Sanger Steel committed over 2 years ago
[Docs] Add ZhenFund as a Sponsor (#5548)
Simon Mo committed over 2 years ago
[misc][distributed] fix benign error in `is_in_the_same_node` (#5512)
youkaichao committed over 2 years ago
[Core] Remove duplicate processing in async engine (#5525)
Cyrus Leung committed over 2 years ago
[mis] fix flaky test of test_cuda_device_count_stateless (#5546)
youkaichao committed over 2 years ago
[Kernel] Suppress mma.sp warning on CUDA 12.5 and later (#5401)
Tyler Michael Smith committed over 2 years ago
[ Misc ] Rs/compressed tensors cleanup (#5432)
Robert Shaw committed over 2 years ago
Allen.Dou committed over 2 years ago
[Kernel] Fix CUTLASS 3.x custom broadcast load epilogue (#5516)
Tyler Michael Smith committed over 2 years ago
[CI/Build] Disable LLaVA-NeXT CPU test (#5529)
Cyrus Leung committed over 2 years ago
Kuntai Du committed over 2 years ago