A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 08:15 UTC
main
[Bugfix][SpecDecode] kv corruption with bonus tokens in spec decode (#9730)
Sungjae Lee committed about 2 years ago
[Bugfix] Fix pickle of input when async output processing is on (#9931)
Wallas Henrique committed about 2 years ago
[Bugfix] Properly propagate trust_remote_code settings (#10047)
zifeitong committed about 2 years ago
[bugfix] fix weak ref in piecewise cudagraph and tractable test (#10048)
youkaichao committed about 2 years ago
[CI] Prune back the number of tests in tests/kernels/* (#9932)
Michael Goin committed about 2 years ago
[CI] Prune tests/models/decoder_only/language/* tests (#9940)
Michael Goin committed about 2 years ago
[Misc] Modify BNB parameter name (#9997)
Jee Jee Li committed about 2 years ago
[Feature] Update benchmark_throughput.py to support image input (#9851)
lkchen committed about 2 years ago
[Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921)
Michael Goin committed about 2 years ago
[CI/Build] Limit github CI jobs based on files changed (#9928)
Russell Bryant committed about 2 years ago
[Misc] Add logging for CUDA memory (#10027)
Chenghao (Alan) Yang committed about 2 years ago
Refactor TPU requirements file and pin build dependencies (#10010)
Richard Liu committed about 2 years ago
[Frontend] Fix tcp port reservation for api server (#10012)
Russell Bryant committed about 2 years ago
[Misc] vllm CLI flags should be ordered for better user readability (#10017)
Chauncey committed about 2 years ago
[BugFix] Lazy import ray (#10021)
Gene Der Su committed about 2 years ago
Tyler Michael Smith committed about 2 years ago
[Core] Make encoder-decoder inputs a nested structure to be more composable (#9604)
Cyrus Leung committed about 2 years ago
[Core] Use os.sched_yield in ShmRingBuffer instead of time.sleep (#9994)
Tyler Michael Smith committed about 2 years ago
[Bugfix] Respect modules_to_not_convert within awq_marlin (#9895)
Michael Goin committed about 2 years ago
[4.5/N] bugfix for quant config in speculative decode (#10007)
youkaichao committed about 2 years ago