A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 19:30 UTC
main
[Misc] Temporarily resolve the error of BitAndBytes (#7308)
Jee Jee Li committed over 2 years ago
[Bugfix] Fix new Llama3.1 GGUF model loading (#7269)
Isotr0py committed over 2 years ago
[CI/Build] Dockerfile.cpu improvements (#7298)
Daniele committed over 2 years ago
[Doc] Put collect_env issue output in a <detail> block (#7310)
Michael Goin committed over 2 years ago
[Bugfix][fast] Fix the get_num_blocks_touched logic (#6849)
Zach Zheng committed over 2 years ago
[Frontend] Kill the server on engine death (#6594)
Joe Runde committed over 2 years ago
[Bugfix][Kernel] Increased atol to fix failing tests (#7305)
Luka Govedič committed over 2 years ago
[Model] Rename MiniCPMVQwen2 to MiniCPMV2.6 (#7273)
Jee Jee Li committed over 2 years ago
[Bugfix] Fix LoRA with PP (#7292)
Murali Andoorveedu committed over 2 years ago
[Frontend] remove max_num_batched_tokens limit for lora (#7288)
Cherilyn Buren committed over 2 years ago
[Misc] Fix typos in scheduler.py (#7285)
Rui Qiao committed over 2 years ago
[Kernel] Fix Flashinfer Correctness (#7284)
Lily Liu committed over 2 years ago
[Doc] Update supported_hardware.rst (#7276)
Michael Goin committed over 2 years ago
[FrontEnd] Make `merge_async_iterators` `is_cancelled` arg optional (#7282)
Nick Hill committed over 2 years ago
[Bugfix] Fix gptq failure on T4s (#7264)
Lucas Wilkinson committed over 2 years ago
[ci] Make building wheels per commit optional (#7278)
Kevin H. Luu committed over 2 years ago
[Bugfix][FP8] Fix dynamic FP8 Marlin quantization (#7219)
Michael Goin committed over 2 years ago
[BugFix] Fix frontend multiprocessing hang (#7217)
Maximilien de Bayser committed over 2 years ago
[Doc] add online speculative decoding example (#7243)
Stas Bekman committed over 2 years ago
[OpenVINO] migrate to latest dependencies versions (#7251)
Ilya Lavrenov committed over 2 years ago