A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 02:00 UTC
main
[Docs] Update FP8 KV Cache documentation (#12238)
Michael Goin committed almost 2 years ago
[Bugfix] Fixing AMD LoRA CI test. (#12329)
Alexei-V-Ivanov-AMD committed almost 2 years ago
[AMD][Quantization] Add TritonScaledMMLinearKernel since int8 is broken for AMD (#12282)
rasmith committed almost 2 years ago
[Frontend][V1] Online serving performance improvements (#12287)
Nick Hill committed almost 2 years ago
[Core] Support `reset_prefix_cache` (#12284)
Cody Yu committed almost 2 years ago
[Bugfix] Fix HPU multiprocessing executor (#12167)
Konrad Zawora committed almost 2 years ago
[Misc] Improve the readability of BNB error messages (#12320)
Jee Jee Li committed almost 2 years ago
[Misc] Fix the error in the tip for the --lora-modules parameter (#12319)
Robin committed almost 2 years ago
[Doc] Add docs for prompt replacement (#12318)
Cyrus Leung committed almost 2 years ago
[Bugfix][VLM] Fix mixed-modality inference backward compatibility for V0 (#12313)
Roger Wang committed almost 2 years ago
[Model][Bugfix]: correct Aria model output (#12309)
zhou fan committed almost 2 years ago
[VLM] Avoid unnecessary tokenization (#12310)
Cyrus Leung committed almost 2 years ago
[Core] Support fully transparent sleep mode (#11743)
youkaichao committed almost 2 years ago
[Build] update requirements of no-device (#12299)
Mengqing Cao committed almost 2 years ago
[core] separate builder init and builder prepare for each batch (#12253)
youkaichao committed almost 2 years ago
[Benchmark] More accurate TPOT calc in `benchmark_serving.py` (#12288)
Nick Hill committed almost 2 years ago
[Ci/Build] Fix mypy errors on main (#12296)
Cyrus Leung committed almost 2 years ago
[CI] add docker volume prune to neuron CI (#12291)
Liangfu Chen committed almost 2 years ago
[ci/lint] Add back default arg for pre-commit (#12279)
Kevin H. Luu committed almost 2 years ago
[VLM] Simplify post-processing of replacement info (#12269)
Cyrus Leung committed almost 2 years ago