A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 20:45 UTC
main
[bugfix][distributed] fix multi-node bug for shared memory (#6597)
youkaichao committed over 2 years ago
[Bugfix] [SpecDecode] AsyncMetricsCollector: update time since last collection (#6578)
Thomas Parnell committed over 2 years ago
[Docs] Update docs for wheel location (#6580)
Simon Mo committed over 2 years ago
[Misc] Small perf improvements (#6520)
Antoni Baum committed over 2 years ago
[Bugfix][Frontend] remove duplicate init logger (#6581)
Daniele committed over 2 years ago
[Model] RowParallelLinear: pass bias to quant_method.apply (#6327)
Thomas Parnell committed over 2 years ago
[BUGFIX] Raise an error for no draft token case when draft_tp>1 (#6369)
Woo-Yeon Lee committed over 2 years ago
[Bugfix][Frontend] Fix missing `/metrics` endpoint (#6463)
Cyrus Leung committed over 2 years ago
[ Misc ] non-uniform quantization via `compressed-tensors` for `Llama` (#6515)
Robert Shaw committed over 2 years ago
[Bugfix] Make spec. decode respect per-request seed. (#6034)
Thomas Parnell committed over 2 years ago
[Core] Multiprocessing Pipeline Parallel support (#6130)
Nick Hill committed over 2 years ago
Add support for a rope extension method (#6553)
Simon Mo committed over 2 years ago
[CI/Build] Build on Ubuntu 20.04 instead of 22.04 (#6517)
Tyler Michael Smith committed over 2 years ago
[Kernel] Implement fallback for FP8 channelwise using torch._scaled_mm (#6552)
Tyler Michael Smith committed over 2 years ago
[ci][test] add correctness test for cpu offloading (#6549)
youkaichao committed over 2 years ago
Kevin H. Luu committed over 2 years ago
[Model] Support Mistral-Nemo (#6548)
Michael Goin committed over 2 years ago
[ci] Reword Github bot comment (#6534)
Kevin H. Luu committed over 2 years ago
[ Misc ] Improve Min Capability Checking in `compressed-tensors` (#6522)
Robert Shaw committed over 2 years ago
[TPU] Refactor TPU worker & model runner (#6506)
Woosuk Kwon committed over 2 years ago