A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 22:35 UTC
main
fix(setup): improve precompiled wheel setup for Docker builds (#22025)
Doug Smith committed over 1 year ago
[ez] Remove a trailing space from compilation/decorators.py (#22028)
Zhengxu Chen committed over 1 year ago
Song committed over 1 year ago
[NVIDIA] Add SM100 Flashinfer MoE per tensor scale fp8 backend (#21458)
amirkl94 committed over 1 year ago
[BugFix] Fix case where `collective_rpc` returns `None` (#22006)
Nick Hill committed over 1 year ago
[Model][CI] Let more pooling models support v1 (#21747)
wang.yuqi committed over 1 year ago
[CI/Build] get rid of unused VLLM_FA_CMAKE_GPU_ARCHES (#21599)
Daniele committed over 1 year ago
[Deprecation] Remove deprecated args and methods (#21907)
Cyrus Leung committed over 1 year ago
[Bugfix]: fix metadata file copy in test_sharded_state_loader (#21830)
Ning Xie committed over 1 year ago
[CI Bugfix] Fix CI OOM for `test_shared_storage_connector_hashes` (#21973)
Michael Goin committed over 1 year ago
[Misc] Expand SUPPORTED_HIDDEN_SIZES for DeepEP low-latency kernels (#21818)
Jee Jee Li committed over 1 year ago
[UX] Rename CUTLASS_MLA_VLLM_V1 to CUTLASS_MLA (#21966)
Michael Goin committed over 1 year ago
[Bugfix] Relax lang pin for voxtral (#21833)
Sanchit Gandhi committed over 1 year ago
[Example] Add `async_llm_streaming.py` example for AsyncLLM streaming in python (#21763)
Michael Goin committed over 1 year ago
[Core] Move EngineCoreRequest to Request conversion out of EngineCore (#21627)
Zebing Lin committed over 1 year ago
[Bugfix] Fix None value handling in trace span creation for cancelled requests (#20272)
Bram committed over 1 year ago
[Feature] Add async tensor parallelism for scaled mm (#20155)
cascade committed over 1 year ago
[Misc] Use dracut on CentOS and skip clone if repo exists for EP kernel installation (#21635)
Ming Yang committed over 1 year ago
For VLLM_USE_PRECOMPILED, only compiled .so files should be extracted (#21964)
Doug Smith committed over 1 year ago
[Misc] Support more collective_rpc return types (#21845)
Nick Hill committed over 1 year ago