A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 3, 05:30 UTC
main
Enable mermaid diagrams in the docs (#43192)
Harry Mellor committed 6 months ago
Chris Leonard committed 6 months ago
[Docs][PD][NIXL] Bidirectional kv-cache transfer (#43097)
Nicolò Lucchesi committed 6 months ago
[Docs][PD][NIXL] Lease extension mechanism for blocks on P (#43099)
Nicolò Lucchesi committed 6 months ago
[ci] Revert model executor test back to L4 (#43188)
Kevin H. Luu committed 6 months ago
[KV Offload] Pass `OffloadingSpec` instead of `VllmConfig` to secondary tiers (#43076)
Ronen Schaffer committed 6 months ago
[CI] Add DSV4-Flash to gsm8k moe-refactor/config-b200.txt (#42111)
Michael Goin committed 6 months ago
[Perf] Avoid forward scan for async output placeholders (#42938)
Izik Golan committed 6 months ago
add enqueue all option to throughput benchmark (#42975)
pmaybank committed 6 months ago
[Perf][Bugfix] Update dflash aux layer indexing (#40727)
Benjamin Chislett committed 6 months ago
[Cohere] Enable Cohere MoE (#43143)
Terrence Zhao committed 6 months ago
[MRV2][BugFix] Fix default-stream CG capture in P/W LoRA case (#43160)
Nick Hill committed 6 months ago
[bug] fix WeightTransferConfig.backend to allow for all strings (#43121)
Aaron Hao committed 6 months ago
[CPU][DOC] Fix installation commands for Arm CPUs (#43115)
Fadi Arafeh committed 6 months ago
Fix error in Dynamic NTK scaling (#41277)
Max de Bayser committed 6 months ago
[Model] Support post-norm architecture for EAGLE-3 supeculators (#42764)
Doğaç Eldenk committed 6 months ago
[ci] Move language models tests (hybrid) back to L4 (#43129)
Kevin H. Luu committed 6 months ago
[CI failure] Temporarily disable using persistent cache for flashinfer autotune (#43119)
Wei Zhao committed 6 months ago
[Bugfix][MoE] FlashInfer one-sided: workspace union across heterogeneous layers (#42976)
tomeras91 committed 6 months ago
[Docs] Fix MooncakeStoreConnector role in disaggregated example (#42994)
Dao007forever committed 6 months ago