A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 7, 14:00 UTC
main
[ROCm] Enabling encoder and encoder-decoder on ROCm and AITER unified backends (#35334)
Gregory Shtrasberg committed 8 months ago
[Feat][RL][2/2] Native Weight Syncing API: IPC (#34171)
Aaron Hao committed 8 months ago
[Bugfix][Model] Fix gpt-oss batch invariance (#35404)
Jakub Zakrzewski committed 8 months ago
[DP] Only use DP padding when cudagraphs are actually used (#34102)
Lucas Wilkinson committed 8 months ago
[Bugfix] Add monkeypatch to prevent race condition from writing (#35420)
Lucas Kabela committed 8 months ago
[Transformers backend] Ignore MTP weights when num_nextn_predict_layers=0 (#34888)
SteadfastAsArt committed 8 months ago
[compile] Fix caching error over pytree slice node. (#35308)
Zhengxu Chen committed 8 months ago
[Model Runner V2] Warmup kernels (#35172)
Nick Hill committed 8 months ago
[BugFix] Fix 3D rope in transformers backend (#35097)
Raushan Turganbay committed 8 months ago
Support parakeet as audio encoder for nemotron-nano-vl (#35100)
Netanel Haber committed 8 months ago
[Doc] Fix link to Llama chat template for usability (#35525)
Martin Hickey committed 8 months ago
[perf] Use pinned memory for async H2D transfer in do_mamba_copy_block (#35480)
Huamin Li committed 8 months ago
[Misc] Fill in some v1 CODEOWNERS gaps (#35524)
Nick Hill committed 8 months ago
[Model] Add huggingface skt/A.X-K1 model (#32407)
fort726 committed 8 months ago
Yanan Cao committed 8 months ago
[Core] Fix `gpu_worker.py` pre-commit errors (#35312)
Nick Hill committed 8 months ago
Add @BoyuanFeng to CODEOWNERS (#35317)
Boyuan Feng committed 8 months ago
[Bugfix] Handle case when kimi ends reasoning with a tool call (#33646)
Koushik Dutta committed 8 months ago
Yueqian Lin committed 8 months ago
[Bugfix] Fix DCP + FA3 crash due to missing num_splits in _forward_with_dcp (#35082)
haosdent committed 8 months ago