A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 11, 06:15 UTC
main
[CI][Feature] Adds auto-rebase PR rule (#30875)
Rafael Vasquez committed 11 months ago
[v1] Add PrefixLM support to TritonAttention backend (#30386)
Isotr0py committed 11 months ago
feat(api): Eager chat template warmup to eliminate first-request latency (#30700)
Nathan Price committed 11 months ago
[BugFix] Workspace allocation during profile run : DeepEPHighThroughput + DeepGEMM (#30899)
Varun Sundar Rabindranath committed 11 months ago
2.9.1 PyTorch release update (#28495)
Andrey Talman committed 11 months ago
[docs]: add ecosystem projects sr in docs/governance (#30844)
Xunzhuo committed 11 months ago
[Attention] Update tests to remove deprecated env vars (#30563)
Matthew Bonanni committed 11 months ago
[CI][Bugfix] Fix flaky `tests/entrypoints/openai/test_audio.py::test_chat_streaming_audio` (#30878)
Nicolò Lucchesi committed 11 months ago
[Chore] Factor out logic for requesting initial memory (#30868)
Cyrus Leung committed 11 months ago
[Model] Gemma3: Support untied word embeddings (#30827)
KimHyemin committed 11 months ago
Adapt the old parameter enable_thinking in chat_template_kwargs (#30852)
高鑫崧 committed 11 months ago
[Docs] fix function name (#30748)
rongfu.leng committed 11 months ago
[Fix] uniform decode batch check (#30747)
Jialin Ouyang committed 11 months ago
chores: adjust the attn register param order (#30688)
Hank_ committed 11 months ago
Harry Mellor committed 11 months ago
[Bugfix] deepseek-V3.2 self.weights_proj has no bias (#30841)
baoqian426 committed 11 months ago
[ci] Sync test areas yaml file with test-pipeline (#30862)
Kevin H. Luu committed 11 months ago
[Refactor] [4/N] Move VLLM_SERVER_DEV endpoints into the serve directory (#30749)
Chauncey committed 11 months ago
[Bug] Fix AttributeError: 'ColumnParallelLinear' object has no attribute `weight_scale_inv` (#30823)
Wentao Ye committed 11 months ago
[compile] Recompile graph module during Dynamo cache loading. (#30743)
Zhengxu Chen committed 11 months ago