A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 8, 20:15 UTC
main
[ROCm][Bugfix][CI] Fix hybrid models and their tests (Mamba/Jamba/Bamba) (#32710)
Andreas Karatzas committed 9 months ago
[Refactor] Move `task` outside of `PoolingParams.verify` (#33796)
Cyrus Leung committed 9 months ago
[Bugfix] Kimi-K2 grouped_topk usage for Flashinfer monolithic kernels. (#33858)
Pavani Majety committed 9 months ago
[KV Connector][Metrics] Do not count local prefix cache hits in connector queries (#30522)
Mark McLoughlin committed 9 months ago
[Perf] Optimize the performance of structured output + reasoning (#33557)
Chauncey committed 9 months ago
[CI/Build] Fix CPU CI test case title (#33870)
Li, Jiang committed 9 months ago
[CPU][BugFix] Allow w8a8 oneDNN quantized matmul to support 3D inputs (#33727)
Fadi Arafeh committed 9 months ago
[Bugfix] Fix ScoreMultiModalParam multi-document scoring returning single result (#33837)
Andreas Karatzas committed 9 months ago
[CI/Build] Parallelize CPU CI tests (#33778)
Li, Jiang committed 9 months ago
[2/N] move responses/serving _make_response_output_items logic to parser (#33281)
Andrew Xia committed 9 months ago
rasmith committed 9 months ago
[docs] fix unintentional misspellings (#33863)
rinbaro committed 9 months ago
[Minor] Include `StreamingInput` in inputs package (#33856)
Nick Hill committed 9 months ago
Revert "[Attention][FA3] Update FA3 to include new swizzle optimization" (#33841)
Luka Govedič committed 9 months ago
[CI][Bugfix]: return McpCall for built-in MCP tools in non-streaming mode (#32762)
Andreas Karatzas committed 9 months ago
[release] Minor fixes to release annotation (#33849)
Kevin H. Luu committed 9 months ago
[Bugfix] fix DeepSeek R1 with CUTLASS MLA Broken on B200 (#33637)
Chauncey committed 9 months ago
[Bugfix] Disable TRTLLM attention when KV transfer is enabled (#33192)
zhanqiuhu committed 9 months ago
[CI][torch.compile] Reduce e2e fusion test time (#33293)
Luka Govedič committed 9 months ago
feat: Add ColBERT late interaction model support (#33686)
Ilya Boytsov committed 9 months ago