A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 11, 01:15 UTC
main
[Model] Fix bagel failed to run (#31132)
Li Wang committed 11 months ago
Update MiniMax-M2 ToolCall and add MiniMax-M2.1 in Docs (#31083)
Roger Young committed 11 months ago
[Bugfix][ROCm] Fix typo: is_linear_fp8_enaled -> is_linear_fp8_enabled (#31109)
Kevin McKay committed 11 months ago
[Misc] Fix typo: 'occured' -> 'occurred' (#31120)
Kevin McKay committed 11 months ago
[Misc] Fix spelling typos in model comments (#31117)
Kevin McKay committed 11 months ago
[Misc] Fix grammar errors in comments and messages (#31115)
Kevin McKay committed 11 months ago
[Misc] Fix quantization-related typos (#31116)
Kevin McKay committed 11 months ago
[Misc] Fix spelling typos in comments (#31114)
Kevin McKay committed 11 months ago
CedricHuang committed 11 months ago
[Kernel] Enable fused_qknorm_rope_kernel supports partial rope (#30821)
Jee Jee Li committed 11 months ago
[CI] Fix "2 Node Tests (4 GPUs in total)" (#31090)
Lucas Wilkinson committed 11 months ago
[Doc] Clarify FP8 KV cache computation workflow (#31071)
Steve Westerhouse committed 11 months ago
[NVFP4][Perf] Tune NVFP4 input quant kernel for small batch size (#30897)
Michael Goin committed 11 months ago
[MoE Refactor][4/N] Marlin Fp8 Mk (#31036)
Robert Shaw committed 11 months ago
ci: add nvidia-smi warmup before Prime-RL integration test (#31093)
Ameen Patel committed 11 months ago
add aarnphm and chaunceyjiang to the new tool_parser directory (#31088)
Chauncey committed 11 months ago
[BugFix]fix gpt-oss v1/completions response bug (#30608)
汪志鹏 committed 11 months ago
[Quantization] add marlin w4a8/w8a8 check (#31061)
Jinzhen Lin committed 11 months ago
[CI] Fix H200 Distributed test (#31054)
Lucas Wilkinson committed 11 months ago
[Quantization] support logical_widths for fp8 marlin (#30962)
Jinzhen Lin committed 11 months ago