A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 7, 04:00 UTC
main
[Model] Add support for OLMo Hybrid (#32550)
Yanhong Li committed 8 months ago
[CI] Add explicit permissions to macOS smoke test workflow (#35775)
Russell Bryant committed 8 months ago
[BugFix] Fallback from FA4->FA2 for Batch Invariance (#36059)
Frank Wang committed 8 months ago
[Perf] Optimize FusedMoEModularKernel output tensor using torch.empty (#35794)
Xin Yang committed 8 months ago
[Bugfix] Disable FlashInfer TRTLLM BF16 path for non-gated MoE (#36146)
tomeras91 committed 8 months ago
ParakeetProjection.norm = RMSNorm instead of nn.LayerNorm (#36133)
Netanel Haber committed 8 months ago
[XPU] Enable ModelRunnerV2 on XPU (#36078)
Xinyu Chen committed 8 months ago
Avery Miao committed 8 months ago
[Bugfix][CI] fix typos (#34934)
Jiayi Yan committed 8 months ago
[ROCm] Refactor ROCm attention backend selection logic (#35246)
Sage Moore committed 8 months ago
refactor funasr model. (#36108)
AllenDou committed 8 months ago
[Bugfix] Fix Qwen-VL tokenizer implementation (#36140)
Cyrus Leung committed 8 months ago
[openai api] log exception in exception handler (1/N) (#31164)
Ning Xie committed 8 months ago
[KVConnector] Scheduler: Fix num_computed_tokens after async KV load (#34616)
Or Ozeri committed 8 months ago
Fix import that was moved in Transformers 5.2.0 (#36120)
Harry Mellor committed 8 months ago
[Docs] Only build docs if `documentation` or `ready` labels are present (#36135)
Harry Mellor committed 8 months ago
[CI] Stabilize test_no_args_tool_call and add ROCm-specific server args (#36107)
Andreas Karatzas committed 8 months ago
[Bugfix] Fix RunAI streamer crash with S3-hosted model paths (#35976)
Ajay Anubolu committed 8 months ago
[Bugfix] Fix mypy errors in hermes_tool_parser.py (#36114)
Shiyan Deng committed 8 months ago
[Misc] Fix SyntaxWarning - invalid escape sequence '\e' (#36020)
cjackal committed 8 months ago