A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 23:45 UTC
main
[DOC] Add fuse_minimax_qk_norm (#39782)
Jee Jee Li committed 7 months ago
[Refactor] Drop direct dependency on librosa (#39079)
Nick Cao committed 7 months ago
[ZenCPU] AMD Zen CPU Backend with supported dtypes via zentorch weekly (#39967)
Chinmay-Kulkarni-AMD committed 7 months ago
[Bugfix] Fix k_proj's bias for GLM-ASR (#40160)
Rishapveer Singh committed 7 months ago
[Doc] Fix outdated source reference comment in anthropic/serving.py (#40189)
z1ying committed 7 months ago
[Frontend] Add multimodal support to /inference/v1/generate endpoint (#38405)
Nithin Chalapathi committed 7 months ago
[Doc] Add Realtime Transcription section to supported_models.md (#39845)
z1ying committed 7 months ago
[Core] Reduce mm scheduler, get_num_embed overhead (#40143)
milesial committed 7 months ago
[XPU] fix all_reduce all-zero accuracy issue under torch.compile (#39844)
Chaojun Zhang committed 7 months ago
[CI] Speed up test_fused_marlin_moe (#40178)
Michael Goin committed 7 months ago
[XPU]fake impl for xpu fp8_gemm (#39984)
Xinyu Chen committed 7 months ago
Remove outdated tests test_mixtral_moe and test_duplicated_ignored_sequence_group (#40175)
Michael Goin committed 7 months ago
[Kernel] Add MXFP4 W4A4 CUTLASS MoE kernel for SM100 (#37463)
Michael Goin committed 7 months ago
[Kernel] [Helion] Force disable HOP path due to performance regression (#40171)
Yanan Cao committed 7 months ago
[ROCm] Fix TurboQuant on ROCm: backend routing, flash-attn compat, int64 overflow (#39953)
aditi-amd committed 7 months ago
[AMD][CI] Update DeepEP branch (#38396)
Ryan Rock committed 7 months ago
[ROCm][CI] Build fastsafetensors from source so it links against libamdhip64 (#39978)
Andreas Karatzas committed 7 months ago
Xinyu Chen committed 7 months ago
[Misc][UX] Map mimo reasoning and tooling parsers (#40089)
Roger Wang committed 7 months ago
feat: Add LoRA support for Gemma4ForConditionalGeneration (#39291)
allgather committed 7 months ago