A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 08:45 UTC
main
[release] Minor fixes to release annotation and wheel upload (#33129)
Kevin H. Luu committed 9 months ago
Add Triton fused MoE config for B200 (Nemotron Nano) (#32804)
danisereb committed 9 months ago
[Bugfix][Kernel] Fix negative memory offset in GDN Triton kernel (#33326)
CarstyYou committed 9 months ago
[NVIDIA] [feat] Integrate flashinfer Trtllmgen bf16 moe (#32954)
Linda committed 9 months ago
[BUGFIX][XPU] fix memory check after XPU reuse GPU_worker (#33358)
Chendi.Xue committed 9 months ago
[Chore] Move `MediaConnector` to `vllm.multimodal.media` (#33324)
Cyrus Leung committed 9 months ago
[ez] Delete torch25_custom_graph_pass (#33287)
Angela Yi committed 9 months ago
[Bugfix] Fix broken GLM-OCR initialization (#33350)
Isotr0py committed 9 months ago
[Multimodal] Simplify MM input definitions (#33331)
Cyrus Leung committed 9 months ago
[Backport] [Kimi-K2.5] Replace torch.cuda with current_platform for d… (#33320)
sthWrong committed 9 months ago
[Intel GPU] refine xpu worker (#32894)
Kunshang Ji committed 9 months ago
Roger Wang committed 9 months ago
[Bugfix][CPU] Fix thread num for shared memory communication (#33317)
Li, Jiang committed 9 months ago
[Voxtral] Streaming example (#33042)
Patrick von Platen committed 9 months ago
[Quantization][Refactor] use platform dict to choose kernel (#33130)
zofia committed 9 months ago
[Bug Fix] Handle variable-length tensors in MultiModalFlatField batching (#31751)
andrii.pasternak committed 9 months ago
[BugFix] Async Eplb fix potential race condition (#32881)
Ilya Markov committed 9 months ago
[fix] tesdt mcp_tool_calling_streaming with a more complex math question (#32769)
daniel-salib committed 9 months ago
[Chore] Remove `use_data_parallel` kwargs from ViT implementation (#33310)
Isotr0py committed 9 months ago
[Misc] Cleanup Kimi-K2.5's vision chunk modality entrypoints (#33157)
Isotr0py committed 9 months ago