A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 09:00 UTC
main
[Misc] Enable async scheduling by default with spec decoding (#31998)
Nick Hill committed 10 months ago
[perf][async] support non cpu sync get logprob tensors for spec (#31336)
zhrrr committed 10 months ago
[NIXL] refine decoder side post process for heterogeneous BlockSize and kv_layout (#30275)
Chendi.Xue committed 10 months ago
[1/N][Attention] Restructure attention: move files (#31916)
Matthew Bonanni committed 10 months ago
[responsesAPI] fix incomplete_messages for simple/parsable context (#31836)
Andrew Xia committed 10 months ago
[Quant] Make static quant support all group shapes (#30833)
Lucas Wilkinson committed 10 months ago
[fix] add cutedsl to global sf (#32001)
jiahanc committed 10 months ago
Add unpermute-aware fused MoE path and small-batch fallback (#29354)
Runkai Tao committed 10 months ago
[Fix] Introduce audio channels spec (#31595)
Jeremy Teboul committed 10 months ago
Wentao Ye committed 10 months ago
[Refactor] Remove numpy split in async scheduling (#32034)
Wentao Ye committed 10 months ago
[Frontend][gpt-oss] Allow system message to overwrite model identity (#31737)
Andrew Xia committed 10 months ago
[Feat][Core] Support multiple KV cache groups in Hybrid KV Coordinator (#31707)
Yifan Qiao committed 10 months ago
[UX] Add vLLM model inspection view (#29450)
Michael Goin committed 10 months ago
[Doc] Add developer guide for CustomOp (#30886)
Shanshan Shen committed 10 months ago
Rename --exclude-log-deltas to --enable-log-deltas (#32020)
Kevin Šuc committed 10 months ago
[Doc] Remove hardcoded Whisper in example openai translation client (#32027)
Isotr0py committed 10 months ago
[Perf][Kernel] Fused SiLU+Mul+Quant kernel for NVFP4 cutlass_moe (#31832)
Michael Goin committed 10 months ago
[CPU] Add head sizes 80 and 112 with vec16 fallback (#31968)
R3hankhan committed 10 months ago
[Model] Remove redundant None check in DeepSeekOCR image input processing (#32016)
maang committed 10 months ago