A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 3, 08:00 UTC
main
[Bugfix][KV Connector] Fix SimpleCPUOffloadScheduler TOCTOU between Phase A and Phase B (#42289)
Qiuyang Yue committed 6 months ago
add cutedsl dsv4 indexer fp8 kernel (#42899)
gnovack committed 6 months ago
[Model Refactoring] Move DeepSeek V4 layers to `models/deepseek_v4/` [2/N] (#43039)
Woosuk Kwon committed 6 months ago
[Bugfix][MRV2] Fix KVCache tensor explicit `kernel_block_size` dim (#42766)
Nicolò Lucchesi committed 6 months ago
[UX] Add a persistent cache for FlashInfer autotuning (#42537)
Mohammad Miadh Angkad committed 6 months ago
[XPU] add gptq(int4) support (#37844)
Kunshang Ji committed 6 months ago
[BugFix][CPU][Spec Decode] Fix Eagle implementation on CPU backend (#42468)
Ofir Zafrir committed 6 months ago
[Bugfix] Use platform-agnostic device in example_connector load (#42926)
Revital Sur committed 6 months ago
[Model Refactoring] Migrate DeepSeek V4 to vllm/models/ [1/N] (#43004)
Woosuk Kwon committed 6 months ago
[Frontend] Add --spec-method/--spec-model/--spec-tokens CLI aliases (#42476)
Michael Goin committed 6 months ago
[CI/Build] Bump nvidia-cutlass-dsl to 4.5.1 (#42991)
Artem Perevedentsev committed 6 months ago
Add parallel drafting to v2 model runner unsupported features (#43010)
shanjiaz committed 6 months ago
Wentao Ye committed 6 months ago
[Refactor] Extract shared coerce_to_schema_type utility from Minimax M2 tool parser (#43006)
Flora Feng committed 6 months ago
[Perf][MLA] Enable FULL cudagraph capture for TRITON_MLA decode (#42885)
haosdent committed 6 months ago
[Refactor] Remove dead code (#42889)
Wentao Ye committed 6 months ago
Tier offload followup (#42529)
Ronen Schaffer committed 6 months ago
[ROCm] Guard AITER GDN decode fast path by layout (#42880)
Tuukka Sarvi committed 6 months ago
[Docs] update attribution to reflect EDEN foundation (#41666)
Amit Portnoy committed 6 months ago
[Refactor] Remove dead cuda kernels (#42767)
Wentao Ye committed 6 months ago