A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 10:15 UTC
main
[Bugfix] Fix Typo from NVFP4 Refactor (#31977)
Robert Shaw committed 10 months ago
[Feature] Add iteration level logging and enhance nvtx marker (#31193)
Max Hu committed 10 months ago
[BugFix] Add spec-decode-incompatible request param validation (#31982)
Nick Hill committed 10 months ago
[Quantization] Deprecate Long Tail of Schemes (#31688)
Robert Shaw committed 10 months ago
Yongye Zhu committed 10 months ago
[Misc] Fix `Current vLLM config is not set.` warnings, assert to avoid issues in the future (#31747)
Lucas Wilkinson committed 10 months ago
[Compressed-Tensors] Simplify NVFP4 Conditions, enable marlin support for NVFP4A16 MoEs (#30881)
Dipika Sikka committed 10 months ago
[Misc][Refactor] Add FusedMoERouter object (#30519)
bnellnm committed 10 months ago
[Frontend] Improve error message (#31987)
Cyrus Leung committed 10 months ago
[Documentation][torch.compile] Add documentation for torch.compile + multimodal encoders (#31627)
Lucas Kabela committed 10 months ago
Revert "feat(moe): Add is_act_and_mul=False support for Triton MoE kernels" (#31978)
Michael Goin committed 10 months ago
[Model Runner V2] Simplify BlockTables with UVA (#31965)
Woosuk Kwon committed 10 months ago
[CI][ROCm] Fix NIXL tests on ROCm (#31728)
Nicolò Lucchesi committed 10 months ago
Fix ijson build for Power. (#31702)
Nishidha Panpaliya committed 10 months ago
[Misc] Tidy up some spec decode logic in GPUModelRunner (#31591)
Nick Hill committed 10 months ago
[Doc] Improve MM models LoRA notes (#31979)
Jee Jee Li committed 10 months ago
[Bugfix] Fix vllm serve failure with Nemotron Nano V3 FP8 (#31960)
danisereb committed 10 months ago
[Bugfix]: Fix Step3ReasoningParser missing is_reasoning_end_streaming (#31969)
Chauncey committed 10 months ago
[Model] Support IQuestCoder model (#31575)
yxing-bj committed 10 months ago
[Docs]: update claude code url (#31971)
Chauncey committed 10 months ago