A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Bugfix] Allow skipping MoE in NVFP4 (fix for MTP) (#25987)
Benjamin Chislett committed about 1 year ago
[BugFix] Pad input buffers in _dummy_run (#26209)
Varun Sundar Rabindranath committed about 1 year ago
[Docs] Fix broken table in moe_kernel_features doc (#26314)
Varun Sundar Rabindranath committed about 1 year ago
[Benchmark] Enable MM Embedding benchmarks (#26310)
Cyrus Leung committed about 1 year ago
[Bugfix] Fix gemma3 with transformers backend (#23178)
Raushan Turganbay committed about 1 year ago
Add bias handling to CPUFusedMOE kernel (#26289)
Crefeda Rodrigues committed about 1 year ago
7mile committed about 1 year ago
[Misc] auto_tune: kill specific vllm process (#26304)
Karan Goel committed about 1 year ago
Support llama3 eagle3 head with llama4 verifier (#25961)
Rahul Tuli committed about 1 year ago
[CI Bugfix] Make sure TRTLLM attention is available in test_blackwell_moe (#26188)
Michael Goin committed about 1 year ago
[Tests] conftest: Extending VllmRunner and HfRunner to accept token_ids as input (#26295)
Yannick Schnider committed about 1 year ago
[Attention] Remove unused reorder_batch method (#24463)
Matthew Bonanni committed about 1 year ago
[Model] EVS support for nano_nemotron_vl (#26269)
tomeras91 committed about 1 year ago
Fix `DotsOCR` tensor type (#26281)
Chatcharin Sangbutsarakum committed about 1 year ago
[Bugfix] Fix mrope in Transformers Backend (#26087)
Raushan Turganbay committed about 1 year ago
Support expert parallel load balancing in Transformers backend (#26287)
Harry Mellor committed about 1 year ago
[Model] Use `merge_by_field_config` for MM models (Llava family) (#26280)
Cyrus Leung committed about 1 year ago
[Frontend] Consolidate tokenizer init code (#26276)
Cyrus Leung committed about 1 year ago
[Doc] add KAITO to integrations (#25521)
abhisheksheth28 committed about 1 year ago
[Docs] Edit HF Inference Endpoints documentation (#26275)
Aritra Roy Gosthipaty committed about 1 year ago