A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 00:30 UTC
main
[BugFix] Reordering extend logic fix (#27739)
Lucas Wilkinson committed about 1 year ago
kernels/moe test pruning (#27053)
Fardin Hoque committed about 1 year ago
[Feat] Adds runai distributed streamer (#27230)
Benjamin Bartels committed about 1 year ago
[BugFix] Handle unscheduled requests properly when async scheduling (#27756)
Nick Hill committed about 1 year ago
[XPU] Update latest IPEX 2.8 release (#27735)
Kunshang Ji committed about 1 year ago
[Bugfix] mamba-block-size is set for vision language model (#27773)
Chen Zhang committed about 1 year ago
[XPU][bugfix] fix rope for llama4 and deepseek (#25145)
Yan Ma committed about 1 year ago
[Temp fix] Disable torch.compile for Qwen2.5 VL's VisionBlock temporarily. (#27760)
Chenheli Hua committed about 1 year ago
[Bug] Fix DBO IMA issue for DeepEPHT (#27666)
Wentao Ye committed about 1 year ago
[BugFix] Fix handling of resumed reqs in `SharedStorageConnector` (#27719)
Nick Hill committed about 1 year ago
Wentao Ye committed about 1 year ago
[KVConnector] Add metrics to Prometheus-Grafana dashboard (#26811)
Nicolò Lucchesi committed about 1 year ago
[Bug] Raise error explicitly if using incompatible backend (#27424)
Wentao Ye committed about 1 year ago
[CI/Build] Test torchrun with 8 cards (#27548)
22quinn committed about 1 year ago
`use_aot_compile` should respect `VLLM_DISABLE_COMPILE_CACHE` (#27698)
Boyuan Feng committed about 1 year ago
[FIXBUG] Qwen3VL hallucinations without Contiguous on Torch.SDPA (#27744)
JartX committed about 1 year ago
[Core] Exposing engine sleep & wake_up state as prometheus metrics (#24176)
Braulio Dumba committed about 1 year ago
[Chore] Optimize P2PNCCLEngine `http_address` (#27488)
Wentao Ye committed about 1 year ago
[CI] Fix flaky `test_two_responses_with_same_prev_id` test (#27745)
Nicolò Lucchesi committed about 1 year ago
[ROCm][Platform] Add MI308X device id in _ROCM_DEVICE_ID_NAME_MAP (#27623)
Xiake Sun committed about 1 year ago