A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
Thomas Parnell committed about 1 year ago
Add `VLLM_NVTX_SCOPES_FOR_PROFILING=1` to enable `nvtx.annotate` scopes (#25501)
Corey Lowman committed about 1 year ago
[BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359)
kourosh hakhamaneshi committed about 1 year ago
Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512)
Michael Goin committed about 1 year ago
[gpt-oss][bugfix] remove logic to require resp_ in ResponseAPI (#25428)
Andrew Xia committed about 1 year ago
Fix triton_reshape_and_cache_flash.py triton import (#25522)
Michael Goin committed about 1 year ago
Add VLLM_ENABLE_INDUCTOR_MAX_AUTOTUNE & VLLM_ENABLE_INDUCTOR_COORDINA… (#25493)
rouchenzi committed about 1 year ago
[V0 Deprecation] Remove placeholder attn (#25510)
Thomas Parnell committed about 1 year ago
[Core] Use KVCacheBlock as much as possible instead of dict[block_id, KVCacheBlock] (#24830)
Jialin Ouyang committed about 1 year ago
[CI/Build] Fix and re-enable v1 PP test on CI (#25496)
Isotr0py committed about 1 year ago
[Bugfix] Fix for the import error from #24588 (#25481)
Gregory Shtrasberg committed about 1 year ago
[Build] Update Xgrammar to 0.1.25 (#25467)
Chauncey committed about 1 year ago
[Bugfix] gpt-oss container tool output bug (#25485)
Alec S committed about 1 year ago
[CI] Fix Pre-commit Issue (#25497)
Wentao Ye committed about 1 year ago
Enable symmetric memory all reduce by default only enabling for TP (#25070)
Ilya Markov committed about 1 year ago
[Bugfix] Lower gpt-oss max cudagraph size to 992 to be compatible with FA3 (#25508)
Michael Goin committed about 1 year ago
[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406)
jiahanc committed about 1 year ago
[Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEWISE (#25444)
Michael Goin committed about 1 year ago
[Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666)
ElizaWszola committed about 1 year ago
[core] add nccl symmetric memory for all reduce (#24532)
Amir Samani committed about 1 year ago