A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[docs] promo pytorch conf and ray summit (#24562)
Simon Mo committed about 1 year ago
[Docs] Enable relative links in examples to function when rendered in the docs (#24041)
Harry Mellor committed about 1 year ago
[Bugfix] Update Run:AI Model Streamer Loading Integration (#23845)
pwschuurman committed about 1 year ago
[Core] Simplify and unify mm uuid handling & auto-generated mm hash overrides processing. (#24271)
Chenheli Hua committed about 1 year ago
[gpt-oss] Cache permute indices for faster MXFP4 MoE layer loading (#24154)
Wei committed about 1 year ago
Wenlong Wang committed about 1 year ago
[P/D] MultiConnector supports shutdown (#24425)
Chauncey committed about 1 year ago
[KV Connector] More async support for `get_num_new_matched_tokens` (#23620)
Yihua Cheng committed about 1 year ago
[CI] Adjust threshold for flaky ngram spec decoding test (#24528)
Nick Hill committed about 1 year ago
[BugFix] Fix async core engine client finalizer (#24540)
Nick Hill committed about 1 year ago
[Hardware][Apple-CPU] Enable native bfloat16 on Apple Silicon (M2 and later) (#24129)
Ignacio Sica committed about 1 year ago
[CI] Retry flaky fp8 cutlass mla tests (#24536)
Nick Hill committed about 1 year ago
Add @heheda12345 to CODEOWNERS of KVCacheManager related code (#24546)
Chen Zhang committed about 1 year ago
[Perf] Convert np array to torch tensor to index into block table for attn chunking (#24474)
Yong Hoon Shin committed about 1 year ago
[Bugfix] Fix for 24530. Fix naive all2all shared expert overlap. (#24538)
bnellnm committed about 1 year ago
[Bugfix] Improve EPLB config validation error message (#24524)
Tyler Michael Smith committed about 1 year ago
[ROCm][Feature] Enable Pipeline Parallelism with Ray Compiled Graph on ROCm (#24275)
Charlie Fu committed about 1 year ago
[Benchmark] Add option to skip oversampling in benchmark (#24457)
Ekagra Ranjan committed about 1 year ago
[Benchmark] Update bench doc with mtbench, blazedit, spec bench (#24450)
Ekagra Ranjan committed about 1 year ago
[Log] Use a relative path in debug-level logs to distinguish files with identical names (#23846)
Jiangyun Zhu committed about 1 year ago