A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 11:00 UTC
main
Removed unused marlin cuda code (#17684)
Michael Goin committed over 1 year ago
Chih-Chieh Yang committed over 1 year ago
Add logging for torch nightly version (#17669)
Yang Wang committed over 1 year ago
[ROCm] fix num_stages for default moe config to avoid triton OutOfResource error (#17744)
Hongxia Yang committed over 1 year ago
Fix doc build performance (#17748)
Harry Mellor committed over 1 year ago
[Kernel] Unified Triton kernel that doesn't distinguish between prefill + decode (#16828)
Thomas Parnell committed over 1 year ago
[Bugfix] Fix for the condition to accept empty encoder inputs for mllama (#17732)
Gregory Shtrasberg committed over 1 year ago
[Misc] Add Next Edit Prediction (NEP) datasets support in `benchmark_serving.py` (#16839)
d.transposed committed over 1 year ago
[TPU] Increase block size and reset block shapes (#16458)
Jevin Jiang committed over 1 year ago
Make right sidebar more readable in "Supported Models" (#17723)
Harry Mellor committed over 1 year ago
[Bugfix] Fix modality limits in vision language example (#17721)
Cyrus Leung committed over 1 year ago
[doc] Add RAG Integration example (#17692)
Reid committed over 1 year ago
[v1] Introduce KVCacheBlocks as interface between Scheduler and KVCacheManager (#17479)
Chen Zhang committed over 1 year ago
[Docs] Use gh-file to add links to tool_calling.md (#17709)
Michael Yao committed over 1 year ago
[Feat] Add deprecated=True to CLI args (#17426)
Aaron Pham committed over 1 year ago
[v1] AttentionMetadata for each layer (#17394)
Chen Zhang committed over 1 year ago
[V1][PP] Support PP for MultiprocExecutor (#14219)
Li, Jiang committed over 1 year ago
[V1] Enable TPU V1 backend by default (#17673)
Michael Goin committed over 1 year ago
[Bugfix] Fix triton import with local TritonPlaceholder (#17446)
Mengqing Cao committed over 1 year ago
Fix `dockerfilegraph` pre-commit hook (#17698)
Harry Mellor committed over 1 year ago