A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 02:15 UTC
main
[Docs] Improve docs for RLHF co-location example (#20599)
Ricardo Decal committed over 1 year ago
[Benchmark] Parameterization of streaming loading of multimodal datasets (#20528)
Li Wang committed over 1 year ago
Cyrus Leung committed over 1 year ago
[V1] [Doc] Update V1 docs for Mamba models (#20499)
Thomas Parnell committed over 1 year ago
Chauncey committed over 1 year ago
[Docs] fix minimax tool_calling docs error (#20667)
qscqesze committed over 1 year ago
[Intel GPU] support ray as distributed executor backend for XPU. (#20659)
Kunshang Ji committed over 1 year ago
[XPU] Use spawn with XPU multiprocessing (#20649)
Dmitry Rogozhkin committed over 1 year ago
[Misc] Fix the size of batched_dummy_mm_inputs in profile_run (#20434)
B-201 committed over 1 year ago
Fix bullets in incremental_build.md (#20642)
Michael Goin committed over 1 year ago
[feat] enable SM100 CUTLASS block scaled group gemm for smaller batch sizes (#20640)
Duncan Moss committed over 1 year ago
[Bugfix] set default set cuda_graph_sizes to min(self.max_num_seqs * 2, 512) (#20628)
zhrrr committed over 1 year ago
[Hardware][PPC64LE] Enable V1 for ppc64le and ARM (#20554)
Akash kaothalkar committed over 1 year ago
[Docs] Improve documentation for multi-node service helper script (#20600)
Ricardo Decal committed over 1 year ago
[BugFix][Intel GPU] Use refactored API for dist_backend in V1 worker (#20596)
Ratnam Parikh committed over 1 year ago
[tech debt] Revisit lora request model checker (#20636)
kourosh hakhamaneshi committed over 1 year ago
[misc]refactor `Platform.set_device` method (#20262)
Kunshang Ji committed over 1 year ago
Wenxin Cheng committed over 1 year ago
[TPU][Bugfix] disable phi-3 test (#20632)
QiliangCui committed over 1 year ago
[Bugfix] Fix GLM-4.1-V video prompt update (#20635)
Isotr0py committed over 1 year ago