A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 00:30 UTC
main
[BUGFIX] [FRONTEND] Correct chat logprobs (#5029)
Breno Faria committed over 2 years ago
[CI/Build] Docker cleanup functionality for amd servers (#5112)
omkar kakarparthi committed over 2 years ago
[Bugfix] gptq_marlin: Ensure g_idx_sort_indices is not a Parameter (#5108)
Alexander Matveev committed over 2 years ago
[Doc][Build] update after removing vllm-nccl (#5103)
youkaichao committed over 2 years ago
[Bugfix] logprobs is not compatible with the OpenAI spec #4795 (#5031)
Itay Etelis committed over 2 years ago
[Core] Avoid the need to pass `None` values to `Sequence.inputs` (#5099)
Cyrus Leung committed over 2 years ago
[Bugfix][CI/Build] Fix codespell failing to skip files in `git diff` (#5097)
Cyrus Leung committed over 2 years ago
[Bugfix][CI/Build] Fix test and improve code for `merge_async_iterators` (#5096)
Cyrus Leung committed over 2 years ago
[Doc]Replace deprecated flag in readme (#4526)
Ronen Schaffer committed over 2 years ago
afeldman-nm committed over 2 years ago
[Core][Distributed] improve p2p access check (#4992)
youkaichao committed over 2 years ago
[Bugfix] Fix arguments passed to `Sequence` in stop checker test (#5092)
Cyrus Leung committed over 2 years ago
[Core][Optimization] remove vllm-nccl (#5091)
youkaichao committed over 2 years ago
[Misc] add gpu_memory_utilization arg (#5079)
Marut Pandya committed over 2 years ago
[Bugfix] Remove the last EOS token unless explicitly specified (#5077)
Junichi Sato committed over 2 years ago
[Core] Consolidate prompt arguments to LLM engines (#4328)
Cyrus Leung committed over 2 years ago
[Docs] Add Dropbox as sponsors (#5089)
Simon Mo committed over 2 years ago
[Kernel][ROCm][AMD] Add fused_moe Triton configs for MI300X (#4951)
Divakar Verma committed over 2 years ago
[BugFix] Fix Embedding Models with TP>1 (#5075)
Robert Shaw committed over 2 years ago
[Core] Sliding window for block manager v2 (#4545)
Michał Moskal committed over 2 years ago