A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 01:45 UTC
main
[Core][Distributed] enable multiple tp group (#4512)
youkaichao committed over 2 years ago
[mypy][6/N] Fix all the core subdirectory typing (#4450)
SangBin Cho committed over 2 years ago
[CI]Add regression tests to ensure the async engine generates metrics (#4524)
Ronen Schaffer committed over 2 years ago
SangBin Cho committed over 2 years ago
[MISC] Rework logger to enable pythonic custom logging configuration to be provided (#4273)
Danny Guinther committed over 2 years ago
[Misc] Fix expert_ids shape in MoE (#4517)
Woosuk Kwon committed over 2 years ago
[Misc] Remove Mixtral device="cuda" declarations (#4543)
Philipp Moritz committed over 2 years ago
[Core][Distributed] fix pynccl del error (#4508)
youkaichao committed over 2 years ago
[Bugfix][Core] Fix and refactor logging stats (#4336)
Roy committed over 2 years ago
[Bugfix] Add validation for seed (#4529)
sasha0552 committed over 2 years ago
[Kernel] Update fused_moe tuning script for FP8 (#4457)
Philipp Moritz committed over 2 years ago
[Core] Add `multiproc_worker_utils` for multiprocessing-based workers (#4357)
Nick Hill committed over 2 years ago
[Core] Enable prefix caching with block manager v2 enabled (#4142)
leiwen83 committed over 2 years ago
[Speculative decoding] Add ngram prompt lookup decoding (#4237)
leiwen83 committed over 2 years ago
[CI/Build][Bugfix] VLLM_USE_PRECOMPILED should skip compilation (#4534)
Travis Johnson committed over 2 years ago
[Bugfix] Use random seed if seed is -1 (#4531)
sasha0552 committed over 2 years ago
[Doc] update(example model): for OpenAI compatible serving (#4503)
Frαnçois committed over 2 years ago
[Bugfix] Fix 307 Redirect for `/metrics` (#4523)
Robert Shaw committed over 2 years ago
AnyISalIn committed over 2 years ago
[Test] Add ignore_eos test (#4519)
SangBin Cho committed over 2 years ago