A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 18:30 UTC
main
[Doc] Add vLLM Beijing meetup slide (#14938)
Chen Zhang committed over 1 year ago
[Misc][XPU] Use None as device capacity for XPU (#14932)
Yan Ma committed over 1 year ago
[V1] Remove input cache client (#14864)
Cyrus Leung committed over 1 year ago
[V1] [Spec Decode] Support random sampling for spec decode (#13933)
Lily Liu committed over 1 year ago
[Benchmark] Do not save detailed info to json by default (#14879)
Simon Mo committed over 1 year ago
[Misc] Replace os environ to monkeypatch in test suite (#14516)
Sibi committed over 1 year ago
[BugFix] Fix MLA + V1 + TP==1 causing reinitialization of cuda context (#14910)
Lucas Wilkinson committed over 1 year ago
[V1][Minor] Add __repr__ to ConstantList (#14907)
Woosuk Kwon committed over 1 year ago
[V1] Optimize the overhead of rewinding (#14905)
Woosuk Kwon committed over 1 year ago
[CI/Build] Update defaults for test reproducibility (#14893)
Cyrus Leung committed over 1 year ago
Robert Shaw committed over 1 year ago
[V1] Enable Entrypoints Tests (#14903)
Robert Shaw committed over 1 year ago
[Doc] Add guidance for using `ccache` with `pip install -e .` in doc (#14901)
Vadim Gimpelson committed over 1 year ago
[Misc] Catching Ray Compiled Graph PP test failures for V1 (#14847)
Rui Qiao committed over 1 year ago
[V1][BugFix] Detect interleaved sliding window attention (#14896)
Woosuk Kwon committed over 1 year ago
[BugFix][V1] Fix overhead related to bad_words sampling when not in use (#14894)
Nick Hill committed over 1 year ago
Revert "[Bugfix] Limit profiling run sequence length by max_model_len (#14785) (#14892)
Cyrus Leung committed over 1 year ago
[Bugfix] Explicitly disable Phi-4-multimodal in V1 (#14889)
Cyrus Leung committed over 1 year ago
[Bugfix] Limit profiling run sequence length by max_model_len (#14785)
Kyle Sayers committed over 1 year ago
[V1] [Spec Decode] Fix ngram tests (#14878)
Lily Liu committed over 1 year ago