A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 09:30 UTC
main
:bug: Fixup more test failures from memory profiling (#9563)
Joe Runde committed about 2 years ago
[CI] Make format checker error message more user-friendly by using emoji (#9564)
Kuntai Du committed about 2 years ago
[Frontend] Don't log duplicate error stacktrace for every request in the batch (#9023)
Wallas Henrique committed about 2 years ago
[BugFix] Update draft model TP size check to allow matching target TP size (#9394)
Nick Hill committed about 2 years ago
youkaichao committed about 2 years ago
[Frontend] Reduce frequency of client cancellation checking (#7959)
Nick Hill committed about 2 years ago
[Model][Bugfix] Fix batching with multi-image in PixtralHF (#9518)
Michael Goin committed about 2 years ago
[BugFix] Use correct python3 binary in Docker.ppc64le entrypoint (#9492)
Varad Ahirwadkar committed about 2 years ago
[Bugfix][Misc]: fix graph capture for decoder (#9549)
yudian0504 committed about 2 years ago
[Model] FalconMamba Support (#9325)
Dhia Eddine Rhaiem committed about 2 years ago
[Doc] Consistent naming of attention backends (#9498)
Thomas Parnell committed about 2 years ago
[CI/Build] Split up decoder-only LM tests (#9488)
Cyrus Leung committed about 2 years ago
[Frontend][Misc] Goodput metric support (#9338)
Andy Dai committed about 2 years ago
[Kernel] Support sliding window in flash attention backend (#9403)
Chen Zhang committed about 2 years ago
[Model][Pixtral] Use memory_efficient_attention for PixtralHFVision (#9520)
Michael Goin committed about 2 years ago
[Bugfix] Pass json-schema to GuidedDecodingParams and make test stronger (#9530)
Chen Zhang committed about 2 years ago
[Model][Pixtral] Optimizations for input_processor_for_pixtral_hf (#9514)
Michael Goin committed about 2 years ago
[Bugfix] Fix missing task for speculative decoding (#9524)
Cyrus Leung committed about 2 years ago
[Frontend] Support simpler image input format (#9478)
Yue Zhang committed about 2 years ago
[CI/Build] Configure matcher for actionlint workflow (#9511)
Russell Bryant committed about 2 years ago