A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 23:30 UTC
main
[V1][BugFix] Add __init__.py to v1/spec_decode/ (#13359)
Woosuk Kwon committed over 1 year ago
[Docs] Change myenv to vllm. Update python_env_setup.inc.md (#13325)
凌 committed over 1 year ago
[Bugfix] Fix 2 Node and Spec Decode tests (#13341)
Cyrus Leung committed over 1 year ago
[platform] add base class for communicators (#13208)
youkaichao committed over 1 year ago
[ci] skip failed tests for flashinfer (#13352)
youkaichao committed over 1 year ago
[V1] Update doc and examples for H2O-VL (#13349)
Roger Wang committed over 1 year ago
[BugFix] Enhance test_pos_encoding to support execution on multi-devices (#13187)
wchen61 committed over 1 year ago
[Bugfix] Pin xgrammar to 0.1.11 (#13338)
Michael Goin committed over 1 year ago
[Quant] Add `SupportsQuant` to phi3 and clip (#13104)
Kyle Sayers committed over 1 year ago
[V1][Spec Decode] Ngram Spec Decode (#12193)
Lily Liu committed over 1 year ago
[Doc] [2/N] Add Fuyu E2E example for multimodal processor (#13331)
Cyrus Leung committed over 1 year ago
[ci/build] update flashinfer (#13323)
youkaichao committed over 1 year ago
[V1][PP] Run engine busy loop with batch queue (#13064)
Cody Yu committed over 1 year ago
[AMD] [Model] DeepSeek tunings (#13199)
rasmith committed over 1 year ago
[V1][Metrics] Add iteration_tokens_total histogram from V0 (#13288)
Mark McLoughlin committed over 1 year ago
[Bugfix] Fix qwen2.5-vl image processor (#13286)
Isotr0py committed over 1 year ago
[Bugfix]Fix search start_index of stop_checker (#13280)
Xu Song committed almost 2 years ago
[BugFix] Don't scan entire cache dir when loading model (#13302)
Nick Hill committed almost 2 years ago
[Bugfix] Massage MLA's usage of flash attn for RoCM (#13310)
Tyler Michael Smith committed almost 2 years ago
[Bugfix][Docs] Fix offline Whisper (#13274)
Nicolò Lucchesi committed almost 2 years ago