A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 14:45 UTC
main
[Bugfix][Frontend] respect provided default guided decoding backend (#15476)
Guillaume Calmettes committed over 1 year ago
[Model] Add `SupportsMultiModal.get_language_model` interface (#16007)
Nicolò Lucchesi committed over 1 year ago
[BugFix] fix some typos found by typos. (#16314)
yihong committed over 1 year ago
ajayvohra2005 committed over 1 year ago
[BugFix] llama4 qknorm should be not shared across head (#16311)
Lucia Fang committed over 1 year ago
[BugFix] logger is not callable (#16312)
yihong committed over 1 year ago
Update label-tpu mergify and remove removal bot (#16298)
Michael Goin committed over 1 year ago
[Bugfix] Avoid transferring cached multi-modal items from P0 to P1 (#16273)
Cyrus Leung committed over 1 year ago
[Misc] Fix test_sharded_state_loader.py(#16004) (#16005)
Accelerator1996 committed over 1 year ago
[BugFix] Fix fusion test and add them to CI (#16287)
Luka Govedič committed over 1 year ago
[TPU] Update PyTorch/XLA (#16288)
Chengji Yao committed over 1 year ago
[CI][Bugfix] Fix bad tolerance for test_batch_base64_embedding (#16221)
Michael Goin committed over 1 year ago
[Core] Upgrade to xgrammar 0.1.18, add cache size limit (#16283)
Russell Bryant committed over 1 year ago
[Feature] Estimate max-model-len use available KV cache memory (#16168)
rongfu.leng committed over 1 year ago
[Bugfix] fix gettid method is not define (#16084)
rongfu.leng committed over 1 year ago
TJian committed over 1 year ago
[Model] Add smolvlm support (#16017)
Chauncey committed over 1 year ago
Add support to modelopt quantization of Mixtral model (#15961)
yueshen2016 committed over 1 year ago
[CI/Build] Fix CI LoRA failure (#16270)
Jee Jee Li committed over 1 year ago
[V1] Update structured output offline inference example (#15721)
Russell Bryant committed over 1 year ago