A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 18:30 UTC
main
[Neuron] trim attention kernel tests to fit trn1.2x instance (#14988)
Liangfu Chen committed over 1 year ago
[Bugfix] Fix bnb quantization for models with both HF-format and Mistral-format weights (#14950)
Tristan Leclercq committed over 1 year ago
[V1] TPU - Fix CI/CD runner (#14974)
Alexander Matveev committed over 1 year ago
[V1] Guard Against Main Thread Usage (#14972)
Robert Shaw committed over 1 year ago
[CI/Build] Use `AutoModelForImageTextToText` to load VLMs in tests (#14945)
Cyrus Leung committed over 1 year ago
[Bugfix] Fix interface for Olmo2 on V1 (#14976)
Roger Wang committed over 1 year ago
[Bugfix] Make Gemma3 MM V0 only for now (#14971)
Roger Wang committed over 1 year ago
[Fix][Structured Output] using vocab_size to construct matcher (#14868)
Aaron Pham committed over 1 year ago
[Bugfix][Model] Mixtral: use unused head_dim config argument (#14961)
Quentin committed over 1 year ago
[Bugfix] Fix precommit - line too long in pixtral.py (#14960)
Tyler Michael Smith committed over 1 year ago
[V1] Default MLA to V1 (#14921)
Simon Mo committed over 1 year ago
Patrick von Platen committed over 1 year ago
[Doc] Fix misleading log during multi-modal profiling (#14955)
Cyrus Leung committed over 1 year ago
[Bugfix][ROCm] running new process using spawn method for rocm in tests. (#14810)
vllmellm committed over 1 year ago
[Misc] Add `--seed` option to offline multi-modal examples (#14934)
Cyrus Leung committed over 1 year ago
[Bugfix] Fix Ultravox on V1 (#14929)
Cyrus Leung committed over 1 year ago
[V1][TPU] Apply the ragged paged attention kernel fix and remove the padding. (#14846)
iefgnoix committed over 1 year ago
fix minor miscalled method (#14327)
kushanam committed over 1 year ago
[MISC] More AMD unused var clean up (#14926)
Lu Fang committed over 1 year ago
setup.py: drop assumption about local `main` branch (#14692)
Russell Bryant committed over 1 year ago