A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 17:00 UTC
main
[misc][ci] fix quant test (#8449)
youkaichao committed about 2 years ago
[Misc] Skip loading extra bias for Qwen2-VL GPTQ-Int8 (#8442)
Jee Jee Li committed about 2 years ago
[doc] recommend pip instead of conda (#8446)
youkaichao committed about 2 years ago
[CI/Build] Enable InternVL2 PP test only on single node (#8437)
Isotr0py committed about 2 years ago
bump version to v0.6.1.post1 (#8440)
Simon Mo committed about 2 years ago
[bugfix] torch profiler bug for single gpu with GPUExecutor (#8354)
William Lin committed about 2 years ago
[Bugfix] Fix async log stats (#8417)
Alexander Matveev committed about 2 years ago
[CI/Build] Update pixtral tests to use JSON (#8436)
Cyrus Leung committed about 2 years ago
[Bugfix] Bump fastapi and pydantic version (#8435)
Cyrus Leung committed about 2 years ago
[Bugfix] Mapping physical device indices for e2e test utils (#8290)
shangmingc committed about 2 years ago
[Core] Factor out input preprocessing to a separate class (#7329)
Cyrus Leung committed about 2 years ago
[BugFix] fix group_topk (#8430)
Dipika Sikka committed about 2 years ago
[Misc] Update Pixtral example (#8431)
Roger Wang committed about 2 years ago
[Bugfix] Fix weight loading issue by rename variable. (#8293)
Wenxiang committed about 2 years ago
[Hotfix][Pixtral] Fix multiple images bugs (#8415)
Patrick von Platen committed about 2 years ago
[CI/Build] Disable multi-node test for InternVL2 (#8428)
Roger Wang committed about 2 years ago
Roger Wang committed about 2 years ago
[Bugfix] multi-step + flashinfer: ensure cuda graph compatible (#8427)
Alexander Matveev committed about 2 years ago
[Core] Add engine option to return only deltas or final output (#7381)
Nick Hill committed about 2 years ago
[multi-step] add flashinfer backend (#7928)
William Lin committed about 2 years ago