A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 15:45 UTC
main
[misc] fix collect env (#8894)
youkaichao committed about 2 years ago
[Core] rename`PromptInputs` and `inputs` (#8876)
Cyrus Leung committed about 2 years ago
[Feature] Add support for Llama 3.1 and 3.2 tool use (#8343)
Maximilien de Bayser committed about 2 years ago
[Installation] Allow lower versions of FastAPI to maintain Ray 2.9 compatibility (#8764)
Cyrus Leung committed about 2 years ago
[BugFix] Fix test breakages from transformers 4.45 upgrade (#8829)
Nick Hill committed about 2 years ago
[Bugfix] Fixup advance_step.cu warning (#8815)
Tyler Michael Smith committed about 2 years ago
fix validation: Only set tool_choice `auto` if at least one tool is provided (#8568)
Chirag Jain committed about 2 years ago
[Bugfix] Fix print_warning_once's line info (#8867)
Tyler Michael Smith committed about 2 years ago
[Misc] Change dummy profiling and BOS fallback warns to log once (#8820)
Michael Goin committed about 2 years ago
[Bugfix] Include encoder prompts len to non-stream api usage response (#8861)
Pernekhan Utemuratov committed about 2 years ago
[ci] Soft fail Entrypoints, Samplers, LoRA, Decoder-only VLM (#8872)
Kevin H. Luu committed about 2 years ago
[misc][installation] build from source without compilation (#8818)
youkaichao committed about 2 years ago
[CI/Build] Fix missing ci dependencies (#8834)
fyuan1316 committed about 2 years ago
[Docs] Add README to the build docker image (#8825)
Michael Goin committed about 2 years ago
[Build/CI] Upgrade to gcc 10 in the base build Docker image (#8814)
Tyler Michael Smith committed about 2 years ago
[Misc] Update config loading for Qwen2-VL and remove Granite (#8837)
Roger Wang committed about 2 years ago
[Misc] Support quantization of MllamaForCausalLM (#8822)
Michael Goin committed about 2 years ago
[Doc] Update doc for Transformers 4.45 (#8817)
Roger Wang committed about 2 years ago
[Model] Add support for the multi-modal Llama 3.2 model (#8811)
Chen Zhang committed about 2 years ago
Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810)
Simon Mo committed about 2 years ago