A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 05:45 UTC
main
[ci] add vllm_test_utils (#10659)
youkaichao committed almost 2 years ago
[Misc] Remove outdated init protocols (#10655)
Cyrus Leung committed almost 2 years ago
custom allreduce + torch.compile (#10121)
Sage Moore committed almost 2 years ago
[v1] EngineArgs for better config handling for v1 (#10382)
Ricky Xu committed almost 2 years ago
[Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228)
Sanket Kale committed almost 2 years ago
[bugfix] fix aria model and add torch.compile (#10645)
youkaichao committed almost 2 years ago
[misc] do not read HOST_IP (#10644)
youkaichao committed almost 2 years ago
[Model] Add OLMo November 2024 model (#10503)
Shane A committed almost 2 years ago
[Doc] Fix typos in docs (#10636)
Cyrus Leung committed almost 2 years ago
[Model] Enable optional prefix when loading embedding models (#10639)
Cyrus Leung committed almost 2 years ago
[Model]: Add support for Aria model (#10514)
zhou fan committed almost 2 years ago
[Docs] Add Snowflake Slides (#10641)
Simon Mo committed almost 2 years ago
[Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850)
Wallas Henrique committed almost 2 years ago
[Bug]: Authorization ignored when root_path is set (#10606)
Chauncey committed almost 2 years ago
Super tiny little typo fix (#10633)
fzyzcjy committed almost 2 years ago
[Model] Support `is_causal` HF config field for Qwen2 model (#10621)
Cyrus Leung committed almost 2 years ago
[misc] move functions to config.py (#10624)
youkaichao committed almost 2 years ago
[misc] add torch.compile compatibility check (#10618)
youkaichao committed almost 2 years ago
[torch.compile] add warning for unsupported models (#10622)
youkaichao committed almost 2 years ago
[torch.compile] force inductor threads (#10620)
Jee Jee Li committed almost 2 years ago