A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 05:45 UTC
main
[Core] Support Lark grammars for XGrammar (#10870)
Michael Goin committed almost 2 years ago
[torch.compile] fix deprecated code (#10948)
youkaichao committed almost 2 years ago
[CI/Build] Fix broken multimodal test (#10950)
Cyrus Leung committed almost 2 years ago
[torch.compile] add logging for compilation time (#10941)
youkaichao committed almost 2 years ago
[torch.compile] use size tuning for specific sizes (#10933)
youkaichao committed almost 2 years ago
[ci][build] add tests for python only compilation (#10915)
youkaichao committed almost 2 years ago
[Misc][Gaudi] Avoid torch.compile and enable lazy collectives (#10897)
Konrad Zawora committed almost 2 years ago
[CI/Build] Bump test transformers version (#10106)
Isotr0py committed almost 2 years ago
[Misc][LoRA] Clean up the function interface of Punica (#10917)
Jee Jee Li committed almost 2 years ago
[Misc] Update llama 3.2 template to support system prompt with images (#10901)
Travis Johnson committed almost 2 years ago
[Bugfix] Fix BNB loader target_modules (#10720)
Jee Jee Li committed almost 2 years ago
[Doc] Create a new "Usage" section (#10827)
Cyrus Leung committed almost 2 years ago
[Bugfix] Fallback to outlines for complex json schemas (#10899)
Michael Goin committed almost 2 years ago
[benchmark] Make H100 benchmark optional (#10908)
Kevin H. Luu committed almost 2 years ago
[V1] Fix when max_model_len is not divisible by block_size (#10903)
Woosuk Kwon committed almost 2 years ago
[CI/Build] improve python-only dev setup (#9621)
Daniele committed almost 2 years ago
Benchmark serving structured output (#10880)
Chendi.Xue committed almost 2 years ago
[Model] Consolidate ViTs attention implementation without mask (#10893)
Isotr0py committed almost 2 years ago
[LoRA] Change lora_tokenizers capacity (#10796)
Xin Yang committed almost 2 years ago
[ci/build] Update vLLM postmerge ECR repo (#10887)
Kevin H. Luu committed almost 2 years ago