A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 07:00 UTC
main
[Bugfix] Internal Server Error when tool_choice is incorrect. (#10567)
Varun Vinayak Shenoy committed almost 2 years ago
[core] gemma2 full context length support (#10584)
youkaichao committed almost 2 years ago
[Misc] Add pynccl wrappers for all_gather and reduce_scatter (#9432)
Tyler Michael Smith committed almost 2 years ago
[Core] Fix broken log configuration (#10458)
Russell Bryant committed almost 2 years ago
[Bugfix][Frontend] Update Llama Chat Templates to also support Non-Tool use (#10164)
Travis Johnson committed almost 2 years ago
[bugfix] fix cpu tests (#10585)
youkaichao committed almost 2 years ago
[V1] EngineCore supports profiling (#10564)
Zhonghua Deng committed almost 2 years ago
[Core] remove temporary local variables in LLMEngine.__init__ (#10577)
Russell Bryant committed almost 2 years ago
support bitsandbytes quantization with qwen model (#10549)
zixuanzhang226 committed almost 2 years ago
[v1] Refactor KVCacheManager for more hash input than token ids (#10507)
Ricky Xu committed almost 2 years ago
[torch.compile] support all attention backends (#10558)
youkaichao committed almost 2 years ago
[bugfix] fix full graph tests (#10581)
youkaichao committed almost 2 years ago
Remove token-adding chat embedding params (#10551)
Noam Gat committed almost 2 years ago
[Bugfix] Fix Phi-3 BNB quantization with tensor parallel (#9948)
Isotr0py committed almost 2 years ago
[platforms] absorb worker cls difference into platforms folder (#10555)
youkaichao committed almost 2 years ago
[Minor] Fix line-too-long (#10563)
Woosuk Kwon committed almost 2 years ago
[9/N] torch.compile LLM usage (#10552)
youkaichao committed almost 2 years ago
[Benchmark] Add new H100 machine (#10547)
Simon Mo committed almost 2 years ago
Add small example to metrics.rst (#10550)
Michael Goin committed almost 2 years ago
[Minor] Revert change in offline inference example (#10545)
Woosuk Kwon committed almost 2 years ago