A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 04:30 UTC
main
[Frontend] Add OpenAI API support for input_audio (#11027)
kYLe committed almost 2 years ago
Michael Goin committed almost 2 years ago
fix gh200 tests on main (#11246)
youkaichao committed almost 2 years ago
[torch.compile] fast inductor (#11108)
youkaichao committed almost 2 years ago
[ci][tests] add gh200 tests (#11244)
youkaichao committed almost 2 years ago
bk-TurbaAI committed almost 2 years ago
[core] overhaul memory profiling and fix backward compatibility (#10511)
youkaichao committed almost 2 years ago
[misc] Layerwise profile updates (#10242)
Varun Sundar Rabindranath committed almost 2 years ago
[Doc] Reorder vision language examples in alphabet order (#11228)
Isotr0py committed almost 2 years ago
[Model] Refactor Ultravox to use merged input processor (#11198)
Isotr0py committed almost 2 years ago
[Model] Support Cohere2ForCausalLM (Cohere R7B) (#11203)
Jani Monoses committed almost 2 years ago
WIP: [CI/Build] simplify Dockerfile build for ARM64 / GH200 (#11212)
cennn committed almost 2 years ago
[Bugfix] Fix the default value for temperature in ChatCompletionRequest (#11219)
yansh97 committed almost 2 years ago
[Bugfix] Fix block size validation (#10938)
chenqianfzh committed almost 2 years ago
Update deploying_with_k8s.rst (#10922)
AlexHe99 committed almost 2 years ago
[V1][Minor] Cache np arange to reduce input preparation overhead (#11214)
Woosuk Kwon committed almost 2 years ago
[Core] Support disaggregated prefill with Mooncake Transfer Engine (#10884)
shangmingc committed almost 2 years ago
[Doc] add documentation for disaggregated prefilling (#11197)
Kuntai Du committed almost 2 years ago
[Bugfix] Fix error handling of unsupported sliding window (#11213)
Cyrus Leung committed almost 2 years ago
[Misc] Clean up multi-modal processor (#11207)
Cyrus Leung committed almost 2 years ago