A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 08:15 UTC
main
[CI/Build] Always run the ruff workflow (#10092)
Russell Bryant committed about 2 years ago
[V1] Make v1 more testable (#9888)
Joe Runde committed about 2 years ago
[CI/Build] change conflict PR comment from mergify (#10080)
Russell Bryant committed about 2 years ago
[CI/Build] Drop Python 3.8 support (#10038)
Russell Bryant committed about 2 years ago
Remove ScaledActivation for AWQ (#10057)
Michael Goin committed about 2 years ago
[Model][LoRA]LoRA support added for Qwen2VLForConditionalGeneration (#10022)
Eric committed about 2 years ago
[Model] Add Idefics3 support (#9767)
Jee Jee Li committed about 2 years ago
[Model][LoRA]LoRA support added for LlamaEmbeddingModel (#10071)
Jee Jee Li committed about 2 years ago
[Hotfix] Fix ruff errors (#10073)
Woosuk Kwon committed about 2 years ago
[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143)
Konrad Zawora committed about 2 years ago
[CI/Build] Fix large_gpu_mark reason (#10070)
Isotr0py committed about 2 years ago
[CI/Build] drop support for Python 3.8 EOL (#8464)
Aaron Pham committed about 2 years ago
[distributed] add function to create ipc buffers directly (#10064)
youkaichao committed about 2 years ago
[V1] Integrate Piecewise CUDA graphs (#10058)
Woosuk Kwon committed about 2 years ago
[Bugfix] Remove CustomChatCompletionContentPartParam multimodal input type (#10054)
zifeitong committed about 2 years ago
[Bugfix] Gpt-j-6B patch kv_scale to k_scale path (#10063)
arakowsk-amd committed about 2 years ago
[Bugfix] Fix edge-case crash when using chat with the Mistral Tekken Tokenizer (#10051)
Travis Johnson committed about 2 years ago
[Model][OpenVINO] Fix regressions from #8346 (#10045)
Peter Salas committed about 2 years ago
[Misc] Sort the list of embedding models (#10037)
Cyrus Leung committed about 2 years ago
[v1] reduce graph capture time for piecewise cudagraph (#10059)
youkaichao committed about 2 years ago