A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
main
[Model] Support NVLM-D and fix QK Norm in InternViT (#9045)
Cyrus Leung committed about 2 years ago
[Core] Refactor GGUF parameters packing and forwarding (#8859)
Isotr0py committed about 2 years ago
[Hardware][CPU] Cross-attention and Encoder-Decoder models support on CPU backend (#9089)
Isotr0py committed about 2 years ago
[Model] Explicit interface for vLLM models and support OOT embedding models (#9108)
Cyrus Leung committed about 2 years ago
[core] remove beam search from the core (#9105)
youkaichao committed about 2 years ago
[BugFix][Core] Fix BlockManagerV2 when Encoder Input is None (#9103)
sroy745 committed about 2 years ago
[Bugfix][Hardware][CPU] Fix CPU model input for decode (#9044)
Isotr0py committed about 2 years ago
[Bugfix] Fix incorrect updates to num_computed_tokens in multi-step scheduling (#9038)
Varun Sundar Rabindranath committed about 2 years ago
[Bugfix] fix tool_parser error handling when serve a model not support it (#8709)
Yanyi Liu committed about 2 years ago
[Model] PP support for embedding models and update docs (#9090)
Cyrus Leung committed about 2 years ago
[Misc] Remove user-facing error for removed VLM args (#9104)
Cyrus Leung committed about 2 years ago
[Frontend] API support for beam search (#9087)
Brendan Wong committed about 2 years ago
[Bugfix] Fix try-catch conditions to import correct Flash Attention Backend in Draft Model (#9101)
TJian committed about 2 years ago
[core] use forward context for flash infer (#9097)
youkaichao committed about 2 years ago
[Bugfix] Fix order of arguments matters in config.yaml (#8960)
Andy Dai committed about 2 years ago
[Bugfix] Deprecate registration of custom configs to huggingface (#9083)
Chen Zhang committed about 2 years ago
[Model] Support Gemma2 embedding model (#9004)
Xin Yang committed about 2 years ago
[Bugfix] Fixes Phi3v & Ultravox Multimodal EmbeddingInputs (#8979)
hhzhang16 committed about 2 years ago
[Bugfix] use blockmanagerv1 for encoder-decoder (#9084)
Chen Zhang committed about 2 years ago
[Doc] Update README.md with Ray summit slides (#9088)
Zhuohan Li committed about 2 years ago