A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 05:45 UTC
main
[Doc] Add encoder-based models to Supported Models page (#10616)
Cyrus Leung committed almost 2 years ago
[torch.compile] support encoder based models (#10613)
youkaichao committed almost 2 years ago
[Refactor][MISC] del redundant code in ParallelConfig.postinit (#10614)
Mengqing Cao committed almost 2 years ago
Support Cross encoder models (#10400)
Maximilien de Bayser committed almost 2 years ago
[Doc] Update README.md with Ray Summit talk links (#10610)
Zhuohan Li committed almost 2 years ago
[doc] update the code to add models (#10603)
youkaichao committed almost 2 years ago
[model][utils] add extract_layer_index utility function (#10599)
youkaichao committed almost 2 years ago
Revert "Print running script to enhance CI log readability" (#10601)
youkaichao committed almost 2 years ago
[CI/Build] Print running script to enhance CI log readability (#10594)
Jee Jee Li committed almost 2 years ago
[Bugfix] Fix LoRA weight sharding (#10450)
Jee Jee Li committed almost 2 years ago
[bugfix] Fix example/tensorize_vllm_model tests (#10595)
Jee Jee Li committed almost 2 years ago
[Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593)
Isotr0py committed almost 2 years ago
For ppc64le, disabled tests for now and addressed space issues (#10538)
Nishidha committed almost 2 years ago
[Platforms] Refactor openvino code (#10573)
JiHuazhong committed almost 2 years ago
[Bugfix] `multi_modal_kwargs` broadcast for CPU tensor parallel (#10541)
Isotr0py committed almost 2 years ago
[2/N] handling placeholders in merged multi-modal processor (#10485)
Cyrus Leung committed almost 2 years ago
Prefix Cache Aware Scheduling [1/n] (#10128)
Ricky Xu committed almost 2 years ago
[AMD] Add support for GGUF quantization on ROCm (#10254)
kliuae committed almost 2 years ago
Update default max_num_batch_tokens for chunked prefill to 2048 (#10544)
Michael Goin committed almost 2 years ago
[Model] Fix Baichuan BNB online quantization (#10572)
Chen Wu committed almost 2 years ago