A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 05:45 UTC
main
[Model] Add TP and BNB quantization support to LlavaMultiModalProjector (#10834)
Isotr0py committed almost 2 years ago
Jani Monoses committed almost 2 years ago
[Misc][XPU] Avoid torch compile for XPU platform (#10747)
Yan Ma committed almost 2 years ago
[Misc][LoRA] Move the implementation of lora bias to punica.py (#10829)
Jee Jee Li committed almost 2 years ago
[misc] use out argument for flash attention (#10822)
youkaichao committed almost 2 years ago
[CI/Build] Update `mistral_common` version for tests and docs (#10825)
Cyrus Leung committed almost 2 years ago
[Model]: add some tests for aria model (#10770)
zhou fan committed almost 2 years ago
[doc]Update config docstring (#10732)
wangxiyuan committed almost 2 years ago
[misc] remove xverse modeling file (#10814)
youkaichao committed almost 2 years ago
Fill TorchSDPAAttentionMetadata seq_lens_field for prefill (#10799)
Maximilien de Bayser committed almost 2 years ago
[Kernel] Use `out` arg in flash_attn_varlen_func (#10811)
Woosuk Kwon committed almost 2 years ago
[core] Avoid metrics log noise when idle - include speculative decodi… (#10809)
cduk committed almost 2 years ago
[Model] Add BNB support to Llava and Pixtral-HF (#10795)
Isotr0py committed almost 2 years ago
[Core] Implement disagg prefill by StatelessProcessGroup (#10502)
Kuntai Du committed almost 2 years ago
[Misc] Adding `MMMU-Pro` vision dataset to serving benchmark (#10804)
Roger Wang committed almost 2 years ago
[doc] add warning about comparing hf and vllm outputs (#10805)
youkaichao committed almost 2 years ago
[Misc] Rename embedding classes to pooling (#10801)
Cyrus Leung committed almost 2 years ago
[Misc] Improve type annotations for `support_torch_compile` (#10763)
Cyrus Leung committed almost 2 years ago
[Model] Replace embedding models with pooling adapter (#10769)
Cyrus Leung committed almost 2 years ago
wangxiyuan committed almost 2 years ago