A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 05:45 UTC
main
[Interleaved ATTN] Support for Mistral-8B (#10591)
Patrick von Platen committed almost 2 years ago
[Model] Refactor Molmo weights loading to use AutoWeightsLoader (#10771)
Isotr0py committed almost 2 years ago
[Bugfix] Fix OpenVino/Neuron `driver_worker` init (#10779)
Nicolò Lucchesi committed almost 2 years ago
[platform] Add verify_quantization in platform. (#10757)
wangxiyuan committed almost 2 years ago
[Bugfix] Fix Idefics3 bug (#10778)
Jee Jee Li committed almost 2 years ago
[Misc] typo find in sampling_metadata.py (#10740)
wang.yuqi committed almost 2 years ago
[Model] Clean up MiniCPMV (#10751)
Cyrus Leung committed almost 2 years ago
[Model] Add Internlm2 LoRA support (#5064)
Isotr0py committed almost 2 years ago
[V1] Optimize the CPU overheads in FlashAttention custom op (#10733)
Woosuk Kwon committed almost 2 years ago
[Kernel] Update vllm-flash-attn version to reduce CPU overheads (#10742)
Woosuk Kwon committed almost 2 years ago
[Model] Added GLM-4 series hf format model support vllm==0.6.4 (#10561)
sixgod committed almost 2 years ago
[TPU] Update requirements-tpu (#10726)
Richard Liu committed almost 2 years ago
[Kernel] Update vllm-flash-attn version (#10736)
Woosuk Kwon committed almost 2 years ago
[V1] Do not allocate beyond the max_model_len (#10730)
Woosuk Kwon committed almost 2 years ago
[Bug][CLI] Allow users to disable prefix caching explicitly (#10724)
Ricky Xu committed almost 2 years ago
[Doc] Update model in arch_overview.rst to match comment (#10701)
罗泽轩 committed almost 2 years ago
[Model] support bitsandbytes quantization with minicpm3 model (#10682)
zixuanzhang226 committed almost 2 years ago
[misc] upgrade filelock version (#10731)
youkaichao committed almost 2 years ago
[Frontend] don't block event loop in tokenization (preprocess) in OpenAI compatible server (#10635)
tomeras91 committed almost 2 years ago
[Bugfix] Ignore `lm_head` when loading embedding models (#10719)
Cyrus Leung committed almost 2 years ago