A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
Fix Auto_Round Quatization Loading on SM75 and Lower GPUs (#24217)
RoadToNowhereX committed about 1 year ago
co63oc committed about 1 year ago
[CI/Build] bump timm dependency (#24189)
Daniele committed about 1 year ago
[CI] Add PPL test for generation models (#24485)
wang.yuqi committed about 1 year ago
[rocm] enable torchao quantization for rocm (#24400)
Lifans committed about 1 year ago
[CI/Build] split true unit tests to Entrypoints Unit Tests (#24418)
Ye (Charlotte) Qi committed about 1 year ago
[BugFix] Ensure integrity of reused CPU tensors during async scheduling (#24527)
Nick Hill committed about 1 year ago
[BugFix][easy] Fix flaky test test_gpt_oss_multi_turn_chat (#24549)
lacora committed about 1 year ago
[BugFix][Multi Modal] Fix TensorSchema shape mismatch in Molmo (#24559)
Wenlong Wang committed about 1 year ago
Move `LoadConfig` from `config/__init__.py` to `config/load.py` (#24566)
Harry Mellor committed about 1 year ago
[LoRA]: Add LoRA support to Mistral's Voxtral models (#24517)
Yash Pratap Singh committed about 1 year ago
Feature/vit attention unification# 23880 (#23978)
baonudesifeizhai committed about 1 year ago
Support for NemotronH Nano VLM (#23644)
danielafrimi committed about 1 year ago
[Docs] Document the extra memory footprint overhead when using EPLB (#24537)
Tyler Michael Smith committed about 1 year ago
[Docs] Improve organisation of API Reference nav (#24569)
Harry Mellor committed about 1 year ago
[ROCm][Bugfix] Fix Aiter RMSNorm (#23412)
vllmellm committed about 1 year ago
[Attention] add DCP support for FLASH_ATTN_MLA backend (#24453)
Lucas Wilkinson committed about 1 year ago
[Model] Limit CPU threads for image transformations in InternVL to reduce cpu contention. (#24519)
li-jinpeng committed about 1 year ago
Consolidate rendering parameters into RenderConfig dataclass (#24543)
Flora Feng committed about 1 year ago
[Bugfix] Guard `_may_reorder_batch` for encoder-only models on CPU (#24319) (#24348)
Remy committed about 1 year ago