A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
main
[torch.compile] fix tensor alias (#8982)
youkaichao committed about 2 years ago
[CI/Build] Fix machete generated kernel files ordering (#8976)
Kevin H. Luu committed about 2 years ago
[Frontend][Core] Move guided decoding params into sampling params (#8252)
Joe Runde committed about 2 years ago
Lily Liu committed about 2 years ago
[ci] Add CODEOWNERS for test directories (#8795)
Kevin H. Luu committed about 2 years ago
[Misc] Adjust max_position_embeddings for LoRA compatibility (#8957)
Jee Jee Li committed about 2 years ago
[Core] Make scheduling policy settable via EngineArgs (#8956)
Sebastian Schoennenbeck committed about 2 years ago
[Model] Expose InternVL2 max_dynamic_patch as a mm_processor_kwarg (#8946)
Isotr0py committed about 2 years ago
[Model][LoRA]LoRA support added for MiniCPMV2.6 (#8943)
Jee Jee Li committed about 2 years ago
[Misc][CI/Build] Include `cv2` via `mistral_common[opencv]` (#8951)
Roger Wang committed about 2 years ago
[Model] support input embeddings for qwen2vl (#8856)
whyiug committed about 2 years ago
[Kernel][Model] Varlen prefill + Prefill chunking support for mamba kernels and Jamba model (#8533)
Mor Zusman committed about 2 years ago
[Frontend] Added support for HF's new `continue_final_message` parameter (#8942)
danieljannai21 committed about 2 years ago
[Misc] Fix typo in BlockSpaceManagerV1 (#8944)
juncheoll committed about 2 years ago
[BugFix] Fix seeded random sampling with encoder-decoder models (#8870)
Nick Hill committed about 2 years ago
[Model][LoRA]LoRA support added for MiniCPMV2.5 (#7199)
Jee Jee Li committed about 2 years ago
[Model] Support Qwen2.5-Math-RM-72B (#8896)
Zilin Zhu committed about 2 years ago
[Build/CI] Set FETCHCONTENT_BASE_DIR to one location for better caching (#8930)
Tyler Michael Smith committed about 2 years ago
[CI/Build] Add test decorator for minimum GPU memory (#8925)
Cyrus Leung committed about 2 years ago
[Bugfix] Fix Marlin MoE act order when is_k_full == False (#8741)
ElizaWszola committed about 2 years ago