A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 19:30 UTC
main
[Bugfix] Fix input processor for InternVL2 model (#7164)
Isotr0py committed over 2 years ago
Fixes typo in function name (#7275)
Rafael Vasquez committed over 2 years ago
[ BugFix ] Move `zmq` frontend to IPC instead of TCP (#7222)
Robert Shaw committed over 2 years ago
Dipika Sikka committed over 2 years ago
[distributed][misc] add specialized method for cuda platform (#7249)
youkaichao committed over 2 years ago
[Frontend] Gracefully handle missing chat template and fix CI failure (#7238)
Cyrus Leung committed over 2 years ago
[BUGFIX]: top_k is expected to be an integer. (#7227)
Atilla Akkuş committed over 2 years ago
[Doc] Mock new dependencies for documentation (#7245)
Roger Wang committed over 2 years ago
[BugFix] Overhaul async request cancellation (#7111)
Nick Hill committed over 2 years ago
[Bugfix] Fix GPTQ and GPTQ Marlin CPU Offloading (#7225)
Michael Goin committed over 2 years ago
afeldman-nm committed over 2 years ago
[Core] Optimize evictor-v2 performance (#7193)
xiaobochen123 committed over 2 years ago
[Kernel] Add per-tensor and per-token AZP epilogues (#5941)
Luka Govedič committed over 2 years ago
[SpecDecode] [Minor] Fix spec decode sampler tests (#7183)
Lily Liu committed over 2 years ago
[Bugfix] add gguf dependency (#7198)
Katarzyna Papis committed over 2 years ago
[ BugFix ] Fix ZMQ when `VLLM_PORT` is set (#7205)
Robert Shaw committed over 2 years ago
[BugFix] Fix DeepSeek remote code (#7178)
Dipika Sikka committed over 2 years ago
[Model] Support SigLIP encoder and alternative decoders for LLaVA models (#7153)
Cyrus Leung committed over 2 years ago
[LoRA] Relax LoRA condition (#7146)
Jee Jee Li committed over 2 years ago
[Build] Add initial conditional testing spec (#6841)
Simon Mo committed over 2 years ago