A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
main
[Doc] Add "Suggest edit" button to doc pages (#5789)
Michael Goin committed over 2 years ago
[doc][faq] add warning to download models for every nodes (#5783)
youkaichao committed over 2 years ago
[Bugfix] Add phi3v resize for dynamic shape and fix torchvision requirement (#5772)
Isotr0py committed over 2 years ago
[Distributed] Add send and recv helpers (#5719)
Murali Andoorveedu committed over 2 years ago
[BugFix] [Kernel] Add Cutlass2x fallback kernels (#5744)
Varun Sundar Rabindranath committed over 2 years ago
[core][distributed] improve shared memory broadcast (#5754)
youkaichao committed over 2 years ago
[Docs][TPU] Add installation tip for TPU (#5761)
Woosuk Kwon committed over 2 years ago
[Bugfix] Fix pin_lora error in TPU executor (#5760)
Woosuk Kwon committed over 2 years ago
[Misc] Remove #4789 workaround left in vllm/entrypoints/openai/run_batch.py (#5756)
zifeitong committed over 2 years ago
[Model] Support Qwen-VL and Qwen-VL-Chat models with text-only inputs (#5710)
Jie Fu (傅杰) committed over 2 years ago
[CI][Hardware][Intel GPU] add Intel GPU(XPU) ci pipeline (#5616)
Kunshang Ji committed over 2 years ago
[LoRA] Add support for pinning lora adapters in the LRU cache (#5603)
rohithkrn committed over 2 years ago
[ci][test] fix ca test in main (#5746)
youkaichao committed over 2 years ago
[BugFix] exclude version 1.15.0 for modelscope (#5668)
zhyncs committed over 2 years ago
[Doc] Documentation on supported hardware for quantization methods (#5745)
Michael Goin committed over 2 years ago
[Kernel][CPU] Add Quick `gelu` to CPU (#5717)
Roger Wang committed over 2 years ago
[Core][Distributed] add shm broadcast (#5399)
youkaichao committed over 2 years ago
[Bugfix] Add fully sharded layer for QKVParallelLinearWithLora (#5665)
Jee Li committed over 2 years ago
[BugFix] Fix test_phi3v.py (#5725)
Chang Su committed over 2 years ago
[Kernel] Add punica dimension for Qwen2 LoRA (#5441)
Jinzhen Lin committed over 2 years ago