A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 17:00 UTC
main
[Kernel/Model] Migrate mamba_ssm and causal_conv1d kernels to vLLM (#7651)
Mor Zusman committed about 2 years ago
Stas Bekman committed about 2 years ago
[TPU] Upgrade PyTorch XLA nightly (#7967)
Woosuk Kwon committed about 2 years ago
rasmith committed about 2 years ago
[Core][Kernels] Use FlashInfer backend for FP8 KV Cache when available. (#7798)
Pavani Majety committed about 2 years ago
[Bugfix][VLM] Fix incompatibility between #7902 and #7230 (#7948)
Cyrus Leung committed about 2 years ago
[Doc] fix the autoAWQ example (#7937)
Stas Bekman committed about 2 years ago
[ci][test] fix pp test failure (#7945)
youkaichao committed about 2 years ago
[Performance] Enable chunked prefill and prefix caching together (#7753)
Cody Yu committed about 2 years ago
[Core] Async_output_proc: Add virtual engine support (towards pipeline parallel) (#7911)
Alexander Matveev committed about 2 years ago
[mypy][CI/Build] Fix mypy errors (#7929)
Cyrus Leung committed about 2 years ago
bnellnm committed about 2 years ago
[hardware][rocm] allow rocm to override default env var (#7926)
youkaichao committed about 2 years ago
[Core][VLM] Stack multimodal tensors to represent multiple images within each prompt (#7902)
Peter Salas committed about 2 years ago
[mypy] Enable mypy type checking for `vllm/core` (#7229)
Jonathan Berkhahn committed about 2 years ago
[Model] Add multi-image input support for LLaVA-Next offline inference (#7230)
zifeitong committed about 2 years ago
[benchmark] Update TGI version (#7917)
Philipp Schmid committed about 2 years ago
[Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766)
Dipika Sikka committed about 2 years ago
[cuda][misc] error on empty CUDA_VISIBLE_DEVICES (#7924)
youkaichao committed about 2 years ago
[Bugfix] Fix phi3v incorrect image_idx when using async engine (#7916)
Isotr0py committed about 2 years ago