A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 19:30 UTC
main
[CI/Build] bump minimum cmake version (#6999)
Daniele committed over 2 years ago
[Bugfix] Handle PackageNotFoundError when checking for xpu version (#7398)
sasha0552 committed over 2 years ago
[Core/Bugfix] Add FP8 K/V Scale and dtype conversion for prefix/prefill Triton Kernel (#7208)
jon-chuang committed over 2 years ago
[Core] Consolidate `GB` constant and enable float GB arguments (#7416)
Cyrus Leung committed over 2 years ago
[Misc] Use scalar type to dispatch to different `gptq_marlin` kernels (#7323)
Lucas Wilkinson committed over 2 years ago
[ci] Cancel fastcheck when PR is ready (#7433)
Kevin H. Luu committed over 2 years ago
[ci] Cancel fastcheck run when PR is marked ready (#7427)
Kevin H. Luu committed over 2 years ago
[ci] Entrypoints run upon changes in vllm/ (#7423)
Kevin H. Luu committed over 2 years ago
[Bugfix] Fix logit soft cap in flash-attn backend (#7425)
Woosuk Kwon committed over 2 years ago
[CI/Build] bump Dockerfile.neuron image base, use public ECR (#6832)
Daniele committed over 2 years ago
[Frontend] Disallow passing `model` as both argument and option (#7347)
Cyrus Leung committed over 2 years ago
[Core][VLM] Support image embeddings as input (#6613)
Roger Wang committed over 2 years ago
[Kernel] Flashinfer correctness fix for v0.1.3 (#7319)
Lily Liu committed over 2 years ago
[CI/Build] Minor refactoring for vLLM assets (#7407)
Roger Wang committed over 2 years ago
Simon Mo committed over 2 years ago
[misc] add commit id in collect env (#7405)
youkaichao committed over 2 years ago
[Doc] building vLLM with VLLM_TARGET_DEVICE=empty (#7403)
tomeras91 committed over 2 years ago
[CI/Build] build on empty device for better dev experience (#4773)
tomeras91 committed over 2 years ago
[core] [2/N] refactor worker_base input preparation for multi-step (#7387)
William Lin committed over 2 years ago
Updating LM Format Enforcer version to v0.10.6 (#7189)
Noam Gat committed over 2 years ago