A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 18:15 UTC
main
[Bugfix] Fix xpu build (#7644)
Kunshang Ji committed about 2 years ago
[Core] Use flashinfer sampling kernel when available (#7137)
Peng Guanwen committed about 2 years ago
[Core] Optimize SPMD architecture with delta + serialization optimization (#7109)
SangBin Cho committed about 2 years ago
[Misc] Refactor Llama3 RoPE initialization (#7637)
Woosuk Kwon committed about 2 years ago
[CI/Build] Add text-only test for Qwen models (#7475)
Alex Brooks committed about 2 years ago
[ Bugfix ] Fix Prometheus Metrics With `zeromq` Frontend (#7279)
Robert Shaw committed about 2 years ago
[TPU] Optimize RoPE forward_native2 (#7636)
Woosuk Kwon committed about 2 years ago
[TPU] Use mark_dynamic only for dummy run (#7634)
Woosuk Kwon committed about 2 years ago
[TPU] Skip creating empty tensor (#7630)
Woosuk Kwon committed about 2 years ago
[VLM] Refactor `MultiModalConfig` initialization and profiling (#7530)
Roger Wang committed about 2 years ago
[Misc]Fix BitAndBytes exception messages (#7626)
Jee Jee Li committed about 2 years ago
[ci][test] allow longer wait time for api server (#7629)
youkaichao committed about 2 years ago
[Model] Pipeline parallel support for JAIS (#7603)
Besher Alkurdi committed about 2 years ago
[core][misc] update libcudart finding (#7620)
youkaichao committed over 2 years ago
[ci][test] fix engine/logger test (#7621)
youkaichao committed over 2 years ago
[misc] use nvml to get consistent device name (#7582)
youkaichao committed over 2 years ago
[Bugfix] add >= 1.0 constraint for openai dependency (#7612)
Xander Johnson committed over 2 years ago
[aDAG] Unflake aDAG + PP tests (#7600)
SangBin Cho committed over 2 years ago
[Bugfix] Clear engine reference in AsyncEngineRPCServer (#7618)
Rui Qiao committed over 2 years ago
.[Build/CI] Enabling passing AMD tests. (#7610)
Alexei-V-Ivanov-AMD committed over 2 years ago