A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 07:00 UTC
main
[Frontend] Add per-request number of cached token stats (#10174)
zifeitong committed about 2 years ago
[Bugfix] Fix QwenModel argument (#10262)
Jie Fu (傅杰) committed about 2 years ago
[Misc]Fix Idefics3Model argument (#10255)
Jee Jee Li committed about 2 years ago
Guillaume Calmettes committed about 2 years ago
[CI][CPU]refactor CPU tests to allow to bind with different cores (#10222)
Yuan committed about 2 years ago
[doc] explain the class hierarchy in vLLM (#10240)
youkaichao committed about 2 years ago
Splitting attention kernel file (#10091)
Aleksandr Malyshev committed about 2 years ago
[Misc][LoRA] Replace hardcoded cuda device with configurable argument (#10223)
Jee Jee Li committed about 2 years ago
[1/N] torch.compile user interface design (#10237)
youkaichao committed about 2 years ago
[Doc] Update help text for `--distributed-executor-backend` (#10231)
Russell Bryant committed about 2 years ago
[doc] update debugging guide (#10236)
youkaichao committed about 2 years ago
[V1] `AsyncLLM` Implementation (#9826)
Robert Shaw committed about 2 years ago
Make shutil rename in python_only_dev (#10233)
Nikolai Shcheglov committed about 2 years ago
[V1] Enable custom ops with piecewise CUDA graphs (#10228)
Woosuk Kwon committed about 2 years ago
[misc][distributed] auto port selection and disable tests (#10226)
youkaichao committed about 2 years ago
Add docs on serving with Llama Stack (#10183)
Yuan Tang committed about 2 years ago
[V1] Use custom ops for piecewise CUDA graphs (#10227)
Woosuk Kwon committed about 2 years ago
[v1][torch.compile] support managing cudagraph buffer (#10203)
youkaichao committed about 2 years ago
[V1] Do not use inductor for piecewise CUDA graphs (#10225)
Woosuk Kwon committed about 2 years ago
[V1] Fix detokenizer ports (#10224)
Woosuk Kwon committed about 2 years ago