A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 7, 09:00 UTC
main
[ROCm] [Release] Change the package from `aiter` to `amd-aiter` (#35198)
TJian committed 8 months ago
Fix TYPE_CHECKING stub defaults in envs.py to match actual runtime defaults (#35645)
lin-shh committed 8 months ago
[Misc] Fix typos in comments: explict→explicit, paramaters→parameters (#35648)
lin-shh committed 8 months ago
[MISC] Removed unused function find_all_indices() from tool_parsers/utils.py (#35683)
Taneem Ibrahim committed 8 months ago
[BugFix] Fix cmake based incremental install (wrong vllm install dir) (#35773)
Lucas Wilkinson committed 8 months ago
[CI/Build] Trigger processor tests on registry update (#35824)
Cyrus Leung committed 8 months ago
[ROCm][CI] Fix Assertion Logic For `test_gpt_oss` (#35806)
Micah Williamson committed 8 months ago
[Refactor] Fix maxsim cuda platform and add cli to control it (#35427)
Wentao Ye committed 8 months ago
[CI/Build] Automatically patch video metadata for multimodal processor test (#35822)
Isotr0py committed 8 months ago
[Core] Add optional flags to check for repetitive token patterns in engine output (#35451)
aykoppol committed 8 months ago
[Model Runner V2] Use ModelState.prepare_attn() for cuda graph capture [5/N] (#35774)
Woosuk Kwon committed 8 months ago
[V0 deprecation] Remove Swin model (#35821)
Isotr0py committed 8 months ago
[Docs][Model Runner V2] Add Design Docs (#35819)
Woosuk Kwon committed 8 months ago
[ModelRunnerV2] Rename sampler functions and variables for clarity (#35459)
Andy Lo committed 8 months ago
[CI] Temporarily Disable Nightly Failures (#35770)
Robert Shaw committed 8 months ago
[Tool Parser] Fix Qwen3Coder streaming parameter loss with speculative decode (#35615)
Martin Vit committed 8 months ago
[Model Runner V2][Perf] align dummy_run tokens to uniform decode for dp cudagraph (#35376)
zhrrr committed 8 months ago
[XPU][NIXL] Add GPUDirect RDMA support for XPU (#35270)
liuzhenwei committed 8 months ago
[Model] Add support for nvidia/llama-nemotron-rerank-vl-1b-v2 (#35735)
Jakub Zakrzewski committed 8 months ago
Andreas Karatzas committed 8 months ago