A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 04:00 UTC
main
[compile] raise on compile_size implicit padding (#32343)
dolpm committed 10 months ago
[BugFix] Assign page_size_padded when unifying kv cache spec. (#32283)
Lumosis committed 10 months ago
vllmellm committed 10 months ago
[MODEL] Fix handling of multiple channels for gpt-oss with speculative decoding (#26291)
Aleksandr Samarin committed 10 months ago
[CI] Move rixl/ucx from Dockerfile.rocm_base to Dockerfile.rocm (#32295)
qli88 committed 10 months ago
[1/N] Reorganize multimodal processing code (#32327)
Cyrus Leung committed 10 months ago
rename tokenize serving api request id prefix to tokenize (#32328)
Ning Xie committed 10 months ago
[Frontend] track responsesAPI server_load (#32323)
Chauncey committed 10 months ago
[Misc] Make mem utils can be reused by other platforms (#32322)
Shanshan Shen committed 10 months ago
[Frontend] Standardize use of `create_error_response` (#32319)
Cyrus Leung committed 10 months ago
[Refactor] [9/N] to simplify the vLLM openai translations serving ar chitecture (#32313)
Chauncey committed 10 months ago
[Refactor] Move top-level dummy data generation to registry (#32310)
Cyrus Leung committed 10 months ago
[Model] Re-implement Qwen3Omni Audio Encoder (#32167)
Roger Wang committed 10 months ago
Add Molmo2 multimodal model support (#30997)
sangho.lee committed 10 months ago
[Refactor] [8/N] to simplify the vLLM openai responsesapi_serving architecture (#32260)
Chauncey committed 10 months ago
[Docs] Add docs about OOT Quantization Plugins (#32035)
Michael Goin committed 10 months ago
AMD CI Test - unskip moe_sum test and moe_align_block_size tests (#32039)
Hongxia Yang committed 10 months ago
[misc] Remove is_torch_equal_or_newer(2.4) cases (#32296)
Angela Yi committed 10 months ago
[Build] Relax anthropic version pin from ==0.71.0 to >=0.71.0 (#32289)
David committed 10 months ago
[ROCm][CI] Handle missing vision_config in Isaac model attention patch (#32281)
Andreas Karatzas committed 10 months ago