[BugFix] --max-model-len=-1 causes over-limit requests to hang and starve the entire service (#39102)

triangleXIV committed 7 months ago

`tests/v1/e2e/spec_decode`: assert async scheduling is used (#39206)

Rishi Puri committed 7 months ago

[Feature] Batch invariant nvfp4 linear support (#39322)

Wentao Ye committed 7 months ago

[MoE] Move DEEP_GEMM into experts/ subdirectory (#39005)

Jackmin801 committed 7 months ago

[Tool] `adjust_request` to reasoning parser, and Gemma4 fixes (#39027)

Ben Browning committed 7 months ago

[Attention] relax the head dim 512 and paged kv for sm90+FA4 (#38835)

Lain committed 7 months ago

[Perf][Kernel] Persistent TopK scheduler: unified CUDAGraph-safe kernel with dynamic per-row dispatch - DeepSeek-V3.2 DSA decode (#37421)

Roberto L. Castro committed 7 months ago

[Feature] NUMA binding support for GPU workers (#38635)

Shengqi Chen committed 7 months ago

[kv_offload+HMA][5/N]: Track group block hashes and block IDs (#37109)

Or Ozeri committed 7 months ago

[Bugfix] Fix Gemma4 streaming tool call corruption for split boolean/number values (#39114)

Flora Feng committed 7 months ago

[ROCm] Enable fused_silu_mul_block_quant on ROCm (#38817)

Gregory Shtrasberg committed 7 months ago

[Core] Use tuple_return in split_module for tuple-conformant subgraphs (#38752)

Frederik Gossen committed 7 months ago

[ROCm][CI-Build] Cherry pick triton BUFFER_OPS fix and update AITER (#38580)

Gregory Shtrasberg committed 7 months ago

[Bugfix] Fix V1 logprobs empty strings for multi-byte UTF-8 tokens when logprobs > 0 (#34875)

haosdent committed 7 months ago

[Bugfix] Add missing ASRDataset import and CLI args in benchmarks/throughput.py (#38114)

nemanjaudovic committed 7 months ago

[FlashAttention] Symlink FA4 instead of copying when using `VLLM_FLASH_ATTN_SRC_DIR` (#38814)

Matthew Bonanni committed 7 months ago

[CI Failure] pin nomic-embed-text-v1 revision (#39292)

wang.yuqi committed 7 months ago

[Bugfix] Cuda Clean up scales Kvcache fp8/int8_per_token_head (#39224)

JartX committed 7 months ago

[Bugfix][Docs] Fix ReadTheDocs build crash from mocked torch decorator (#39284)

Kevin H. Luu committed 7 months ago

[CI][AMD][BugFix][Kernel] Cast induction variable to int64 on MI350 for chunk_gated_delta_rule_fwd_kernel_h_blockdim64 to avoid illegal memory access (#39087)

rasmith committed 7 months ago