[ROCm][Quantization] add fp8xfp8 attn support for rocm_aiter_unified_attn (#36927)

Divakar Verma committed 8 months ago

[Kernel] Add non-gated support for NVFP4 CUTLASS MoE (#37320)

Michael Goin committed 8 months ago

[ROCm] Validate block_size for explicitly selected attention backends (#36846)

Andreas Karatzas committed 8 months ago

[Bugfix][ROCm] Fix worker startup OOM on ROCm by skipping unreliable cudagraph memory profiling (#36720)

JartX committed 8 months ago

Make KV connector metadata build overridable via plugin (#37336)

Yong Hoon Shin committed 8 months ago

[Model] Add ColQwen3.5 4.5B support (#36887)

Athrael Soju committed 8 months ago

[Models] Cohere ASR (#35809)

Ekagra Ranjan committed 8 months ago

[Bugfix] Rescale NVFP4 weight scales to fix BF16 dequant underflow (#34577)

Chao-Ju Chen committed 8 months ago

[Bugfix] Fix mock.patch resolution failure for standalone_compile.FakeTensorMode on Python <= 3.10 (#37158)

Dimitrios Bariamis committed 8 months ago

[Perf] Set Flashinfer sparse MLA as default backend for FP8 kv cache (#37252)

Wei Zhao committed 8 months ago

[Deprecation] Deprecate `--calculate-kv-scales` option (#37201)

Michael Goin committed 8 months ago

[Model] Remove unused `handle_oov_mm_token` (#37321)

Cyrus Leung committed 8 months ago

bump compressed-tensors version to 0.14.0.1 (#36988)

Brian Dellabetta committed 8 months ago

[Bug] Fix FlashInfer MNNVL socket collisions under concurrent vLLM jobs (#36674)

Wentao Ye committed 8 months ago

[Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673)

Andrey Talman committed 8 months ago

[CI] Split Distributed Tests (4 GPUs) and Kernel MoE tests (#37100)

Avinash Singh committed 8 months ago

[Perf] Optimize top-k search in apply_top_k_top_p_triton sampler (#37225)

Michael Goin committed 8 months ago

[CI] Fix GPU memory leak when RemoteOpenAIServer fails to start in __init__ (#37230)

Andreas Karatzas committed 8 months ago

[Bugfix] Standardize custom HF Processor init (#37289)

Cyrus Leung committed 8 months ago

[BugFix] PyTorch Compilation Tests should error if any test fails (#37300)

Richard Zou committed 8 months ago