[MISC] fix pin_memory=torch.cuda.is_available(), use is_pin_memory_available (#37415)

Kunshang Ji committed 8 months ago

[CI] Fix wrong path test file, missing `rlhf_async_new_apis.py` (#37532)

TJian committed 8 months ago

Support temporal compression for Nemotron-3-VL videos (#36808)

Collin McCarthy committed 8 months ago

[Perf] Fix slow hasattr in CUDAGraphWrapper.__getattr__ (#37425)

Ziming Huang committed 8 months ago

fix(anthropic): remove non-standard 'data: [DONE]' from Anthropic streaming (#37510)

cdpath committed 8 months ago

[SSM/Mamba] Follow-up: N-1 prefill for P/D disaggregation (#37310)

zhanqiuhu committed 8 months ago

[Refactor] Relocate endpoint tests to mirror serving code directory structure (#37504)

Flora Feng committed 8 months ago

[ROCm] issue management - request information for bug issues on ROCm (#37009)

Hongxia Yang committed 8 months ago

[Perf] Optimize token_embed for pooling models, 1.0% token throughput improvement (#37347)

Wentao Ye committed 8 months ago

[bug] Fix deadlock with pause resume and collective_rpc (#37024)

Aaron Hao committed 8 months ago

[Model Runner V2] Spec decode rejection sampler logprobs support (#37237)

Giancarlo Delfin committed 8 months ago

[BUG] Exclude SKIP_TENSORS from get_layer_size() + new weight sync example for dpep (#37334)

Aaron Hao committed 8 months ago

[EPLB] Simplify EPLB rearrange by only returning one map (#36267)

Sage Moore committed 8 months ago

[Bugfix] Zero-init MLA attention output buffers to prevent NaN from CUDA graph padding (#37442)

Elvir Crnčević committed 8 months ago

[XPU]Unify xpu test dependencies in dockerfile.xpu (#36477)

sihao_li committed 8 months ago

[Bugfix] Expand quantization method support in perf metrics (#37231)

Thillai Chithambaram committed 8 months ago

[Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype "auto" leading to gibberish (#37054)

Andy Lo committed 8 months ago

[Model Runner V2] Spec decode rejection sampler greedy support (#37238)

Giancarlo Delfin committed 8 months ago

[Bugfix] Remove assertion for NVFP4 scale dynamic range (#37465)

Michael Goin committed 8 months ago

[LoRA][BugFix] Fix skipped LoRA adapters for Mistral3 (#36928)

Woosuk Kwon committed 8 months ago