[doc] add "Failed to infer device type" to faq (#14200)

youkaichao committed over 1 year ago

[sleep mode] error out with expandable_segments (#14189)

youkaichao committed over 1 year ago

[platform] add debug logging during inferring the device type (#14195)

youkaichao committed over 1 year ago

Fix benchmark_moe.py tuning for CUDA devices (#14164)

Michael Goin committed over 1 year ago

Use math.prod instead of np.prod for trivial ops (#14142)

Zhanwen Chen committed over 1 year ago

[core] Pass all driver env vars to ray workers unless excluded (#14099)

Rui Qiao committed over 1 year ago

[Misc] Remove lru_cache in NvmlCudaPlatform (#14156)

Cody Yu committed over 1 year ago

[core] moe fp8 block quant tuning support (#14068)

Divakar Verma committed over 1 year ago

[Model] Add support for GraniteMoeShared models (#13313)

Travis Johnson committed over 1 year ago

[v1] Add comments to the new ragged paged attention Pallas kernel (#14155)

iefgnoix committed over 1 year ago

[Docs] Add GPTQModel (#14056)

Qubitium-ModelCloud committed over 1 year ago

[Kernel] Optimize moe intermediate_cache usage (#13625)

Michael Goin committed over 1 year ago

[Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (#14100)

Michael Goin committed over 1 year ago

[WIP][[V1][Metrics] Implement max_num_generation_tokens, request_params_n, and request_params_max_tokens metrics (#14055)

Mark McLoughlin committed over 1 year ago

[V1] Simplify stats logging (#14082)

Nick Hill committed over 1 year ago

[V0][Metrics] Deprecate some KV/prefix cache metrics (#14136)

Mark McLoughlin committed over 1 year ago

[V0][Metrics] Remove unimplemented `vllm:tokens_total` (#14134)

Mark McLoughlin committed over 1 year ago

Fix `head_dim` not existing in all model configs (Transformers backend) (#14141)

Harry Mellor committed over 1 year ago

[ROCm] Faster Custom Paged Attention kernels (#12348)

TJian committed over 1 year ago

Improve the docs for `TransformersModel` (#14147)

Harry Mellor committed over 1 year ago