gpu_model_runner: Cache is_encoder_decoder from model config (#35099)

pschlan-amd committed 9 months ago

[Quantization] Support FP8 MoE bias for models like GPT-OSS (#34906)

Jia Guo committed 9 months ago

[ROCm] AITER fused RoPE+KVCache (#33443)

Rohan Potdar committed 9 months ago

[Mamba1] - Change supports_update_block_table to True (#35054)

Asaf Gardin committed 9 months ago

[Bugfix] Fix lora_ids in FusedMoE LoRA test (#35135)

Xin Yang committed 9 months ago

[Perf] Enable FlashInfer DeepGEMM swapAB on SM90 by default (#34924)

Michael Goin committed 9 months ago

[Bugfix] Fix DSV3 kernels breaking _C and _moe_C on unsupported arches (#35123)

Michael Goin committed 9 months ago

[Perf] Improve default triton fused moe configs (#34846)

Michael Goin committed 9 months ago

[RL] Validation for pause_mode='keep' (#34992)

Aaron Hao committed 9 months ago

[Misc] Monitor interface changes (#35113)

Nicolò Lucchesi committed 9 months ago

Enforce that `model` is the first positional arg when `--served-model-name` is used (#34973)

Harry Mellor committed 9 months ago

Fix custom processors that use deleted import for Transformers v5 (#35101)

Harry Mellor committed 9 months ago

[Bugfix] Fix prefix caching for Mamba 'all' mode (Nemotron models) (#34874)

haosdent committed 9 months ago

Use Xet high performance mode for Transformers v5 (#35098)

Harry Mellor committed 9 months ago

[Bugfix] Fix MRotaryEmbedding missing `truncate` attr with YaRN scaling (#35080)

haosdent committed 9 months ago

[CI] Skip Responses API (#34990)

Robert Shaw committed 9 months ago

[Metrics] Add Prometheus counters for Model FLOPs Utilization (MFU) (#30950)

Mark McLoughlin committed 9 months ago

[kv-cache, ct] Use compressed-tensors as a source of ground-truth for quant strategies (#34254)

Eldar Kurtić committed 9 months ago

[Refactor] Decouple TimingContext from InputProcessingContext (#35083)

Cyrus Leung committed 9 months ago

[Llama4,CI] Bring back Llama-4 bug fixes, and also fix Maverick tests (#35033)

Eldar Kurtić committed 9 months ago