[Bugfix][Model] Validate runai_streamer model_loader_extra_config (#45291)

Ting SUN committed 5 months ago

Added real /v1/embeddings support for messages + chat_template_kw (#45173)

Taneem Ibrahim committed 5 months ago

[BugFix] Fix prompt_embeds for multimodal models (#45383)

Michael Ma committed 5 months ago

[Perf] Use bisect for mm feature lookup in model runner v2 (#45566)

Roger Wang committed 5 months ago

[ROCm][Perf] Enable W4A16 FlyDSL MoE (#44400)

Amanzhol Salykov committed 5 months ago

[XPU] Support int4 group_size=32 W4A16 MoE (#45136)

Marceli Fylcek committed 5 months ago

[Bugfix][DCP] Fix illegal memory access in DCP a2a decode under full CUDA graphs (#45487)

Jeff (Junze) Ma committed 5 months ago

[V1][Spec Decode] Add Dynamic SD (#32374)

Ekagra Ranjan committed 5 months ago

[Build] Fix CUDA arch build coverage gaps (#45277)

Shengqi Chen committed 5 months ago

[Bugfix] nightly Docker images crash with ImportError: AnthropicOutputConfig since May 28 (#44795)

achyuthan.s committed 5 months ago

[Core] Simplify MRV2 async output handling (#45442)

Nick Hill committed 5 months ago

[Perf] SM90 cutlass fp8 mm supports odd M by swap_ab, 180~290% kernel performance improvement (#44572)

Wentao Ye committed 5 months ago

[Bugfix] Reject structured outputs for diffusion decoders with a clear error (#45468)

WEI CHENG CHIU committed 5 months ago

Fix docs build on `main` (#45536)

Harry Mellor committed 5 months ago

[Security] Fix DoS via prompt_embeds on M-RoPE models (#45252)

Juan Pérez de Algaba committed 5 months ago

[Security] Add timeout guard for regex compilation in structured outp… (#45118)

Juan Pérez de Algaba committed 5 months ago

[CI Bug] Fix `ValueError: There is no module or parameter named 'model.vision_tower.vision_model'` (#45478)

Wentao Ye committed 5 months ago

Treat null completion max_tokens like the default (#45491)

Andreas Karatzas committed 5 months ago

[Model Runner V2] Fix `openai.InternalServerError: Error code: 500 - 'list index out of range'` (#45467)

Wentao Ye committed 5 months ago

[Doc] Fix uv dependency resolution failure for setuptools during CPU source builds (x86 & ARM) (#45412)

midas committed 5 months ago