[Attention] Update to lastest FA3 code (#13111)

Lucas Wilkinson committed over 1 year ago

[P/D][V1] KV Connector API V1 (#15960)

Yihua Cheng committed over 1 year ago

[MLA] Simplification to batch P/D reordering (#16673)

Nick Hill committed over 1 year ago

[V1][Metrics] Fix http metrics middleware (#15894)

Mark McLoughlin committed over 1 year ago

[ROCM] enable aiter fused moe kernel for llama4 bf16 checkpoints (#16674)

Sijia(Jackson) Chen committed over 1 year ago

[V1][TPU] Enable Top K (#15489)

Nicolò Lucchesi committed over 1 year ago

[TPU][V1] Fix padding recompilation when `max-num-batched-tokens` is not even (#16726)

Nicolò Lucchesi committed over 1 year ago

[Model] use AutoWeightsLoader for olmoe,opt,orion,persimmon,phi3_small (#16548)

rongfu.leng committed over 1 year ago

[Kernel] Add fp8_w8a8 fused MoE kernel tuning configs for DeepSeek V3/R1 on NVIDIA H20 (#16753)

Ximingwang-09 committed over 1 year ago

[Bugfix]Fix index out of range error in api server log (#16787)

Robin committed over 1 year ago

[V1][Perf] Faster incremental detokenization (#15137)

Nick Hill committed over 1 year ago

[Doc] Changed explanation of generation_tokens_total and prompt_tokens_total counter type metrics to avoid confusion (#16784)

Insu Kim committed over 1 year ago

[Doc] Document Matryoshka Representation Learning support (#16770)

wang.yuqi committed over 1 year ago

[Doc] Make sure to update vLLM when installing latest code (#16781)

Cyrus Leung committed over 1 year ago

fix: hyperlink (#16778)

Reid committed over 1 year ago

Improve configs - `TokenizerPoolConfig` + `DeviceConfig` (#16603)

Harry Mellor committed over 1 year ago

[Misc] refactor examples series - lmcache (#16758)

Reid committed over 1 year ago

[Doc] Fix a 404 link in installation/cpu.md (#16773)

Michael Yao committed over 1 year ago

[Bugfix] Fix GLM4 model (#16618)

intervitens committed over 1 year ago

[doc] add open-webui example (#16747)

Reid committed over 1 year ago