Allocate kv_cache with stride order (#16605)

Shu Wang committed over 1 year ago

[Minor][Models] Fix Return Types of Llama & Eagle (#17220)

Woosuk Kwon committed over 1 year ago

[Doc] Minor fix for the vLLM TPU setup page (#17206)

yarongmu-google committed over 1 year ago

[Minor][Spec Decode] Add use_eagle to SpeculativeConfig (#17213)

Woosuk Kwon committed over 1 year ago

[doc] add Anything LLM integration (#17216)

Reid committed over 1 year ago

[MISC][AMD] Add unused annotation to rocm kernel file (#17097)

Lu Fang committed over 1 year ago

[Bugfix] [pytorch] Patch AOTAutogradCache._get_shape_env (#17142)

James Wu committed over 1 year ago

[v1] [P/D] Adding LMCache KV connector for v1 (#16625)

Yihua Cheng committed over 1 year ago

[AMD][FP8][BugFix] Remove V1 check in arg_utils.py for FP8 since it is not necessary (#17215)

rasmith committed over 1 year ago

[Bugfix] gemma[2,3] interleaved attention when sliding window is disabled (#17180)

Chen Zhang committed over 1 year ago

[Misc] Refine ray_serve_deepseek example (#17204)

Rui Qiao committed over 1 year ago

[V1][Spec Decode] EAGLE-3 Support (#16937)

Benjamin Chislett committed over 1 year ago

[BugFix][Frontend] Fix `LLM.chat()` tokenization (#16081)

Nick Hill committed over 1 year ago

Fix Python packaging edge cases (#17159)

Christian Heimes committed over 1 year ago

[Bugfix] Fix hybrid model tests (#17182)

Cyrus Leung committed over 1 year ago

[V1] Move usage stats to worker and start logging TPU hardware (#16211)

Daniel Li committed over 1 year ago

[Security] Use safe serialization and fix zmq setup for mooncake pipe (#17192)

Russell Bryant committed over 1 year ago

[Misc] Inline Molmo requirements (#17190)

Cyrus Leung committed over 1 year ago

[doc] update wrong hf model links (#17184)

Reid committed over 1 year ago

Use Transformers helper `get_text_config()` instead of checking for `text_config` (#17105)

Harry Mellor committed over 1 year ago