[Feature]:Allow for Granite MoE Hybrid models with _only_ shared experts. (#19652)

Shawn Tan committed over 1 year ago

[DOC] Add reasoning capability to vLLM streamlit code (#19557)

Navanit Dubey committed over 1 year ago

[BugFix] Don't catch BaseException when dumping execute_model errors (#19626)

Nick Hill committed over 1 year ago

[Kernel] GGUF MMVQ kernel for multiple input vectors (#18754)

Szymon Ożóg committed over 1 year ago

[Docs] Move multiproc doc to v1 dir (#19651)

Russell Bryant committed over 1 year ago

[CI] Add mteb testing for rerank models (#19344)

wang.yuqi committed over 1 year ago

[MISC] typo fix (#19672)

Ning Xie committed over 1 year ago

[TPU] support attention head dim smaller than 128 (#19620)

Chengji Yao committed over 1 year ago

[Misc] Fix skipped max-model-len validation when deriving max model length from tokenizer config (#19660)

Ye (Charlotte) Qi committed over 1 year ago

[Misc][Frontend] passthrough `bad_words` (#19564)

Francesco Bertolotti committed over 1 year ago

[Bugfix][Core] Prefix caching causes incorrect outputs due to outdated ComputedBlocksTracker (#18957)

quanliu committed over 1 year ago

[MISC] Remove unused variableds in C++ (#19609)

Lu Fang committed over 1 year ago

[Misc] Remove duplicate multiproc method setting for CPU platform (#19649)

Isotr0py committed over 1 year ago

[CI/Build] Fix torch nightly CI dependencies part 2 (#19589)

Richard Zou committed over 1 year ago

Enable prefix caching with full cuda graphs (#19617)

Woosuk Kwon committed over 1 year ago

[Benchmark] Refactor benchmark script for fp8 & int8 (#19627)

Wentao Ye committed over 1 year ago

[Kernel] Raise verbose error and consolidate `num_heads/num_kv_heads` divisibility check (#19339)

22quinn committed over 1 year ago

[Bugfix][2/n] Fix speculative decoding CI - Fix test_ngram_e2e_greedy_correctness (#19644)

Lu Fang committed over 1 year ago

[Perf] Further tunings for SM100 FP8 CUTLASS kernel (#19566)

Ilya Markov committed over 1 year ago

[Fix] Convert kv_transfer_config from dict to KVTransferConfig (#19262)

maobaolong committed over 1 year ago