A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 03:30 UTC
main
Documentation update tool_calling: mapping back to function from response (#20373)
cronoik-inceptionai committed over 1 year ago
[Model] Adds support for SlimMoE models Phi-tiny-MoE-instruct (#20286)
zichongli5 committed over 1 year ago
[Bugfix] Fix the max_seq_len limit of 16384 for DeepSeek models (#20322)
WangHuaqiang committed over 1 year ago
[NVIDIA] Support Cutlass w8a8 FP8 for Blackwell Geforce GPUs (sm120) (#17280)
Joonchen Liau committed over 1 year ago
[Build/CI] Automatically tag DeepSeek related PRs (#20370)
Lu Fang committed over 1 year ago
[Model] Add Ernie4.5 and Ernie4.5MoE Model Support (#20220)
CSWYF3634076 committed over 1 year ago
[Docs] Make TPU ref prettier in google_tpu.md (#20356)
Michael Yao committed over 1 year ago
[Docs] Fix indentations for 2-level items in deprecation_policy.md (#20352)
Michael Yao committed over 1 year ago
[Bugfix] Keye-VL compatibility with `tok_kwargs` (#20058) (#20353)
Cyrus Leung committed over 1 year ago
[Model][VLM] Support Keye-VL-8B-Preview (#20126)
Kwai-Keye committed over 1 year ago
Chenheli Hua committed over 1 year ago
[TPU] kv cache update kernel supports dynamic grid (#20235)
Chengji Yao committed over 1 year ago
[Doc][TPU] Add models and features supporting matrix. (#20230)
QiliangCui committed over 1 year ago
fix[Docs]: link anchor is incorrect #20309 (#20315)
yyzxw committed over 1 year ago
[Bugfix] Fix dynamic rotary embedding (#20343)
Cyrus Leung committed over 1 year ago
[FIX][Intel GPU]fix ipex flash_attn_varlen_func api missing parameter (#20348)
Kunshang Ji committed over 1 year ago
[Misc][Doc] Add missing comment for LLM (#20285)
Lifans committed over 1 year ago
[Refactor] Remove Unused Env `VLLM_ENABLE_MOE_ALIGN_BLOCK_SIZE_TRITON` (#20334)
Wentao Ye committed over 1 year ago
[Refactor] Remove duplicate `find_free_port` (#20333)
Wentao Ye committed over 1 year ago
[UT][intel GPU] use current_platform instead of device hardcode in v1 tests (#20169)
Liangliang Ma committed over 1 year ago