A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Misc] Support bench serve long context (#24373)
Ming Yang committed about 1 year ago
[Doc] mention fpdb for multiprocess breakpoints (#24452)
Mickaël Seznec committed about 1 year ago
[Misc] Improve Worker process title and logging prefix (#22205)
22quinn committed about 1 year ago
[Bugfix] Fix Apertus HF repo name (#24447)
Cyrus Leung committed about 1 year ago
Add data_parallel_size to VllmConfig string representation (#24298)
cong-meta committed about 1 year ago
[Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673)
Zebing Lin committed about 1 year ago
[Flashinfer] Support Flashinfer TRTLLM FP8-qkv BF16/FP16-out Attention Kernel (#23647)
elvischenv committed about 1 year ago
[BugFix][Model] Fix Ernie4.5-VL hanging on long inputs (#24074)
CSWYF3634076 committed about 1 year ago
Move `KVTransferConfig` from `config/__init__.py` to `config/kv_transfer.py` (#24434)
Harry Mellor committed about 1 year ago
Bump actions/github-script from 7.0.1 to 8.0.0 (#24413)
dependabot[bot] committed about 1 year ago
Bump actions/stale from 9.1.0 to 10.0.0 (#24412)
dependabot[bot] committed about 1 year ago
[doc] update `vllm serve` cli args documentation (#24329)
cjackal committed about 1 year ago
Bump actions/setup-python from 5.4.0 to 6.0.0 (#24414)
dependabot[bot] committed about 1 year ago
[gpt-oss] Harmony changes with container tool support (#23386)
zhiweiz committed about 1 year ago
[Bugfix][Wide EP] Fix redundant work when using DeepEP, TP Attn, and EP MoE (#24134)
Tyler Michael Smith committed about 1 year ago
Update reviewers for modelopt related files (#24468)
Zhiyu committed about 1 year ago
[CI] Add nightly multiarch manifests to dockerhub (#24102)
Sahithi Chigurupati committed about 1 year ago
[Hardware][IBM Z] Fix Outlines Core issue for s390x (#24034)
R3hankhan committed about 1 year ago
[Bugfix] Disable the statslogger if the api_server_count is greater than 1 (#22227)
Chauncey committed about 1 year ago
[Attention] FlashAttention MLA cudagraph support (#23958)
Matthew Bonanni committed about 1 year ago