A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Bugfix] Fix task field initialization when PYTHONOPTIMIZE is enabled (#23718)
cndoit18 committed about 1 year ago
[model] Support MiniCPM-V 4.5 (#23586)
tc-mb committed about 1 year ago
[Docs] Fix a 1-2-3 list and style issues in tpu.md (#23729)
Michael Yao committed about 1 year ago
[XPU] Fix OOM issue for data parallel with Ray backend (#22500)
Fanli Lin committed about 1 year ago
[CI/Build] Reduce LoRA layer test cases (#23721)
Jee Jee Li committed about 1 year ago
[FlashInfer] Cache hyper params in metadata builder (#23732)
Woosuk Kwon committed about 1 year ago
[Misc] Move CpuGpuBuffer to vllm/v1/utils.py (#23728)
Woosuk Kwon committed about 1 year ago
Optimize input preparation for FlashInfer [2/N] (#23174)
Woosuk Kwon committed about 1 year ago
[Docs] Fix an admonition important (#23726)
Michael Yao committed about 1 year ago
[CI/Build] Remove redundant register in model init tests (#23715)
Cyrus Leung committed about 1 year ago
[XPU]fix cuda event used in XPU model runner (#23708)
Kunshang Ji committed about 1 year ago
[Core] Use key-only cache for `BaseMultiModalProcessor` (#23018)
Cyrus Leung committed about 1 year ago
[Bugfix] fix when config.yaml config value is list parse error (#23528)
rongfu.leng committed about 1 year ago
[CI/Build] Remove redundant LoRA model tests (#23706)
Jee Jee Li committed about 1 year ago
[XPU] Add xpu torch.compile support (#22609)
Kunshang Ji committed about 1 year ago
[Quantization] Expand compressed-tensors MoE matching logic to support NFP4 + FP8 MoEs (#22674)
Dipika Sikka committed about 1 year ago
[Frontend] Optimize beam search performance by limiting concurrency (#23599)
Chen Zhang committed about 1 year ago
[Frontend] Add --log-error-stack to print stack trace for error response (#22960)
Chen Zhang committed about 1 year ago
[Model] Add Ernie4.5 VL Model Support (#22514)
CSWYF3634076 committed about 1 year ago
[Core] Asynchronous h2d in merge_multimodal_embeddings via pinned memory. (#23686)
Chenheli Hua committed about 1 year ago