A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 03:15 UTC
main
[V1] Simplify Shutdown (#11659)
Robert Shaw committed almost 2 years ago
[Model] Whisper model implementation (#11280)
Aurick Qiao committed almost 2 years ago
[perf-benchmark] Fix dependency for steps in benchmark pipeline (#11710)
Kevin H. Luu committed almost 2 years ago
[Bugfix] Check chain_speculative_sampling before calling it (#11673)
Lu Fang committed almost 2 years ago
Update default max_num_batch_tokens for chunked prefill (#11694)
Sachin Varghese committed almost 2 years ago
[Misc] Minimum requirements for SageMaker compatibility (#11576)
Nathan Azrak committed almost 2 years ago
Resolve race conditions in Marlin kernel (#11493)
wchen61 committed almost 2 years ago
[Bugfix] Change kv scaling factor by param json on nvidia gpu (#11688)
bjmsong committed almost 2 years ago
[V1][Minor] Optimize token_ids_cpu copy (#11692)
Woosuk Kwon committed almost 2 years ago
[Bugfix] Free cross attention block table for preempted-for-recompute sequence group. (#10013)
Kathy Yu committed almost 2 years ago
According to vllm.EngineArgs, the name should be distributed_executor_backend (#11689)
Chunyang Wen committed almost 2 years ago
[VLM] Merged multi-modal processor for LLaVA-NeXT (#11682)
Cyrus Leung committed almost 2 years ago
[mypy] Pass type checking in vllm/inputs (#11680)
Tobias Pitters committed almost 2 years ago
[VLM][Bugfix] Multi-modal processor compatible with V1 multi-input (#11674)
Cyrus Leung committed almost 2 years ago
[VLM] Move supported limits and max tokens to merged multi-modal processor (#11669)
Cyrus Leung committed almost 2 years ago
[V1] Implement Cascade Attention (#11635)
Woosuk Kwon committed almost 2 years ago
Kazuhiro Serizawa committed almost 2 years ago
[Misc] Replace space with - in the file names (#11667)
Lu Fang committed almost 2 years ago
[Misc] Optimize Qwen2-VL LoRA test (#11663)
Jee Jee Li committed almost 2 years ago
[VLM] Add max-count checking in data parser for single image models (#11661)
Cyrus Leung committed almost 2 years ago