A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 04:30 UTC
main
[V1][VLM] Fix edge case bug for InternVL2 (#11165)
Roger Wang committed almost 2 years ago
[Frontend] Separate pooling APIs in offline inference (#11129)
Cyrus Leung committed almost 2 years ago
[Bugfix][CI][CPU] add missing datasets package to requirements-cpu.txt (#11159)
Li, Jiang committed almost 2 years ago
Jani Monoses committed almost 2 years ago
[core] clean up cudagraph batchsize padding logic (#10996)
youkaichao committed almost 2 years ago
[Bugfix][V1] Fix 'NoneType' object has no attribute 'hash_value' (#11157)
Cody Yu committed almost 2 years ago
[ROCm][AMD] Disable auto enabling chunked prefill on ROCm (#11146)
Gregory Shtrasberg committed almost 2 years ago
[Bugfix] Update starcoder2 to remap k/v scale names for kv_cache quantization (#11148)
Dipika Sikka committed almost 2 years ago
[Bugfix] Use runner_type instead of task in GritLM (#11144)
Pooya Davoodi committed almost 2 years ago
[torch.compile] Dynamic fp8 + rms_norm fusion (#10906)
Luka Govedič committed almost 2 years ago
[Misc][V1] Fix type in v1 prefix caching (#11151)
Cody Yu committed almost 2 years ago
[Bugfix] Fix value unpack error of simple connector for KVCache transfer. (#11058)
shangmingc committed almost 2 years ago
Fix logging of the vLLM Config (#11143)
Jeremy Arnold committed almost 2 years ago
[Misc] Validate grammar and fail early (#11119)
Cody Yu committed almost 2 years ago
[Bugfix] Quick fix to make Pixtral-HF load correctly again after 39e227c7ae. (#11024)
Jeff Cook committed almost 2 years ago
fix(docs): typo in helm install instructions (#11141)
Ramon Ziai committed almost 2 years ago
[V1] Fix torch profiling for offline inference (#11125)
Roger Wang committed almost 2 years ago
[Misc][LoRA] Ensure Lora Adapter requests return adapter name (#11094)
Jiaxin Shan committed almost 2 years ago
[core][distributed] initialization from StatelessProcessGroup (#10986)
youkaichao committed almost 2 years ago
[Hardware][Intel-Gaudi] Enable LoRA support for Intel Gaudi (HPU) (#10565)
Sanju C Sudhakaran committed almost 2 years ago