A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 13, 07:00 UTC
main
[CI/Build] Install uv for AMD MI300: Language Models Tests (Hybrid) %N (#28142)
amdfaa committed 12 months ago
[ROCm][BugFix]Fix `get_cu_count` in rocm_aiter_fa.py (#28618)
Pleaplusone committed 12 months ago
Akash kaothalkar committed 12 months ago
[Bugfix] Fix FPS value type for Qwen2.5-Omni video processing (#28630)
Fanli Lin committed 12 months ago
[XPU] add sym params to IPEXConfig (#28611)
zofia committed 12 months ago
Fix io processor pooling #28273 (#28484)
baonudesifeizhai committed 12 months ago
[BugFix] Fix type error when assign a trition kernel tensor to a torch.nn.Parameter (#28603)
Zijing Liu committed 12 months ago
Fix: Correctly filter special tokens in benchmark_prefix_caching (#28615)
Di Wu committed 12 months ago
Support all interleaved layer types (#28485)
Yong Hoon Shin committed 12 months ago
[Frontend] supports interleaved thinking (#28531)
Chauncey committed 12 months ago
[Bugfix] fix kimi-linear crash (#28445)
Jiangyun Zhu committed 12 months ago
Add NUMA node validation for CPU thread binding (#28555)
usberkeley committed about 1 year ago
Use official xformers-0.0.33 built for PT 2.9 (#28600)
Huy Do committed about 1 year ago
[Bugfix] Prevent crash on empty grammar string (#28210)
tjandy98 committed about 1 year ago
[ROCm][BugFix] Remove the usage of `device_info` from aiter (#28383)
Pleaplusone committed about 1 year ago
[build][cmake]: Bundle static ACL and torch libgomp for CPU extension builds (#28059)
Radu Salavat committed about 1 year ago
Lucia Fang committed about 1 year ago
[XPU] Support Triton path for LoRA operations on XPU (#28511)
Fanli Lin committed about 1 year ago
[BugFix][ROCm] Fix `get_cu_count` missing variable error (#28608)
Pleaplusone committed about 1 year ago
Andrew Xia committed about 1 year ago