A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 19:30 UTC
main
[ci] set timeout for test_oot_registration.py (#7082)
youkaichao committed over 2 years ago
[Hardware][Intel CPU] Update torch 2.4.0 for CPU backend (#6931)
Jie Fu (傅杰) committed over 2 years ago
[Core] Comment out unused code in sampler (#7023)
Peng Guanwen committed over 2 years ago
[cuda][misc] remove error_on_invalid_device_count_status (#7069)
youkaichao committed over 2 years ago
Bongwon Jang committed over 2 years ago
[ci][distributed] try to fix pp test (#7054)
youkaichao committed over 2 years ago
[mypy] Speed up mypy checking (#7056)
Cyrus Leung committed over 2 years ago
[Kernel] Fix input for flashinfer prefill wrapper. (#7008)
Lily Liu committed over 2 years ago
[Performance] Optimize `get_seqs` (#7051)
Woosuk Kwon committed over 2 years ago
[CI/Build][Bugfix] Fix CUTLASS header-only line (#7034)
Tyler Michael Smith committed over 2 years ago
[Misc] Support attention logits soft-capping with flash-attn (#7022)
Woosuk Kwon committed over 2 years ago
Update run-amd-test.sh (#7044)
omkar kakarparthi committed over 2 years ago
[Models] Support Qwen model with PP (#6974)
Murali Andoorveedu committed over 2 years ago
[Bugfix] Lower gemma's unloaded_params exception to warning (#7002)
Michael Goin committed over 2 years ago
[CI/Build] Remove sparseml requirement from testing (#7037)
Michael Goin committed over 2 years ago
[Bugfix] Fix RMSNorm forward in InternViT attention qk_layernorm (#6992)
Isotr0py committed over 2 years ago
[CI/Build] Update PyTorch to 2.4.0 (#6951)
Sage Moore committed over 2 years ago
[Build/CI] Fixing Docker Hub quota issue. (#7043)
Alexei-V-Ivanov-AMD committed over 2 years ago
[core][scheduler] simplify and improve scheduler (#6867)
youkaichao committed over 2 years ago
[Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user (#6954)
zifeitong committed over 2 years ago