A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 23:45 UTC
main
Add interleaved RoPE test for Llama4 (Maverick) (#21478)
Yong Hoon Shin committed over 1 year ago
[Perf] Cuda Kernel for Int8 Per Token Group Quant (#21476)
Wentao Ye committed over 1 year ago
Add Unsloth to RLHF.md (#21636)
Daniel Han committed over 1 year ago
[TPU][Test] Rollback PR-21550. (#21619)
QiliangCui committed over 1 year ago
[Docs] add auto-round quantization readme (#21600)
Wenhua Cheng committed over 1 year ago
[CI] Unifying Dockerfiles for ARM and X86 Builds (#21343)
Kebe committed over 1 year ago
Add support for Prithvi in Online serving mode (#21518)
mgazz committed over 1 year ago
[Kernel] Improve machete memory bound perf (#21556)
czhu-cohere committed over 1 year ago
[ROCm][AITER] Enable fp8 kv cache on rocm aiter backend. (#20295)
who who who committed over 1 year ago
[Model] Replace Mamba2 RMSNorm Gated with Fused Triton Kernel (#20839)
Chih-Chieh Yang committed over 1 year ago
kourosh hakhamaneshi committed over 1 year ago
[MODEL] New model support for naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B (#20931)
bigshanedogg committed over 1 year ago
[Model] Fix Ernie4.5MoE e_score_correction_bias parameter (#21586)
xyxinyang committed over 1 year ago
[Bugfix][Logprobs] Fix logprobs op to support more backend (#21591)
Mengqing Cao committed over 1 year ago
[V1] Get supported tasks from model runner instead of model config (#21585)
Cyrus Leung committed over 1 year ago
[Quantization] Enable BNB support for more MoE models (#21370)
Jee Jee Li committed over 1 year ago
[Bugfix] GGUF: fix AttributeError: 'PosixPath' object has no attribute 'startswith' (#21579)
Kebe committed over 1 year ago
Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#21598)
Xu Wenqing committed over 1 year ago
[Tests] Harden DP tests (#21508)
Nick Hill committed over 1 year ago
[TPU][Bugfix] fix OOM issue in CI test (#21550)
Chengji Yao committed over 1 year ago