A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 08:45 UTC
main
[CI] Add ci-fetch-log.sh helper for Buildkite job logs (#41517)
Michael Goin committed 6 months ago
Revert "[Build] Make bundled DeepGEMM wheel portable across Python versions" (#41512)
Michael Goin committed 6 months ago
[Build] Switch CUDA 13.0 wheel builds to PyTorch manylinux_2_28 base (#41416)
Michael Goin committed 6 months ago
[Bugfix][Gemma 4] Clamp soft-token estimate to max_soft_tokens (#40796)
Hoang Nguyen committed 6 months ago
Refactor Step3Text loading to use AutoWeightsLoader (#41492)
Matthew Santiago committed 6 months ago
[Doc] Add Codex usage example (#41358)
Chauncey committed 6 months ago
[ROCm][Bugfix] Fix init-time bias dtype cast when gate.out_dtype is None (#41405)
Rita Brugarolas committed 6 months ago
[vLLM IR] 2/N fused_add_rms_norm and maybe_inplace overload (#36823)
Luka Govedič committed 6 months ago
Limit concurrency on `test_transcription_api_correctness.py` (#41478)
Ekagra Ranjan committed 6 months ago
[MM][CG] Support ViT CG for Qwen2.5-VL (#40830)
John Calderon committed 6 months ago
[Fix] Sync gemma4 chat template from hf (#39570)
FredericOdermatt committed 6 months ago
Re-enable allreduce rms fusion for DP / PP (#41458)
Andy Lo committed 6 months ago
[Build] Make bundled DeepGEMM wheel portable across Python versions (#41476)
Michael Goin committed 6 months ago
[Bugfix] Fix persistent_topk inter-CTA init race on RadixRowState (#41444)
Yongye Zhu committed 6 months ago
[DSV4] Add knob to enable pre-attn gemm (#41443)
Yongye Zhu committed 6 months ago
[Perf] Intergrate Tile Kernels `head_compute_mix_kernel` for Deepseek-V4 (#41255)
Isotr0py committed 6 months ago
[Attention] Abstract the MLA prefill backends and eliminate cuDNN (#32623)
Matthew Bonanni committed 6 months ago
[Model Runner V2] Add `logprob_token_ids` support (#40559)
Wentao Ye committed 6 months ago
[Eval][CI] Add basic mrcr eval to tests/evals/ (#40164)
Michael Goin committed 6 months ago
[ROCm][FEAT] AITER Fused Allreduce + RMSNorm (#37646)
vllmellm committed 6 months ago