A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[CI] fix docs build failed (#27082)
Chauncey committed about 1 year ago
[Kernel][Performance] Fuse float cast and renormalize to topk softmax kernel (#26717)
zhrrr committed about 1 year ago
[CI/Build] Update compressed tensor test path to fix CPU CI (#27068)
Li, Jiang committed about 1 year ago
[Model] Add support for LightOnOCR (#26916)
Said Taghadouini committed about 1 year ago
[Model] Always use Transformers backend for PaliGemma and Gemma3-MM (#26715)
Cyrus Leung committed about 1 year ago
[CI/Build] Update Llama4 eval yaml (#27070)
Zhewen Li committed about 1 year ago
[Kernel] Lazy import FlashInfer (#26977)
Jee Jee Li committed about 1 year ago
[Bugfix] Fix ReplicatedLinearWithLoRA (#27065)
Jee Jee Li committed about 1 year ago
[Core] Change `execute_model_with_error_logging()` to be a ctx manager (#27060)
Nick Hill committed about 1 year ago
[Bugfix][Qwen] fixes the weights dtype in qwen3_next: it is actually a bfloat16 (#27030)
Tao He committed about 1 year ago
disable graph partition in custom op (#26952)
Boyuan Feng committed about 1 year ago
[Perf] Exploit out-of-band buffers in shm_broadcast (#26961)
Nick Hill committed about 1 year ago
[Docs] Reduce custom syntax used in docs (#27009)
Harry Mellor committed about 1 year ago
vllm bench serve shows num of failed requests (#26478)
Tomas Ruiz committed about 1 year ago
Remove unused imports (#26972)
Lukas Geiger committed about 1 year ago
[torch.compile] fix simple inductor graph partition test (#27050)
Boyuan Feng committed about 1 year ago
[Chore] Separate out `vllm.utils.importlib` (#27022)
Cyrus Leung committed about 1 year ago
[torch.compile] Passing only necessary compilation config to inductor pass config (#27041)
Lucia Fang committed about 1 year ago
[NVIDIA] [Perf] Update to leverage flashinfer trtllm FP4 MOE throughput kernel (#26714)
jiahanc committed about 1 year ago
Refactor Transformers backend to use mixins (#26906)
Harry Mellor committed about 1 year ago