A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[CI] Add timeouts to tests (#24260)
Rafael Vasquez committed about 1 year ago
[RFC] allow cancelation after shutdown in blocking collective_rpc (#23390)
Shiyan Deng committed about 1 year ago
[Bugfix] Fix silu_mul+quant fusion test (#24341)
elvischenv committed about 1 year ago
[gpt-oss][Bugfix]Fix streamableparser for missing handling of certain token_ids (#24306)
Chauncey committed about 1 year ago
[docs] add shenzhen meetup (#24326)
youkaichao committed about 1 year ago
Adding int4 and int8 models for CPU benchmarking (#23709)
Louie Tsai committed about 1 year ago
[XPU][P/D] Add XPU support in NixlConnector (#22436)
liuzhenwei committed about 1 year ago
[gpt-oss] tool parser supports for /chat/completions [1/n] (#22386)
Aaron Pham committed about 1 year ago
[Frontend] Skip unnecessary detokenization when token_id is requested (#24236)
Nicolò Lucchesi committed about 1 year ago
[CI/Build] Reduce the number of redundant cases to test for LoRA (#24276)
Zhuohan Li committed about 1 year ago
elvischenv committed about 1 year ago
[Misc] Have AsyncLLM `custom_stat_loggers` extend default logger list (#20952)
Seiji Eicher committed about 1 year ago
QWEN3 Coder Fused MoE kernels Optimization configs (#24266)
Saman A. Pour committed about 1 year ago
Upgrade FlashInfer to v0.3.0 (#24086)
Po-Han Huang (NVIDIA) committed about 1 year ago
[Misc] Slight improve deepgemm print (#24085)
Jee Jee Li committed about 1 year ago
[Doc]: fix typos in Python comments (#24173)
Didier Durand committed about 1 year ago
[Perf] Freeze core engine proc heap after init (#24008)
Nick Hill committed about 1 year ago
[Misc] Removed force_fp8_e4m3fnuz from FP8LinearOp (#23725)
nvjullin committed about 1 year ago
[LoRA]: Add lora support to qwen-2.5-omni (#24231)
Yash Pratap Singh committed about 1 year ago
[XPU] support Triton Attention backend on Intel GPU (#24149)
Kunshang Ji committed about 1 year ago