A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
break execute_model in gpu_model_runner into sub-functions for custom scopes (#24265)
Bangsheng Tang committed about 1 year ago
[Bugfix] Fix unstable silu_mul+nvfp4 quant fusion test (#24370)
elvischenv committed about 1 year ago
Ye (Charlotte) Qi committed about 1 year ago
[Fix] [gpt-oss] fix non-tool calling path for chat completion (#24324)
Aaron Pham committed about 1 year ago
Lora bias(enable_lora_bias) deprecate warning (#24339)
Ashwin Phadke committed about 1 year ago
[Bugfix] Fix test_mixtral_moe (#24371)
Jee Jee Li committed about 1 year ago
[Bugfix] Fix broken deepseek fp8 TP weights loading (#24367)
Isotr0py committed about 1 year ago
[Bugfix] Avoid uninitialized usage of azp_val when AZP is false. (#24335)
mohankku committed about 1 year ago
Add @benchislett to codeowner for spec decode and structured outputs (#24362)
Benjamin Chislett committed about 1 year ago
[CI] Disable flaky structured output test from CI (#24366)
Roger Wang committed about 1 year ago
[Multimodal] Improve max video embedding length estimation in V1 (#24312)
Roger Wang committed about 1 year ago
refactor: Turn GPUModelRunner.inputs_embeds to a CpuGpuBuffer (#24345)
Andrew Sansom committed about 1 year ago
[New Model]: google/embeddinggemma-300m (#24318)
wang.yuqi committed about 1 year ago
[Core] Allow disabling TP sharding for parallel Linear layer (#23024)
Isotr0py committed about 1 year ago
[Bugfix] Catch and log invalid token ids in detokenizer (#24351)
Nick Hill committed about 1 year ago
[Feature] Support Decode Context Parallel (DCP) for MLA (#23734)
yzds committed about 1 year ago
[KV Sharing] Raise error if using eagle with fast prefill (#24350)
Yong Hoon Shin committed about 1 year ago
[Doc]: fix typos in Python comments (#24294)
Didier Durand committed about 1 year ago
Add @22quinn as code reviewer for RL related components (#24346)
22quinn committed about 1 year ago
[Perf][V1] Fully overlap model execution (#23569)
Benjamin Chislett committed about 1 year ago