A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Optimization] Avoid repeated model architecture conversion for pooling models (#25261)
Cyrus Leung committed about 1 year ago
[Bugfix] fix tool call arguments is empty (#25223)
Chauncey committed about 1 year ago
[BUG FIX][NON-CUDA]quick fix to avoid call cudagraph_unsafe in attention (#25298)
Chendi.Xue committed about 1 year ago
[BugFix] Exclude self when checking for port collision (#25286)
Nick Hill committed about 1 year ago
JartX committed about 1 year ago
Improve weight loading for encoder models in Transformers backend (#25289)
Harry Mellor committed about 1 year ago
[Misc] Support more collective_rpc return types (#25294)
Nick Hill committed about 1 year ago
[BugFix] Ensure appropriate guards in destructors (#25284)
Nick Hill committed about 1 year ago
[torch.compile] CUDAGraph Inductor partition integration (#24281)
Boyuan Feng committed about 1 year ago
[docs] Prompt Embedding feature support (#25288)
Andrew Sansom committed about 1 year ago
test: Remove vestigial skip for prompt embeds tests after landing v1 Prompt Embeds support (#25291)
Andrew Sansom committed about 1 year ago
Don't skip special tokens with hermes-style tool calling (#25281)
Maximilien de Bayser committed about 1 year ago
[Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090)
Lucas Kabela committed about 1 year ago
[BugFix] Fix async scheduling CPU tensor race take 2 (#25279)
Nick Hill committed about 1 year ago
allow disable flashinfer prefill (#25276)
Lucia Fang committed about 1 year ago
Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771)
Zhiyu committed about 1 year ago
[Compile] Fix Compile Warning for Ignoring `MIN_BLOCK_PER_SM` (#25193)
Wentao Ye committed about 1 year ago
[Frontend] Responses API messages out, just harmony for now (#24985)
Alec S committed about 1 year ago
Fix: Correct FusedMoE layer reference in auto_round quantization (#24818)
David-Wen committed about 1 year ago
[BugFix] Make FlashInferMetadataBuilder non-blocking (#25040)
nvjullin committed about 1 year ago