A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 03:15 UTC
main
[Bugfix][Refactor] Unify model management in frontend (#11660)
Joe Runde committed almost 2 years ago
[Benchmark] Add benchmark script for CPU offloading (#11533)
Yihua Cheng committed almost 2 years ago
[V1][VLM] V1 support for selected single-image models. (#11632)
Roger Wang committed almost 2 years ago
[V1] Simpify vision block hash for prefix caching by removing offset from hash (#11646)
Chen Zhang committed almost 2 years ago
sakunkun committed almost 2 years ago
[Misc][LoRA] Support Rank Stabilized LoRA (RSLoRA) (#6909)
John Giorgi committed almost 2 years ago
[Bugfix] Fix OpenAI parallel sampling when using xgrammar (#11637)
Michael Goin committed almost 2 years ago
[Model][LoRA]LoRA support added for MolmoForCausalLM (#11439)
Matthias Vogler committed almost 2 years ago
[benchmark] Remove dependency for H100 benchmark step (#11572)
Kevin H. Luu committed almost 2 years ago
[Bugfix] Validate and concatenate image embeddings in MiniCPMVBaseModel (#11631)
whyiug committed almost 2 years ago
[V1] [6/N] API Server: Better Shutdown (#11586)
Robert Shaw committed almost 2 years ago
[VLM] Abstract out multi-modal data parsing in merged processor (#11620)
Cyrus Leung committed almost 2 years ago
[platforms] enable platform plugins (#11602)
youkaichao committed almost 2 years ago
[CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels (#11618)
Li, Jiang committed almost 2 years ago
[Build][Kernel] Update CUTLASS to v3.6.0 (#11607)
Tyler Michael Smith committed almost 2 years ago
[Docker] bump up neuron sdk v2.21 (#11593)
Liangfu Chen committed almost 2 years ago
[v1] fix compilation cache (#11598)
youkaichao committed almost 2 years ago
Remove print statement in DeepseekScalingRotaryEmbedding (#11604)
Michael Goin committed almost 2 years ago
[Misc] KV cache transfer connector registry (#11481)
Kuntai Du committed almost 2 years ago
[v1][bugfix] fix cudagraph with inplace buffer assignment (#11596)
youkaichao committed almost 2 years ago