A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 17:00 UTC
main
Add NVIDIA Meetup slides, announce AMD meetup, and add contact info (#8319)
Simon Mo committed about 2 years ago
[Misc] Fused MoE Marlin support for GPTQ (#8217)
Dipika Sikka committed about 2 years ago
[Misc] GPTQ Activation Ordering (#8135)
Kyle Sayers committed about 2 years ago
[Bugfix] Correct adapter usage for cohere and jamba (#8292)
Vladislav Kruglikov committed about 2 years ago
[Frontend] Add progress reporting to run_batch.py (#8060)
Adam Lugowski committed about 2 years ago
Kyle Mistele committed about 2 years ago
[Bugfix] Fix async postprocessor in case of preemption (#8267)
Alexander Matveev committed about 2 years ago
[CI/Build] Use python 3.12 in cuda image (#8133)
Joe Runde committed about 2 years ago
ppc64le: Dockerfile fixed, and a script for buildkite (#8026)
sumitd2 committed about 2 years ago
[Model][VLM] Decouple weight loading logic for `Paligemma` (#8269)
Isotr0py committed about 2 years ago
[Model][VLM] Support multi-images inputs for InternVL2 models (#8201)
Isotr0py committed about 2 years ago
[Bugfix] Fix broken OpenAI tensorizer test (#8258)
Cyrus Leung committed about 2 years ago
youkaichao committed about 2 years ago
Enable Random Prefix Caching in Serving Profiling Tool (benchmark_serving.py) (#8241)
Wei-Sheng Chin committed about 2 years ago
[Model] Multi-input support for LLaVA (#8238)
Cyrus Leung committed about 2 years ago
[Bugfix] Fix Hermes tool call chat template bug (#8256)
Kyle Mistele committed about 2 years ago
[misc] [doc] [frontend] LLM torch profiler support (#7943)
William Lin committed about 2 years ago
[Model] Allow loading from original Mistral format (#8168)
Patrick von Platen committed about 2 years ago
[Misc] Remove `SqueezeLLM` (#8220)
Dipika Sikka committed about 2 years ago
[Kernel] [Triton] Memory optimization for awq_gemm and awq_dequantize, 2x throughput (#8248)
rasmith committed about 2 years ago