A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 07:15 UTC
main
[Bugfix] fix v1 cpu worker fails on macOS (#19121)
Kebe committed over 1 year ago
[TPU] Update dynamo dump file name in compilation test (#19108)
Siyuan Liu committed over 1 year ago
[Perf] Tunings for SM100 FP8 CUTLASS kernel (#18778)
Michael Goin committed over 1 year ago
[Bugfix][EP+DP] Fix internode check (#19112)
Tyler Michael Smith committed over 1 year ago
Allow AsyncLLMEngine.generate to target a specific DP rank (#19102)
jmswen committed over 1 year ago
[Doc] Update V1 Guide for embedding models (#19141)
Cyrus Leung committed over 1 year ago
Sm100 blockwise fp8 swap ab (#18564)
Lain committed over 1 year ago
Add DeepSeek-R1-0528 function call chat template (#18874)
Xu Wenqing committed over 1 year ago
[CI/Build][Bugfix] Ensure compatibility with transformers 4.52 (#18678)
Cyrus Leung committed over 1 year ago
Improve the output precision of embedding models (#19092)
wang.yuqi committed over 1 year ago
[Misc] Add packages for benchmark as extra dependency (#19089)
Isotr0py committed over 1 year ago
Fix ValueError: Missing value for tag key(s): model_name,engine. (#19113)
Seiji Eicher committed over 1 year ago
[TPU] Skip hanging tests (#19115)
Siyuan Liu committed over 1 year ago
汪志鹏 committed over 1 year ago
[Bugfix] Fix FA3 full cuda graph correctness (#19106)
Woosuk Kwon committed over 1 year ago
[NVIDIA] Add Cutlass MLA backend (#17625)
Kaixi Hou committed over 1 year ago
[Cleanup][v1]:remote guided-decoding-backend for example (#19059)
Calvin Chen committed over 1 year ago
[KERNEL] Sampler. CUDA kernel for applying repetition penalty (#18437)
Vadim Gimpelson committed over 1 year ago
[Core] Cast multimodal input in hf processor (#18862)
Lukas Geiger committed over 1 year ago
[CPU] V1 support for the CPU backend (#16441)
Li, Jiang committed over 1 year ago