A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 7, 05:15 UTC
main
[Hardware] Replace `torch.cuda.synchronize()` api with `torch.accelerator.synchronize` (#36085)
Kunshang Ji committed 8 months ago
[Release] Include source distribution (sdist) in PyPI uploads (#35136)
Doug Smith committed 8 months ago
[Docs] add Dynamo/aibrix integration and kubeai/aks link (#32767)
Paco Xu committed 8 months ago
[Docs] Update docs to include mm processor + encoder benchmarks (#34083)
Reagan Lee committed 8 months ago
qwen3coder tool parser fix anyOf double encoded parameters (#36032)
Christian Munley committed 8 months ago
[Chore] Correct MTP models test registry ordering (#36115)
Isotr0py committed 8 months ago
[Docs] Update `CacheConfig` block_size docstring to remove inaccurate limit when using CUDA (#35632)
Seiji Eicher committed 8 months ago
[Misc] Remove deprecated items that are due for removal (#36006)
Martin Hickey committed 8 months ago
[Bugfix] Fix block_size for hybrid model MTP (#36036)
Benjamin Chislett committed 8 months ago
[Doc] Add Parallel Draft Models (#35973)
zihaoanllm committed 8 months ago
[Kernel] [Helion] [11/N] Retune configs for silu_mul_fp8 (#36062)
Yanan Cao committed 8 months ago
Hanjun Cho committed 8 months ago
[Perf] Use dummy M for weight prepacking on x86 (#35890)
Tianmu Li committed 8 months ago
Order `config.py` in Lexicographical order (#35866)
Andrii Skliar committed 8 months ago
[compile] Fix extra cache save on warm start. (#35921)
Zhengxu Chen committed 8 months ago
[CI] Don't leave docs preview comment on closed PRs (#36087)
Harry Mellor committed 8 months ago
[Core] Move ray-specific WorkerWrapperBase methods to RayWorkerWrapper (#35328)
Nick Hill committed 8 months ago
[Model] Add LoRA support for Whisper models (#29856)
daje0601 committed 8 months ago
[CI] Fix pre-commit mypy issue in main (#36049)
Wentao Ye committed 8 months ago
Add platform method to enable custom collective ops registration (#34760)
nkm-meta committed 8 months ago