A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 18:30 UTC
main
[BugFix] Fix torch distributed stateless PG backend init (#14870)
Nick Hill committed over 1 year ago
[Kernel] Add more tuned configs (#14877)
Simon Mo committed over 1 year ago
[V1] Remove V0 fallback for mistral-tokenizer (#14873)
Roger Wang committed over 1 year ago
[VLM] Clean up Phi-4-MM ViT implementation (#14812)
Isotr0py committed over 1 year ago
[Misc][Doc] Minor benchmark README update (#14874)
Roger Wang committed over 1 year ago
[VLM] Merged multi-modal processor for Pixtral (#12211)
Rémi Delacourt committed over 1 year ago
[Core] Expose API endpoint `/is_sleeping` (#14312)
Jun Duan committed over 1 year ago
[CI][Intel GPU] refine intel GPU ci docker build (#14860)
Kunshang Ji committed over 1 year ago
[VLM] Limit multimodal input cache by memory (#14805)
Cyrus Leung committed over 1 year ago
[Bugfix] EAGLE output norm bug (#14464)
Bryan Lu committed over 1 year ago
[Build/CI] Upgrade jinja2 to get 3 moderate CVE fixes (#14839)
Russell Bryant committed over 1 year ago
Jennifer Zhao committed over 1 year ago
[V1][Structured Output] calculate vocab_size eagerly (#14851)
Aaron Pham committed over 1 year ago
[CI/Build] Delete LoRA bias test (#14849)
Jee Jee Li committed over 1 year ago
[Attention] Get rid of mla cache alignment (#14842)
Lucas Wilkinson committed over 1 year ago
[CPU] Support FP8 KV cache (#14741)
Li, Jiang committed over 1 year ago
[Docs] Add new East Coast vLLM Meetup slides to README and meetups.md (#14852)
Simon Mo committed over 1 year ago
[V1] V1 Enablement Oracle (#13726)
Robert Shaw committed over 1 year ago
[Misc][Easy] Annotate unused vars in the csrc files (#14798)
Lu Fang committed over 1 year ago
[Misc] Remove misleading message in gemma2 and gemma3 (#14850)
Isotr0py committed over 1 year ago