A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Docs] [V1] [Hybrid] Update docs to remove FlashInfer constraint for hybrid models (#23665)
Thomas Parnell committed about 1 year ago
[Docs] Fix warnings in `mkdocs build` (#23649)
Hyogeun Oh (오효근) committed about 1 year ago
[Bugfix] Fix cuda event usage with CPU model runner (#23643)
Li, Jiang committed about 1 year ago
[CI] [Doc]: Add GH Action for auto labeling issues with `rocm` tag (#20988)
vllmellm committed about 1 year ago
[Misc] Add override for allreduce fusion thresholds (#23639)
nvjullin committed about 1 year ago
[model] support qwen2audio embedding input (#23625)
Yuekai Zhang committed about 1 year ago
[Bugfix] fix bf16 multimodal model hash (#23623)
Yuekai Zhang committed about 1 year ago
[Docs] Reduce requirements for docs build (#23651)
Harry Mellor committed about 1 year ago
[ROCm] Starting to add AMD code reviewers for ROCm components (#23496)
Hongxia Yang committed about 1 year ago
[CPU] add cpu fused moe pytorch native implementation (#23146)
TianyuLi0 committed about 1 year ago
[Doc]: fix various spelling issues in multiple files (#23636)
Didier Durand committed about 1 year ago
[Kernel] Added flashinfer fp8 per-tensor gemms (#22895)
nvjullin committed about 1 year ago
[Hardware][Mac] Fix the installation fail for Apple Silicon (CPU) (#23565)
En Ouyang committed about 1 year ago
[Docs] Fix broken links to `docs/api/summary.md` (#23637)
Harry Mellor committed about 1 year ago
Support FlashAttention Backend for Hybrid SSM Models (#23299)
Chen Zhang committed about 1 year ago
feat: add usage to TranscriptionResponse (text and json response_format) (#23576)
Guillaume Calmettes committed about 1 year ago
[Bugfix] Add missing enable_log_outputs parameter to init_app_state function (#23634)
Matúš Námešný committed about 1 year ago
[Model] fix DeepSeek e_score_correction_bias dtype to fp32 (#23640)
Jee Jee Li committed about 1 year ago
[Perf] Remove duplicated NVFP4 blockscales to save memory (#23379)
Michael Goin committed about 1 year ago
Fix writing benchmark results with tuple keys (#23633)
Huy Do committed about 1 year ago