A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 8, 17:45 UTC
main
[torch.compile] Reorganize vllm/compilation and tests/compile (0/N for vLLM IR) (#33731)
Luka Govedič committed 9 months ago
[CPU][BugFix] Fix loading of w8a8int models with bias (#33582)
Fadi Arafeh committed 9 months ago
Bump HF Hub client to get bug fix (#33984)
Harry Mellor committed 9 months ago
[PaddleOCR-VL] Add BC for transformers 5.0 config (#33976)
zhang-prog committed 9 months ago
Consolidate and fix forbidden import `pre-commit` checks (#33982)
Harry Mellor committed 9 months ago
support view_from_cpu_tensor on XPU (#33868)
Xinyu Chen committed 9 months ago
Fix `main` pre-commit (#33975)
Harry Mellor committed 9 months ago
[cpu][performance] CPU Paged Attention NEON BFMMLA BF16 Implementation (#32263)
Gassan Salama committed 9 months ago
Onboard voyage-4-nano (#33720)
chengchengpei committed 9 months ago
[XPU]Replace pip in docker.xpu with uv pip (#31112)
sihao_li committed 9 months ago
[XPU][4/N] add mxfp4 moe model support (#33679)
Kunshang Ji committed 9 months ago
[CPU] Add BF16 Kernel type for s390x (#33788)
R3hankhan committed 9 months ago
[Misc] Update code for encoder-decoder models (#33900)
Cyrus Leung committed 9 months ago
feat(frontend): early-fail tokenization guard for user requests (#31366)
Mingliang Li committed 9 months ago
fix(ROCm): Make flash_attn import optional in MLA attention (#33511)
Rabi Mishra committed 9 months ago
[Docs] Add reo analytics (#33957)
Simon Mo committed 9 months ago
[Perf] Disable clean_logits in deepgemm fp8_mqa_logits kernel (#33568)
Xin Yang committed 9 months ago
[Feature] OTEL tracing during loading (#31162)
emricksini-h committed 9 months ago
[Bugfix] Fix DeepSeek v3.2 tokenizer outputting None issue (#33832)
Wei Zhao committed 9 months ago
Adds padding and perf improvements to wvSplitK_fp8 (#33527)
Hashem Hashemi committed 9 months ago