A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 14:45 UTC
main
[CI][ROCm] Add gpt-oss w4a8 in CI (#38292)
Bowen Bao committed 7 months ago
[Frontend] Re-enable running MaxSim on GPU (#38620)
wang.yuqi committed 7 months ago
Triton MLA perf fixes (#33529)
Koushik Dutta committed 7 months ago
[Bugfix] Enable batch-invariant Triton matmul on all Ampere GPUs (SM 8x) (#38427)
Yusuf Mohammad committed 7 months ago
[Model] Add support for Cheers multimodal model (#38788)
bsliu committed 7 months ago
Don't compile vision encoder for Transformers backend (#30518)
Harry Mellor committed 7 months ago
[Feature] KV cache per-token-head INT8/FP8 quantization (#38378)
JartX committed 7 months ago
[Perf] fuse kernels in gdn (#37813)
Jiangyun Zhu committed 7 months ago
vllmellm committed 7 months ago
[CPU] Support gelu act in cpu_fused_moe (#38770)
Li, Jiang committed 7 months ago
Revert "[Kernel] Add gpt-oss Router GEMM kernel (#37205)" (#38778)
Xin Yang committed 7 months ago
[Kernel] [Helion] Use warning_once in get_gpu_name to prevent log spam (#38743)
Yanan Cao committed 7 months ago
[ROCm][Bugfix] Fix ROCm runtime failure due to missing symbol (#38750)
Gregory Shtrasberg committed 7 months ago
[Bugfix] Use dedicated MM processor cache in /tokenize to prevent sender-cache pollution (#38545)
Sergey Zinchenko committed 7 months ago
[Perf] DSV3.2 Indexer Fused Weights Projection (#38684)
Benjamin Chislett committed 7 months ago
Fix multiline-format string for python 3.10 (#38739)
Luka Govedič committed 7 months ago
[BugFix] Fix precommit breakage due to conflicting in-flight merges (#38759)
Nick Hill committed 7 months ago
Kevin H. Luu committed 7 months ago
[Bugfix] Preserve original ImportError in gRPC server entrypoint (#38673)
Chang Su committed 7 months ago
[Feat][Executor] Introduce RayExecutorV2 (#36836)
Jeffrey Wang committed 7 months ago