A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 20:15 UTC
main
[MoE Refactor][12/N] Marlin Fp8 MoE Pure Function (#31499)
Robert Shaw committed 11 months ago
[Core] Enable async scheduling by default (#27614)
Nick Hill committed 11 months ago
implements register kv caches in lmcache connector (#31397)
chunxiaozheng committed 11 months ago
Optimize QKNorm for MiniMax-M2/M2.1 (#31493)
Roger Young committed 11 months ago
[Feature] Add offline FastAPI documentation support for air-gapped environments (#30184)
RickyChen / 陳昭儒 committed 11 months ago
Replace `nn.ConvNd` with vLLM's `ConvNdLayer` for Transformers modeling backend (#31498)
Harry Mellor committed 11 months ago
Migrate doc to website: Hardware Plugins (1/N) (#31496)
Roy Wang committed 11 months ago
[Docs] Use relative `md` links instead of absolute `html` links for cross referencing (#31494)
Harry Mellor committed 11 months ago
[Bugfix] Preserve tool call id/type/name in streaming finish chunk (#31438)
amittell committed 11 months ago
[CI] fix test_chat_truncation_content_not_null test (#31488)
Chauncey committed 11 months ago
[Bugfix][Frontend] Fix Jina reranker multimodal input compatibility (#31445)
twj committed 11 months ago
[ROCm][GPTQ][Bugfix] Fix GPTQ GEMM kernel output zeroing race condition (#30719)
Andreas Karatzas committed 11 months ago
[ROCm][CI] Skip DeepGemm-dependent test on ROCm platform (#31462)
Andreas Karatzas committed 11 months ago
[CI/Build][CPU] Update CPU CI test cases (#31466)
Li, Jiang committed 11 months ago
Add Fused MoE Triton kernels for GLM-4.5-Air, GLM-4.5v, GLM-4.6v on 2x RTX Pro 6000 (#31407)
Mamy Ratsimbazafy committed 11 months ago
[Model] Add tuned triton fused_moe configs for Qwen3Moe on B200 (#31448)
Jzz1943 committed 11 months ago
[BugFix] Re-fix async multimodal cpu tensor race condition (#31373)
Nick Hill committed 11 months ago
[ROCm] Migrate xgrammar to upstream release (#31327)
Andreas Karatzas committed 11 months ago
[ROCm][CI] Add TorchCodec source build for transcription tests (#31323)
Andreas Karatzas committed 11 months ago
[ROCm][CI] Added perceptron lib in requirements for isaac multi-modal test (#31441)
Andreas Karatzas committed 11 months ago