A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Docs] Fix warnings in mkdocs build (continued) (#24740)
Hyogeun Oh (오효근) committed about 1 year ago
[Doc]: fix typos in various files (#24726)
Didier Durand committed about 1 year ago
[Multimodal] Remove legacy multimodal fields in favor of MultiModalFeatureSpec (#24548)
Flora Feng committed about 1 year ago
[Multi Modal] Add FA3 in VIT (#24347)
Wenlong Wang committed about 1 year ago
[sleep mode] save memory for on-the-fly quantization (#24731)
youkaichao committed about 1 year ago
[Kernel] [CPU] refactor `cpu_attn.py:_run_sdpa_forward` for better memory access (#24701)
Ignacio Sica committed about 1 year ago
[Bugfix] Fix BNB name match (#24735)
Jee Jee Li committed about 1 year ago
Chen Zhang committed about 1 year ago
[CI/Build] Skip prompt embeddings tests on V1-only CPU backend (#24721)
Li, Jiang committed about 1 year ago
[CI] Add ci_envs for convenient local testing (#24630)
wang.yuqi committed about 1 year ago
[CI] Fix flaky test v1/worker/test_gpu_model_runner.py::test_kv_cache_stride_order (#24640)
Chen Zhang committed about 1 year ago
[BugFix] Fix Qwen3-Next PP (#24709)
Nick Hill committed about 1 year ago
[Bugfix] Fix MRoPE dispatch on CPU (#24712)
Li, Jiang committed about 1 year ago
Andrew Sansom committed about 1 year ago
Enable conversion of multimodal models to pooling tasks (#24451)
Maximilien de Bayser committed about 1 year ago
[DOCs] Update ROCm installation docs section (#24691)
Gregory Shtrasberg committed about 1 year ago
[Qwen3-Next] MoE configs for H20 TP=1,2,4,8 (#24707)
Jee Jee Li committed about 1 year ago
[Bugfix] fixes the causal_conv1d_update kernel update non-speculative decoding cases (#24680)
Tao He committed about 1 year ago
[V1] feat:add engine v1 tracing (#20372)
RichardoMu committed about 1 year ago
[Startup] Make DeepGEMM warmup scale with max-num-batched-tokens (#24693)
Lucas Wilkinson committed about 1 year ago