A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 21:00 UTC
main
[ROCm] Fix GPT-OSS import for triton 3.6 (#37453)
Gregory Shtrasberg committed 7 months ago
{ROCm]: gpt-oss fusion/padding fixes (#38043)
Rohan Potdar committed 7 months ago
Remove need for explicit `\n` in docstring lists for `--help` formatting (#38350)
Harry Mellor committed 7 months ago
enable skipping of SW attention layers when using FP8 KV cache (#33695)
Jonas M. Kübler committed 7 months ago
fix(security): Add VLLM_MAX_N_SEQUENCES environment variable and enforce limit (#37952)
Juan Pérez de Algaba committed 7 months ago
[EPLB] Cleanup the transfer logic for the various eplb maps (#34520)
Sage Moore committed 7 months ago
[P/D] Mooncake: Add unit tests and minor fixes for mooncake connector (#36946)
dtc committed 7 months ago
[CI/Build] Move nightly wheel index generation to a single post-build step (#38322)
Shengqi Chen committed 7 months ago
[Doc] Clarify Helm chart location in deployment guide (#38328)
Yuichiro Utsumi committed 7 months ago
[Bugfix] Fix Hermes tool parser when stream interval > 1 (#38168)
Flora Feng committed 7 months ago
[Refactor] Move FusedMoE hidden_size roundup to quant_method (#34285)
Bowen Bao committed 7 months ago
[CI] Add xpu auto-label rule for Intel GPU/XPU PRs (#38320)
wenjun liu committed 7 months ago
[CPU] Support CT W4A16 on CPU MP kernel (#38219)
Li, Jiang committed 7 months ago
[Model] Extract GatedDeltaNetAttention into shared layer for Qwen3Next and Qwen3.5 (#37975)
Xiaoshuang Wang committed 7 months ago
[frontend] dump openai responses type by alias (#38262)
cjackal committed 7 months ago
[kv_offload+HMA][7/N]: Support register_kv_caches for hybrid models (#37853)
Or Ozeri committed 7 months ago
[Bugfix] Offload blocking tokenizer ops to shared thread pool to unblock event loop (#34789)
Bvicii committed 7 months ago
[CI/Build] enable Intel XPU test flow with prebuilt image (#37447)
wenjun liu committed 7 months ago
Various Transformers v5 config fixes (#38247)
Harry Mellor committed 7 months ago
[Bugfix] Add missing f-string prefix in xgrammar choices error message (#38162)
yzong-rh committed 7 months ago