A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[CI/Build] Skip Mllama HF runner tests with Transformers v4.55.0 (#22659)
Isotr0py committed over 1 year ago
[Model] Pooling models default to using chunked prefill & prefix caching if supported. (#20930)
wang.yuqi committed over 1 year ago
fix: NIXL connector transfers partial block to pass full multi-modal context (#21074)
GuanLuo committed over 1 year ago
[Feature] Add `VLLM_USE_DEEP_GEMM_E8M0` Env to Control E8M0 Scale (#21968)
Wentao Ye committed over 1 year ago
[Misc] Further clean up some redundant config definitions (#22649)
Isotr0py committed over 1 year ago
Document aarch64 CPU support works (#22646)
Eric Curtin committed over 1 year ago
[Model] NemotronH Support (#22349)
danielafrimi committed over 1 year ago
[Misc] Move jsontree to utils (#22622)
Cyrus Leung committed over 1 year ago
[Misc] Move tensor schema tests (#22612)
Cyrus Leung committed over 1 year ago
[Docs] Add comprehensive CLI reference for all large `vllm` subcommands (#22601)
Harry Mellor committed over 1 year ago
[BUGFIX] KeyError 'layers.14.mlp.gate.g_idx' for Qwen3-MoE with GPTQ on ROCm (#22017)
JartX committed over 1 year ago
[Misc] benchmark_moe supports expert parallel (#22251)
Jee Jee Li committed over 1 year ago
Support token_type_ids in V1 with less code changes (#21985)
Maximilien de Bayser committed over 1 year ago
[ROCm][AITER] Support AITER Rope ops in RotaryEmbedding Module. (#22521)
vllmellm committed over 1 year ago
[BugFix] Fix logits repetition penalty cuda check (#22592)
Eugene Cheah committed over 1 year ago
[Misc][gpt-oss] guard import when triton kernel when not up to date (#22584)
Zhewen Li committed over 1 year ago
[Misc][gpt-oss] Add rules to label gpt-oss related PRs (#22600)
Lifans committed over 1 year ago
[BugFix] Fix KVConnectorOutput TPU breakage (#22598)
Nick Hill committed over 1 year ago
[CI/Build] Fix tensorizer test for load_format change (#22583)
22quinn committed over 1 year ago
Migrate LlavaImageInputs to TensorSchema (#21770)
Benji Beck committed over 1 year ago