A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 11, 00:00 UTC
main
[CI] Add Qwen3-Next-FP8 to Blackwell model tests (#31049)
Vadim Gimpelson committed 11 months ago
[ROCm][Bugfix] Fix RuntimeError in MMEncoderAttention by replacing .view() with .reshape() (#31203)
Andreas Karatzas committed 11 months ago
[Core] Add a random suffix to frontend-provided request IDs (#27987)
Mark McLoughlin committed 11 months ago
[Mamba] - Consolidate Mambas Attention Logic (#28133)
Asaf Joseph Gardin committed 11 months ago
docs: Add llm-d integration to the website (#31234)
Yuan Tang committed 11 months ago
Use helper function instead of looping through attribute names (#29788)
Harry Mellor committed 11 months ago
Only patch `original_max_position_embeddings` for Transformers v4 (#31214)
Harry Mellor committed 11 months ago
[FIX] FP4 quantization kernel padding initialization bug (#31097)
danielafrimi committed 11 months ago
Fix edge case Mistral tool parser (#30724)
Joachim Studnia committed 11 months ago
[Misc] Introduce `encode_*_url` utility function (#31208)
Cyrus Leung committed 11 months ago
Patrick von Platen committed 11 months ago
Add util function for checking nesting of rope parameters (#31146)
Harry Mellor committed 11 months ago
[OpenAI] Add parameter metadata to validation errors (#30134)
R3hankhan committed 11 months ago
[Frontend] Support using chat template as custom score template for reranking models (#30550)
Jakub Zakrzewski committed 11 months ago
[Bugfix] Fix MoE LoRA bin/pt loading (#31161)
Jee Jee Li committed 11 months ago
Correct position of docstring of class attributes (#31209)
Weida Hong committed 11 months ago
[ROCm][FEAT] Support AITER RMSNorm quantization fusion pass (#26575)
vllmellm committed 11 months ago
[Bugfix] Fix Jais2ForCausalLM (#31198)
Jee Jee Li committed 11 months ago
[XPU] decrease IGC_ForceOCLSIMDWidth for speculative decoding triton-xpu kernel compilation (#30538)
Yan Ma committed 11 months ago
[Chore] Update more locations to use `attention_config.backend` (#31153)
Cyrus Leung committed 11 months ago