A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 06:00 UTC
main
Ximingwang-09 committed over 1 year ago
Add cache to cuda get_device_capability (#19436)
Michael Goin committed over 1 year ago
Support non-string values in JSON keys from CLI (#19471)
Cyrus Leung committed over 1 year ago
[Misc] Reduce warning message introduced in env_override (#19476)
Lu Fang committed over 1 year ago
[Doc] Support "important" and "announcement" admonitions (#19479)
Cyrus Leung committed over 1 year ago
[Doc] Fix quantization link titles (#19478)
Cyrus Leung committed over 1 year ago
[Doc] Update V1 User Guide for Hardware and Models (#19474)
Cyrus Leung committed over 1 year ago
[Kernel] Support deep_gemm for linear methods (#19085)
artetaout committed over 1 year ago
[ROCm] Add rules to automatically label ROCm related PRs (#19405)
Lu Fang committed over 1 year ago
Fix Typo in Documentation and Function Name (#19442)
leopardracer committed over 1 year ago
Add fused MOE config for Qwen3 30B A3B on B200 (#19455)
Junhao Li committed over 1 year ago
[Misc] Remove unused `MultiModalHasher.hash_prompt_mm_data` (#19422)
Lukas Geiger committed over 1 year ago
[CI] Disable failing GGUF model test (#19454)
Michael Goin committed over 1 year ago
Fix test_max_model_len in tests/entrypoints/llm/test_generate.py (#19451)
Lu Fang committed over 1 year ago
[BugFix] Fix docker build cpu-dev image error (#19394)
niu_he committed over 1 year ago
[New Model]: Support Qwen3 Embedding & Reranker (#19260)
wang.yuqi committed over 1 year ago
[BugFix] Allow use_cudagraph to work with dynamic VLLM_USE_V1 (#19390)
Richard Zou committed over 1 year ago
Add H20-3e fused MoE kernel tuning configs for Qwen3-235B-A22B-FP8 (#19401)
Xu Wenqing committed over 1 year ago
[Model] use AutoWeightsLoader for commandr (#19399)
py-andy-c committed over 1 year ago
[ROCm][V1] Adding ROCm to the list of plaforms using V1 by default (#19440)
Gregory Shtrasberg committed over 1 year ago