A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 11:00 UTC
main
Add missing content type headers to /ping and /health (#17036) (#17786)
Ximo Guanter committed over 1 year ago
fix broken test vllm:test_kernels - test_attention_selector.py::test_flash_attn (#17873)
tracelogfb committed over 1 year ago
[V1][Spec Decoding] Log accumulated metrics after system goes idle (#17913)
Mark McLoughlin committed over 1 year ago
[Hardware/NVIDIA/Kernel] Enable nvidia/DeepSeek-R1-FP4 Model (#16362)
Pavani Majety committed over 1 year ago
AMD conditional all test execution // new test groups (#17556)
Alexei-V-Ivanov-AMD committed over 1 year ago
Improve configs - the rest! (#17562)
Harry Mellor committed over 1 year ago
[V1][Spec Decoding] Include bonus tokens in mean acceptance length (#17908)
Mark McLoughlin committed over 1 year ago
Add option to use torch._inductor.standalone_compile (#17057)
Richard Zou committed over 1 year ago
Handle error when `str` passed to `/v1/audio/transcriptions` (#17909)
Harry Mellor committed over 1 year ago
Update CT WNA16MarlinMoE integration (#16666)
Michael Goin committed over 1 year ago
[Misc] Auto fallback to float16 for pre-Ampere GPUs when detected bfloat16 config (#17265)
Isotr0py committed over 1 year ago
[Misc] Add references in ray_serve_deepseek example (#17907)
Rui Qiao committed over 1 year ago
Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910)
Michael Goin committed over 1 year ago
[CI/Build] Automatically retry flaky tests (#17856)
Cyrus Leung committed over 1 year ago
[v1] Move block management logic from KVCacheManager to SpecializedManager (#17474)
Chen Zhang committed over 1 year ago
[BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864)
qli88 committed over 1 year ago
[Misc] add dify integration (#17895)
Reid committed over 1 year ago
Change `top_k` to be disabled with `0` (still accept `-1` for now) (#17773)
Harry Mellor committed over 1 year ago
Fix Whisper crash caused by invalid``` max_num_batched_tokens``` config (#17853)
inkcherry committed over 1 year ago
[Bugfix][CPU] Fix broken AVX2 CPU TP support (#17252)
Isotr0py committed over 1 year ago