A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 00:45 UTC
main
Bugfix for whisper quantization due to fake k_proj bias (#12524)
Michael Goin committed almost 2 years ago
[V1][Metrics] Add TTFT and TPOT histograms (#12530)
Mark McLoughlin committed almost 2 years ago
[Doc] Convert docs to use colon fences (#12471)
Harry Mellor committed almost 2 years ago
[Frontend] Support reasoning content for deepseek r1 (#12473)
Ce Gao committed almost 2 years ago
[TPU] Add example for profiling TPU inference (#12531)
Michael Goin committed almost 2 years ago
[Kernel] Pipe attn_logits_soft_cap through paged attention TPU kernels (#12482)
fenghuizhang committed almost 2 years ago
[V1][Metrics] Add per-request prompt/generation_tokens histograms (#12516)
Mark McLoughlin committed almost 2 years ago
Do not run `suggestion` `pre-commit` hook multiple times (#12521)
Harry Mellor committed almost 2 years ago
Replace missed warning_once for rerank API (#12472)
Michael Goin committed almost 2 years ago
[V1][Metrics] Hook up IterationStats for Prometheus metrics (#12478)
Mark McLoughlin committed almost 2 years ago
[Doc] Fix typo for x86 CPU installation (#12514)
Jun Duan committed almost 2 years ago
[VLM] Merged multi-modal processor and V1 support for Qwen-VL (#12504)
Cyrus Leung committed almost 2 years ago
[Core] Make raw_request optional in ServingCompletion (#12503)
Sebastian Schoennenbeck committed almost 2 years ago
[V1] Include Engine Version in Logs (#12496)
Robert Shaw committed almost 2 years ago
Update README.md with V1 alpha release (#12495)
Roger Wang committed almost 2 years ago
[CI] fix pre-commit error (#12494)
Mengqing Cao committed almost 2 years ago
[FEATURE] Enables offline /score for embedding models (#12021)
Gabriel Marinho committed almost 2 years ago
[CI/Build] Fixed the xla nightly issue report in #12451 (#12453)
Hossein Sarshar committed almost 2 years ago
Fix bad path in prometheus example (#12481)
Michael Goin committed almost 2 years ago
[Neuron][Kernel] NKI-based flash-attention kernel with paged KV cache (#11277)
Liangfu Chen committed almost 2 years ago