A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 22:15 UTC
main
[HTTP Server] Make model param optional in request (#13568)
Keyun Tong committed over 1 year ago
docs: Add a note on full CI run in contributing guide (#13646)
Yuan Tang committed over 1 year ago
[Misc] Fix yapf linting tools etc not running on pre-commit (#13695)
Isotr0py committed over 1 year ago
[Attention] MLA with chunked prefill (#12639)
Lucas Wilkinson committed over 1 year ago
fix typo of grafana dashboard, with correct datasource (#13668)
John Zheng committed over 1 year ago
[Bugfix][CPU] Fix cpu all-reduce using native pytorch implementation (#13586)
Isotr0py committed over 1 year ago
[Kernel]Add streamK for block-quantized CUTLASS kernels (#12978)
leoneo committed over 1 year ago
[ci] Fix metrics test model path (#13635)
Kevin H. Luu committed over 1 year ago
[FEATURE] Enables /score endpoint for embedding models (#12846)
Gabriel Marinho committed over 1 year ago
Missing comment explaining VDR variable in GGUF kernels (#13290)
Szymon Ożóg committed over 1 year ago
[V1][Sampler] Avoid an operation during temperature application (#13587)
Nick Hill committed over 1 year ago
[Bugfix] Add `mm_processor_kwargs` to chat-related protocols (#13644)
Roger Wang committed over 1 year ago
Use pre-commit to update `requirements-test.txt` (#13617)
Harry Mellor committed over 1 year ago
[NVIDIA] Fix an issue to use current stream for the nvfp4 quant (#13632)
Kaixi Hou committed over 1 year ago
[Misc] Adding script to setup ray for multi-node vllm deployments (#12913)
Edwin Hernandez committed over 1 year ago
Add llmaz as another integration (#13643)
Kante Yin committed over 1 year ago
[Neuron][Kernel] Vectorize KV cache load in FlashPagedAttention to maximize DMA bandwidth (#13245)
Lingfan Yu committed over 1 year ago
[Bugfix] Fix max_num_batched_tokens for MLA (#13620)
Michael Goin committed over 1 year ago
[Frontend] Add backend-specific options for guided decoding (#13505)
Joe Runde committed over 1 year ago
fix neuron performance issue (#13589)
ajayvohra2005 committed over 1 year ago