A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
main
Fix failing spec decode test (#9054)
sroy745 committed about 2 years ago
[misc] add forward context for attention (#9029)
youkaichao committed about 2 years ago
[Frontend] [Neuron] Parse literals out of override-neuron-config (#8959)
xendo committed about 2 years ago
[Core] Make BlockSpaceManagerV2 the default BlockManager to use. (#8678)
sroy745 committed about 2 years ago
[BugFix] Enforce Mistral ToolCall id constraint when using the Mistral tool call parser (#9020)
Guillaume Calmettes committed about 2 years ago
[Misc] log when using default MoE config (#8971)
Divakar Verma committed about 2 years ago
[Bugfix] example template should not add parallel_tool_prompt if tools is none (#9007)
Travis Johnson committed about 2 years ago
[Doc] Update Granite model docs (#9025)
Nick Hill committed about 2 years ago
[Model] Adding Granite MoE. (#8206)
Shawn Tan committed about 2 years ago
[OpenVINO] Enable GPU support for OpenVINO vLLM backend (#8192)
Sergey Shlyapnikov committed about 2 years ago
[Core] CUDA Graphs for Multi-Step + Chunked-Prefill (#8645)
Varun Sundar Rabindranath committed about 2 years ago
[Misc] Update Default Image Mapper Error Log (#8977)
Alex Brooks committed about 2 years ago
[Core] Combined support for multi-step scheduling, chunked prefill & prefix caching (#8804)
afeldman-nm committed about 2 years ago
[Spec Decode] (1/2) Remove batch expansion (#8839)
Lily Liu committed about 2 years ago
vlsav committed about 2 years ago
[Doc] Update list of supported models (#8987)
Cyrus Leung committed about 2 years ago
[Core] [Frontend] Priority scheduling for embeddings and in the OpenAI-API (#8965)
Sebastian Schoennenbeck committed about 2 years ago
[Bugfix] Fix Token IDs Reference for MiniCPM-V When Images are Provided With No Placeholders (#8991)
Alex Brooks committed about 2 years ago
[Bugfix] Fix Fuyu tensor parallel inference (#8986)
Isotr0py committed about 2 years ago
[Misc] add process_weights_after_loading for DummyLoader (#8969)
Divakar Verma committed about 2 years ago