A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 03:15 UTC
main
[Platform] Move model arch check to platform (#11503)
Mengqing Cao committed almost 2 years ago
[BugFix] Fix quantization for all other methods (#11547)
Robert Shaw committed almost 2 years ago
[V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (#11534)
Robert Shaw committed almost 2 years ago
youkaichao committed almost 2 years ago
Robert Shaw committed almost 2 years ago
Woosuk Kwon committed almost 2 years ago
[V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (#11394)
Woosuk Kwon committed almost 2 years ago
Update openai_compatible_server.md (#11536)
Robert Shaw committed almost 2 years ago
[Docs] Document Deepseek V3 support (#11535)
Simon Mo committed almost 2 years ago
Simon Mo committed almost 2 years ago
[2/N] API Server: Avoid ulimit footgun (#11530)
Robert Shaw committed almost 2 years ago
[Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523)
Michael Goin committed almost 2 years ago
[1/N] API Server (Remove Proxy) (#11529)
Robert Shaw committed almost 2 years ago
[Doc] Add video example to openai client for multimodal (#11521)
Isotr0py committed almost 2 years ago
[Misc] Add placeholder module (#11501)
Cyrus Leung committed almost 2 years ago
[Model] Modify MolmoForCausalLM MLP (#11510)
Jee Jee Li committed almost 2 years ago
[V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681)
sroy745 committed almost 2 years ago
[Doc] Add `QVQ` and `QwQ` to the list of supported models (#11509)
Roger Wang committed almost 2 years ago
[Misc][LoRA] Fix LoRA weight mapper (#11495)
Jee Jee Li committed almost 2 years ago
Mypy checking for vllm/compilation (#11496)
Lucas Tucker committed almost 2 years ago