A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
main
Build CUDA11.8 wheels for release (#1596)
Woosuk Kwon committed about 3 years ago
Fix input_metadata.selected_token_indices in worker prepare_inputs (#1546)
Yanming W committed about 3 years ago
Zhuohan Li committed about 3 years ago
GoHomeToMacDonal committed about 3 years ago
Roy committed about 3 years ago
Add Quantization and AutoAWQ to docs (#1235)
Casper committed about 3 years ago
Antoni Baum committed about 3 years ago
Added logits processor API to sampling params (#1469)
Noam Gat committed about 3 years ago
lots-o committed about 3 years ago
Force paged attention v2 for long contexts (#1510)
Antoni Baum committed about 3 years ago
Woosuk Kwon committed about 3 years ago
Add `/health` Endpoint for both Servers (#1540)
Fluder-Paradyne committed about 3 years ago
[BugFix] Set engine_use_ray=True when TP>1 (#1531)
ljss committed about 3 years ago
Add `MptForCausalLM` key in model_loader (#1526)
Wenfei Yan committed about 3 years ago
[Small] Formatter only checks lints in changed files (#1528)
Cade Daniel committed about 3 years ago
Fix integer overflows in attention & cache ops (#1514)
Woosuk Kwon committed about 3 years ago
Stephen Krider committed about 3 years ago
[Fix] Fix duplicated logging messages (#1524)
Zhuohan Li committed about 3 years ago
Add support for `spaces_between_special_tokens`
Dan Lord committed about 3 years ago
Add py.typed so consumers of vLLM can get type checking (#1509)
Jared Roesch committed about 3 years ago