A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 03:00 UTC
main
[Bugfix] Add kv_scale input parameter to CPU backend (#3840)
Woosuk Kwon committed over 2 years ago
[Core] Enable hf_transfer by default if available (#3817)
Michael Feil committed over 2 years ago
[BugFix] Pass tokenizer_config to local_tokenizer_group (#3754)
Tao He committed over 2 years ago
[Misc] Publish 3rd meetup slides (#3835)
Woosuk Kwon committed over 2 years ago
Enable scaled FP8 (e4m3fn) KV cache on ROCm (AMD GPU) (#3290)
Adrian Abeyta committed over 2 years ago
[3/N] Refactor scheduler for chunked prefill scheduling (#3550)
SangBin Cho committed over 2 years ago
[Doc] Update contribution guidelines for better onboarding (#3819)
Michael Feil committed over 2 years ago
[Doc] Update README.md (#3806)
Robert Shaw committed over 2 years ago
[BugFix] Use different mechanism to get vllm version in `is_cpu()` (#3804)
Nick Hill committed over 2 years ago
[Speculative decoding] Adding configuration object for speculative decoding (#3706)
Cade Daniel committed over 2 years ago
[CI/Build] 0.4.0.post1, fix sm 7.0/7.5 binary (#3803)
youkaichao committed over 2 years ago
[Bugfix] Add `__init__.py` files for `vllm/core/block/` and `vllm/spec_decode/` (#3798)
Michael Goin committed over 2 years ago
Fix early CUDA init via get_architecture_class_name import (#3770)
leiwen83 committed over 2 years ago
[CI/Build] fix TORCH_CUDA_ARCH_LIST in wheel build (#3801)
youkaichao committed over 2 years ago
[Doc] Fix vLLMEngine Doc Page (#3791)
Roger Wang committed over 2 years ago
[Frontend][Bugfix] allow using the default middleware with a root path (#3788)
A-Mahla committed over 2 years ago
[HotFix] [CI/Build] Minor fix for CPU backend CI (#3787)
bigPYJ1151 committed over 2 years ago
[Hardware][Intel] Add CPU inference backend (#3634)
bigPYJ1151 committed over 2 years ago
Cade Daniel committed over 2 years ago
[Misc] Add support for new autogptq checkpoint_format (#3689)
Qubitium committed over 2 years ago