A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 09:30 UTC
main
[Bugfix] fix detokenizer shallow copy (#5919)
Aurick Qiao committed about 2 years ago
[torch.compile] auto infer dynamic_arg_dims from type annotation (#9589)
youkaichao committed about 2 years ago
[BugFix] Prevent exporting duplicate OpenTelemetry spans (#9017)
Ronen Schaffer committed about 2 years ago
[Frontend] Support custom request_id from request (#9550)
Yuhong Guo committed about 2 years ago
[Hardware][Intel CPU][DOC] Update docs for CPU backend (#6212)
Yuan committed about 2 years ago
[Bugfix] Eagle: change config name for fc bias (#9580)
gopalsarda committed about 2 years ago
[Model][VLM] Initialize support for Mono-InternVL model (#9528)
Isotr0py committed about 2 years ago
[Neuron] [Bugfix] Fix neuron startup (#9374)
xendo committed about 2 years ago
[CI/Build][LoRA] Temporarily fix long context failure issue (#9579)
Jee Jee Li committed about 2 years ago
[V1] Implement vLLM V1 [1/N] (#9289)
Woosuk Kwon committed about 2 years ago
[Hardware][CPU] using current_platform.is_cpu (#9536)
wangshuai09 committed about 2 years ago
support TP in qwen2 bnb (#9574)
chenqianfzh committed about 2 years ago
[Doc] Use shell code-blocks and fix section headers (#9508)
Rafael Vasquez committed about 2 years ago
[Core] Remove evictor_v1 (#9572)
Kuntai Du committed about 2 years ago
[CI/Build] Replaced some models on tests for smaller ones (#9570)
Wallas Henrique committed about 2 years ago
[Bugfix]: phi.py get rope_theta from config file (#9503)
Falko1 committed about 2 years ago
[Bugfix][OpenVINO] fix_dockerfile_openvino (#9552)
ngrozae committed about 2 years ago
[CI/Build] Remove unnecessary `fork_new_process` (#9484)
Cyrus Leung committed about 2 years ago
[Bugfix]: serialize config by value for --trust-remote-code (#6751)
Travis Johnson committed about 2 years ago
[core] move parallel sampling out from vllm core (#9302)
youkaichao committed about 2 years ago