A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 07:00 UTC
main
[misc] improve error message (#10553)
youkaichao committed almost 2 years ago
[CI][Installation] Avoid uploading CUDA 11.8 wheel (#10535)
Yunmeng committed almost 2 years ago
[V1] Fix Compilation config & Enable CUDA graph by default (#10528)
Woosuk Kwon committed almost 2 years ago
[8/N] enable cli flag without a space (#10529)
youkaichao committed almost 2 years ago
[Bugfix] Allow token ID-only inputs in Qwen2-Audio (#10536)
Cyrus Leung committed almost 2 years ago
[Misc] Suppress duplicated logging regarding multimodal input pipeline (#10530)
Roger Wang committed almost 2 years ago
[Kernel] Register punica ops directly (#10522)
Jee Jee Li committed almost 2 years ago
[Bug]: When apply continue_final_message for OpenAI server, the "echo":false is ignored (#10180)
Chauncey committed almost 2 years ago
[Bugfix] Embedding model pooling_type equals ALL and multi input's bug (#10494)
Xiaoyu Zhang committed almost 2 years ago
[Model] Expose `dynamic_image_size` as mm_processor_kwargs for InternVL2 models (#10518)
Isotr0py committed almost 2 years ago
fix the issue that len(tokenizer(prompt)["input_ids"]) > prompt_len (#10524)
Wang, Yi committed almost 2 years ago
[Model] Add Support for Multimodal Granite Models (#10291)
Alex Brooks committed almost 2 years ago
[Doc] fix a small typo in docstring of llama_tool_parser (#10513)
Zhong Qishuai committed almost 2 years ago
[platforms] improve error message for unspecified platforms (#10520)
youkaichao committed almost 2 years ago
[Misc] Increase default video fetch timeout (#10495)
Cyrus Leung committed almost 2 years ago
[torch.compile] Inductor code caching fix (#10273)
Luka Govedič committed almost 2 years ago
[Platforms] Add `device_type` in `Platform` (#10508)
Mengqing Cao committed almost 2 years ago
[Core] Add Sliding Window Support with Flashinfer (#10462)
Pavani Majety committed almost 2 years ago
[torch.compile] limit inductor threads and lazy import quant (#10482)
youkaichao committed almost 2 years ago
[TPU] Implement prefix caching for TPUs (#10307)
Woosuk Kwon committed almost 2 years ago