A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 08:15 UTC
main
bugfix: fix the bug that stream generate not work (#2756)
Zhao Yingzhuo committed about 2 years ago
[doc] explaining the integration with huggingface (#10173)
youkaichao committed about 2 years ago
[bugfix] fix broken tests of mlp speculator (#10177)
youkaichao committed about 2 years ago
[Bugfix] Enable some fp8 and quantized fullgraph tests (#10171)
bnellnm committed about 2 years ago
[CI/Build] Add run-hpu-test.sh script (#10167)
Chendi.Xue committed about 2 years ago
[5/N] pass the whole config to model (#9983)
youkaichao committed about 2 years ago
[Doc] Adjust RunLLM location (#10176)
Cyrus Leung committed about 2 years ago
[CI/Build] Fix VLM broadcast tests `tensor_parallel_size` passing (#10161)
Isotr0py committed about 2 years ago
[Bugfix] Ignore GPTQ quantization of Qwen2-VL visual module (#10169)
Michael Goin committed about 2 years ago
[0/N] Rename `MultiModalInputs` to `MultiModalKwargs` (#10040)
Cyrus Leung committed about 2 years ago
[CI/Build] Adding timeout in CPU CI to avoid CPU test queue blocking (#6892)
Li, Jiang committed about 2 years ago
rasmith committed about 2 years ago
[Bugfix] SymIntArrayRef expected to contain concrete integers (#10170)
bnellnm committed about 2 years ago
[torch.compile] Fuse RMSNorm with quant (#9138)
Luka Govedič committed about 2 years ago
Rename vllm.logging to vllm.logging_utils (#10134)
Florian Zimmermeister committed about 2 years ago
[CI/Build] Ignore .gitignored files for shellcheck (#10162)
Luka Govedič committed about 2 years ago
[V1] Fix non-cudagraph op name (#10166)
Woosuk Kwon committed about 2 years ago
[Misc] Improve Web UI (#10090)
Rafael Vasquez committed about 2 years ago
Disable spec-decode + chunked-prefill for draft models with tensor parallelism > 1 (#10136)
sroy745 committed about 2 years ago
Fix edge case Mistral tokenizer (#10152)
Patrick von Platen committed about 2 years ago