A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 15:45 UTC
main
[Misc] Add argument to disable FastAPI docs (#8554)
Jiaxin Shan committed about 2 years ago
[CI/Build] fix Dockerfile.cpu on podman (#8540)
Daniele committed about 2 years ago
[Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012)
Tyler Michael Smith committed about 2 years ago
[CI/Build] Excluding kernels/test_gguf.py from ROCm (#8520)
Alexey Kondratiev(AMD) committed about 2 years ago
[doc] improve installation doc (#8550)
youkaichao committed about 2 years ago
[Bugfix] Fix TP > 1 for new granite (#8544)
Joe Runde committed about 2 years ago
[Misc] Don't dump contents of kvcache tensors on errors (#8527)
Nick Hill committed about 2 years ago
[Model] Add mistral function calling format to all models loaded with "mistral" format (#8515)
Patrick von Platen committed about 2 years ago
[Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434)
chenqianfzh committed about 2 years ago
[Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631)
sroy745 committed about 2 years ago
[Benchmark] Support sample from HF datasets and image input for benchmark_serving (#8495)
Isotr0py committed about 2 years ago
[Misc] Limit to ray[adag] 2.35 to avoid backward incompatible change (#8509)
Rui Qiao committed about 2 years ago
[torch.compile] register allreduce operations as custom ops (#8526)
youkaichao committed about 2 years ago
[Misc][Bugfix] Disable guided decoding for mistral tokenizer (#8521)
Roger Wang committed about 2 years ago
[Frontend] Improve Nullable kv Arg Parsing (#8525)
Alex Brooks committed about 2 years ago
[refactor] remove triton based sampler (#8524)
Simon Mo committed about 2 years ago
[Bugfix] Fix 3.12 builds on main (#8510)
Joe Runde committed about 2 years ago
[misc] small qol fixes for release process (#8517)
Simon Mo committed about 2 years ago
[perf bench] set timeout to debug hanging (#8516)
Simon Mo committed about 2 years ago
[Bugfix] Bind api server port before starting engine (#8491)
Kevin Lin committed about 2 years ago