A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 17:00 UTC
main
[CI/Build][ROCm] Enabling LoRA tests on ROCm (#7369)
alexeykondrat committed about 2 years ago
[MISC] Consolidate FP8 kv-cache tests (#8131)
Cody Yu committed about 2 years ago
[Bugfix] remove post_layernorm in siglip (#8106)
wnma committed about 2 years ago
chore: Update check-wheel-size.py to read MAX_SIZE_MB from env (#8103)
TimWang committed about 2 years ago
[Frontend] Multimodal support in offline chat (#8098)
Cyrus Leung committed about 2 years ago
[Model] Add Ultravox support for multiple audio chunks (#7963)
Peter Salas committed about 2 years ago
[Misc] Update fbgemmfp8 to use `vLLMParameters` (#7972)
Dipika Sikka committed about 2 years ago
[TPU][Bugfix] Use XLA rank for persistent cache path (#8137)
Woosuk Kwon committed about 2 years ago
[Benchmark] Add `--async-engine` option to benchmark_throughput.py (#7964)
Nick Hill committed about 2 years ago
[Misc] Update `GPTQ` to use `vLLMParameters` (#7976)
Dipika Sikka committed about 2 years ago
[CI] Change PR remainder to avoid at-mentions (#8134)
Simon Mo committed about 2 years ago
[TPU][Bugfix] Fix next_token_ids shape (#8128)
Woosuk Kwon committed about 2 years ago
Kevin H. Luu committed about 2 years ago
[CI/Build] make pip install vllm work in macos (for import only) (#8118)
tomeras91 committed about 2 years ago
[Misc] Raise a more informative exception in add/remove_logger (#7750)
Antoni Baum committed about 2 years ago
[Core] Optimize Async + Multi-step (#8050)
Alexander Matveev committed about 2 years ago
[CI] Only PR reviewers/committers can trigger CI on PR (#8124)
Kevin H. Luu committed about 2 years ago
[Performance] Enable chunked prefill and prefix caching together (#8120)
Cody Yu committed about 2 years ago
[Bugfix][VLM] Add fallback to SDPA for ViT model running on CPU backend (#8061)
Isotr0py committed about 2 years ago
[Bugfix] Fix single output condition in output processor (#7881)
Woosuk Kwon committed about 2 years ago