A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 18:15 UTC
main
[CI/Build]Reduce the time consumption for LoRA tests (#7396)
Jee Jee Li committed over 2 years ago
[Bugfix][CI] Import ray under guard (#7486)
Woosuk Kwon committed over 2 years ago
[TPU] Support multi-host inference (#7457)
Woosuk Kwon committed over 2 years ago
[misc][plugin] add plugin system implementation (#7426)
youkaichao committed over 2 years ago
[Misc] `compressed-tensors` code reuse (#7277)
Kyle Sayers committed over 2 years ago
[frontend] spawn engine process from api server process (#7484)
youkaichao committed over 2 years ago
Announce NVIDIA Meetup (#7483)
Simon Mo committed over 2 years ago
[Misc] Update `awq` and `awq_marlin` to use `vLLMParameters` (#7422)
Dipika Sikka committed over 2 years ago
[Misc] Update Fused MoE weight loading (#7334)
Dipika Sikka committed over 2 years ago
[Misc] Update `gptq_marlin` to use new vLLMParameters (#7281)
Dipika Sikka committed over 2 years ago
[Misc] Update LM Eval Tolerance (#7473)
Dipika Sikka committed over 2 years ago
[Frontend][Core] Add plumbing to support audio language models (#7446)
Peter Salas committed over 2 years ago
Revert "[Doc] Update supported_hardware.rst (#7276)" (#7467)
Woosuk Kwon committed over 2 years ago
[TPU] Suppress import custom_ops warning (#7458)
Woosuk Kwon committed over 2 years ago
[hardware] unify usage of is_tpu to current_platform.is_tpu() (#7102)
youkaichao committed over 2 years ago
[Bugfix] Fix weight loading for Chameleon when TP>1 (#7410)
Cyrus Leung committed over 2 years ago
[ci] Remove fast check cancel workflow (#7455)
Kevin H. Luu committed over 2 years ago
[Misc] improve logits processors logging message (#7435)
Andrew Wang committed over 2 years ago
[mypy] Misc. typing improvements (#7417)
Cyrus Leung committed over 2 years ago
[Core] Shut down aDAG workers with clean async llm engine exit (#7224)
Rui Qiao committed over 2 years ago