A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 23:30 UTC
main
[Misc] Fix typo at comments at metrics.py (#13024)
Jewon Lee committed almost 2 years ago
Set `torch_dtype` in `TransformersModel` (#13088)
Harry Mellor committed almost 2 years ago
[V1][Metrics] Add several request timing histograms (#12644)
Mark McLoughlin committed almost 2 years ago
[Bugfix] fix flaky test (#13089)
மனோஜ்குமார் பழனிச்சாமி committed almost 2 years ago
[Platform] add pre_register_and_update function (#12432)
wangxiyuan committed almost 2 years ago
[Build] Fix cuda link target of cumem_allocator in CPU env (#12863)
Yuhong Guo committed almost 2 years ago
[ROCm] Using a more precise memory profiling (#12624)
Gregory Shtrasberg committed almost 2 years ago
[executor] init `local_rank` as device index (#13027)
Mengqing Cao committed almost 2 years ago
[V1][Metrics] Add GPU prefix cache hit rate % gauge (#12592)
Cody Yu committed almost 2 years ago
[Bugfix]: Reasoning output bug according to the chat template change (#13025)
Ce Gao committed almost 2 years ago
[Misc] LoRA - Refactor Punica ops tests (#12970)
Varun Sundar Rabindranath committed almost 2 years ago
[Core] Don't do platform detection at import time (#12933)
Russell Bryant committed almost 2 years ago
[Benchmark] Add BurstGPT to benchmark_serving (#13063)
Woosuk Kwon committed almost 2 years ago
[Bugfix] Support missing tool parameters in mistral tokenizer (#12884)
Florian Greinacher committed almost 2 years ago
[Docs] Annouce Meta Meetup (#13065)
Simon Mo committed almost 2 years ago
[V1][Minor] Move scheduler outputs to a separate file (#13062)
Woosuk Kwon committed almost 2 years ago
[misc] Fix setup.py condition to avoid AMD from being mistaken with CPU (#13022)
Kevin H. Luu committed almost 2 years ago
[Model] Ultravox Model: Support v0.5 Release (#12912)
Farzad Abdolhosseini committed almost 2 years ago
Fix seed parameter behavior in vLLM (#13007)
மனோஜ்குமார் பழனிச்சாமி committed almost 2 years ago
[Bugfix] Clean up and fix multi-modal processors (#13012)
Cyrus Leung committed almost 2 years ago