@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 15, 23:30 UTC

main

[Misc] Fix typo at comments at metrics.py (#13024)

Jewon Lee committed almost 2 years ago

Set `torch_dtype` in `TransformersModel` (#13088)

Harry Mellor committed almost 2 years ago

[V1][Metrics] Add several request timing histograms (#12644)

Mark McLoughlin committed almost 2 years ago

[Bugfix] fix flaky test (#13089)

மனோஜ்குமார் பழனிச்சாமி committed almost 2 years ago

[ROCm] Using a more precise memory profiling (#12624)

Gregory Shtrasberg committed almost 2 years ago

[executor] init `local_rank` as device index (#13027)

Mengqing Cao committed almost 2 years ago

[Misc] LoRA - Refactor Punica ops tests (#12970)

Varun Sundar Rabindranath committed almost 2 years ago

[Core] Don't do platform detection at import time (#12933)

Russell Bryant committed almost 2 years ago

[Benchmark] Add BurstGPT to benchmark_serving (#13063)

Woosuk Kwon committed almost 2 years ago

[Docs] Annouce Meta Meetup (#13065)

Simon Mo committed almost 2 years ago

[Model] Ultravox Model: Support v0.5 Release (#12912)

Farzad Abdolhosseini committed almost 2 years ago

Fix seed parameter behavior in vLLM (#13007)

மனோஜ்குமார் பழனிச்சாமி committed almost 2 years ago

← Newer Page 894 of 1120 Older →