@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 18:15 UTC

main

[Misc] Enhance prefix-caching benchmark tool (#6568)

Jiaxin Shan committed about 2 years ago

[Misc] update fp8 to use `vLLMParameter` (#7437)

Dipika Sikka committed about 2 years ago

[Bugfix] Don't build machete on cuda <12.0 (#7757)

Lucas Wilkinson committed about 2 years ago

[Misc] Use torch.compile for GemmaRMSNorm (#7642)

Woosuk Kwon committed about 2 years ago

[ci] refine dependency for distributed tests (#7776)

youkaichao committed about 2 years ago

[TPU] Avoid initializing TPU runtime in is_tpu (#7763)

Woosuk Kwon committed about 2 years ago

[Model] Add UltravoxModel and UltravoxConfig (#7615)

Peter Salas committed about 2 years ago

[misc] Add Torch profiler support (#7451)

William Lin committed about 2 years ago

[Frontend] Improve Startup Failure UX (#7716)

Robert Shaw committed about 2 years ago

← Newer Page 999 of 1120 Older →