@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

main

Bump up to v0.2.5 (#2095)

Woosuk Kwon committed almost 3 years ago

Optimize Mixtral with expert parallelism (#2090)

Antoni Baum committed almost 3 years ago

[Docs] Add notes on ROCm-supported models (#2087)

Woosuk Kwon committed almost 3 years ago

Implement lazy model loader (#2044)

Woosuk Kwon committed almost 3 years ago

Fix peak memory profiling (#2031)

Woosuk Kwon committed almost 3 years ago

Update Dockerfile to build Megablocks (#2042)

Simon Mo committed almost 3 years ago

Support MPT with GQA (#1938)

Megha Agarwal committed almost 3 years ago

Remove einops from requirements (#2049)

Woosuk Kwon committed almost 3 years ago

Upgrade transformers version to 4.36.0 (#2046)

Woosuk Kwon committed almost 3 years ago

[CI/CD] Upgrade PyTorch version to v2.1.1 (#2045)

Woosuk Kwon committed almost 3 years ago

Remove python 3.10 requirement (#2040)

Woosuk Kwon committed almost 3 years ago

Update Dockerfile to support Mixtral (#2027)

Simon Mo committed almost 3 years ago

Bump up to v0.2.4 (#2034)

Woosuk Kwon committed almost 3 years ago

Fix import error msg for megablocks (#2038)

Woosuk Kwon committed almost 3 years ago

[Minor] Fix type annotation in Mixtral (#2036)

Woosuk Kwon committed almost 3 years ago

Fix latency benchmark script (#2035)

Woosuk Kwon committed almost 3 years ago

← Newer Page 1092 of 1120 Older →