@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

main

[Minor] Fix a typo in .pt weight support (#2160)

Woosuk Kwon committed almost 3 years ago

Make sampler less blocking (#1889)

Antoni Baum committed almost 3 years ago

[Minor] Add Phi 2 to supported models (#2159)

Woosuk Kwon committed almost 3 years ago

[Minor] Fix xformers version (#2158)

Woosuk Kwon committed almost 3 years ago

Temporarily enforce eager mode for GPTQ models (#2154)

Woosuk Kwon committed almost 3 years ago

[Docs] Add CUDA graph support to docs (#2148)

Woosuk Kwon committed almost 3 years ago

Remove dependency on CuPy (#2152)

Woosuk Kwon committed almost 3 years ago

Pin PyTorch & xformers versions (#2155)

Woosuk Kwon committed almost 3 years ago

Fix all-reduce memory usage (#2151)

Woosuk Kwon committed almost 3 years ago

[Minor] Delete Llama tokenizer warnings (#2146)

Woosuk Kwon committed almost 3 years ago

Optimize model execution with CUDA graph (#1926)

Woosuk Kwon committed almost 3 years ago

Simplify weight loading logic (#2133)

Roy committed almost 3 years ago

[ROCm] Temporarily remove GPTQ ROCm support (#2138)

Woosuk Kwon committed almost 3 years ago

Add GPTQ support (#916)

CHU Tianxiang committed almost 3 years ago

Add a flag to include stop string in output text (#1976)

Yunfeng Bai committed almost 3 years ago

Avoid multiple redefinition (#1817)

Mingcan Xiang committed almost 3 years ago

Fix Dockerfile.rocm (#2101)

TJian committed almost 3 years ago

← Newer Page 1091 of 1120 Older →