@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 09:15 UTC

main

Add support for GPT-NeoX (Pythia) (#50)

Woosuk Kwon committed over 3 years ago

Change plotting script

Woosuk Kwon committed over 3 years ago

Memcpy kernel for flash attention (#29)

Siyuan (Ryans) Zhuang committed over 3 years ago

Support block size 32 (#35)

Woosuk Kwon committed over 3 years ago

Add an option to use dummy model weights (#33)

Woosuk Kwon committed over 3 years ago

Add CUDA graph-based all reduce launcher (#26)

Woosuk Kwon committed over 3 years ago

Optimize data movement (#20)

Woosuk Kwon committed over 3 years ago

Merge QKV into one linear layer (#15)

Zhuohan Li committed over 3 years ago

Add ninja to dependency (#21)

Woosuk Kwon committed over 3 years ago

Use FP32 for log probabilities (#19)

Woosuk Kwon committed over 3 years ago

Add custom kernel for RMS normalization (#16)

Woosuk Kwon committed over 3 years ago

Optimize tensor parallel execution speed (#17)

Zhuohan Li committed over 3 years ago

← Newer Page 1115 of 1120 Older →