@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 09:15 UTC

main

Implement presence and frequency penalties (#95)

Woosuk Kwon committed over 3 years ago

Support top-k sampling (#94)

Woosuk Kwon committed over 3 years ago

Log system stats (#90)

Woosuk Kwon committed over 3 years ago

Rename variables and methods (#91)

Woosuk Kwon committed over 3 years ago

Update sample prompts in `simple_server.py` (#89)

Woosuk Kwon committed over 3 years ago

Use slow tokenizer for LLaMA (#84)

Woosuk Kwon committed over 3 years ago

Enhance model loader (#83)

Woosuk Kwon committed over 3 years ago

Refactor system architecture (#82)

Woosuk Kwon committed over 3 years ago

Add a system logger (#85)

Woosuk Kwon committed over 3 years ago

[Minor] Fix a dtype bug (#79)

Woosuk Kwon committed over 3 years ago

Replace FlashAttention with xformers (#70)

Woosuk Kwon committed over 3 years ago

Use dtype from model config & Add Dolly V2 (#63)

Woosuk Kwon committed over 3 years ago

Add support for GPT-2 (#60)

Woosuk Kwon committed over 3 years ago

Fix a bug in attention kernel (#68)

Woosuk Kwon committed over 3 years ago

Support bfloat16 data type (#54)

Woosuk Kwon committed over 3 years ago

Refactor attention kernels (#53)

Woosuk Kwon committed over 3 years ago

New weight loader without np copy (#52)

Zhuohan Li committed over 3 years ago

Add an option to launch cacheflow without ray (#51)

Zhuohan Li committed over 3 years ago

← Newer Page 1114 of 1120 Older →