@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 09:15 UTC

main

Implement LLaMA (#9)

Woosuk Kwon committed over 3 years ago

Refactor the test code for attention kernels (#13)

Woosuk Kwon committed over 3 years ago

FastAPI-based working frontend (#10)

Zhuohan Li committed over 3 years ago

Minor

Woosuk Kwon committed over 3 years ago

Support tensor parallel (#2)

Zhuohan Li committed over 3 years ago

Add miscellaneous updates (#8)

Woosuk Kwon committed over 3 years ago

Support beam search & parallel generation (#7)

Woosuk Kwon committed over 3 years ago

Fix a bug in 1D input shape (#5)

Woosuk Kwon committed over 3 years ago

Fix a bug in tying OPT embeddings (#1)

Woosuk Kwon committed over 3 years ago

Add README

Woosuk Kwon committed over 3 years ago

Clean up the server script

Woosuk Kwon committed over 3 years ago

[Minor] Fix printing format

Woosuk Kwon committed over 3 years ago

Add Frontend

Woosuk Kwon committed over 3 years ago

Add max_num_steps to SamplingParams

Woosuk Kwon committed over 3 years ago

Add is_finished

Woosuk Kwon committed over 3 years ago

← Newer Page 1116 of 1120 Older →