@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 08:00 UTC

main

Add quickstart guide (#148)

Zhuohan Li committed over 3 years ago

Change the name to vLLM (#150)

Woosuk Kwon committed over 3 years ago

Rename servers to engines (#152)

Zhuohan Li committed over 3 years ago

[Minor] Fix benchmark_throughput.py (#151)

Woosuk Kwon committed over 3 years ago

Add script for benchmarking serving throughput (#145)

Woosuk Kwon committed over 3 years ago

Remove redundant code in ColumnParallelLinear (#146)

Woosuk Kwon committed over 3 years ago

Non-streaming simple fastapi server (#144)

Zhuohan Li committed over 3 years ago

Support FP32 (#141)

Woosuk Kwon committed over 3 years ago

[PyPI] Packaging for PyPI distribution (#140)

Woosuk Kwon committed over 3 years ago

[Docs] Write the `Adding a New Model` section (#138)

Woosuk Kwon committed over 3 years ago

Fix various issues of async servers (#135)

Zhuohan Li committed over 3 years ago

Add docstrings for LLM (#137)

Woosuk Kwon committed over 3 years ago

Document supported models (#127)

Woosuk Kwon committed over 3 years ago

Add .readthedocs.yaml (#136)

Woosuk Kwon committed over 3 years ago

Add throughput benchmarking script (#133)

Woosuk Kwon committed over 3 years ago

Enable LLaMA fast tokenizer (#132)

Woosuk Kwon committed over 3 years ago

Add a doc for installation (#128)

Woosuk Kwon committed over 3 years ago

← Newer Page 1112 of 1120 Older →