@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 08:00 UTC

main

fix: typo (#948)

Wen Sun committed about 3 years ago

Initialize AsyncLLMEngine bg loop correctly (#943)

Antoni Baum committed about 3 years ago

Refactor AsyncLLMEngine (#880)

Antoni Baum committed about 3 years ago

Avoid compiling kernels for double data type (#933)

Woosuk Kwon committed about 3 years ago

Add tests for models (#922)

Woosuk Kwon committed about 3 years ago

Fix README.md Link (#927)

Zhuohan Li committed about 3 years ago

[Docs] Minor fixes in supported models (#920)

Woosuk Kwon committed about 3 years ago

fix: bug fix when penalties are negative (#913)

Dong-Yong Lee committed about 3 years ago

Improve _prune_hidden_states micro-benchmark (#707)

Aman Gupta Karmani committed about 3 years ago

Add acknowledgement to a16z grant

Zhuohan Li committed about 3 years ago

Accelerate LLaMA model loading (#234)

JFDuan committed about 3 years ago

use flash-attn via xformers (#877)

Aman Gupta Karmani committed about 3 years ago

[Fix] Fix a condition for ignored sequences (#867)

Zhuohan Li committed about 3 years ago

Add support for CodeLlama (#854)

Antoni Baum committed about 3 years ago

Bump up the version to v0.1.4 (#846)

Woosuk Kwon committed about 3 years ago

Set replacement=True in torch.multinomial (#858)

Woosuk Kwon committed about 3 years ago

← Newer Page 1105 of 1120 Older →