@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

main

Add args for mTLS support (#3410)

Dan Clark committed over 2 years ago

Install `flash_attn` in Docker image (#3396)

Thomas Parnell committed over 2 years ago

Fix lint (#3388)

Antoni Baum committed over 2 years ago

Add batched RoPE kernel (#3095)

Terry committed over 2 years ago

add hf_transfer to requirements.txt (#3031)

Ronan McGovern committed over 2 years ago

Add kernel for GeGLU with approximate GELU (#3337)

Woosuk Kwon committed over 2 years ago

Fixes #1556 double free (#3347)

Breno Faria committed over 2 years ago

docs: Add BentoML deployment doc (#3336)

Sherlock Xu committed over 2 years ago

Add distributed model executor abstraction (#3191)

Zhuohan Li committed over 2 years ago

[docs] Add LoRA support information for models (#3299)

Philipp Moritz committed over 2 years ago

← Newer Page 1077 of 1120 Older →