A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 08:00 UTC
main
Qing committed over 3 years ago
[OPTIMIZATION] Optimizes the single_query_cached_kv_attention kernel (#420)
Dean Leitersdorf committed over 3 years ago
fix: incorrect bigcode attention heads num (#676)
Wen Sun committed over 3 years ago
Bump up version to 0.1.3 (#657)
Zhuohan Li committed over 3 years ago
[Doc] Add Baichuan 13B to supported models (#656)
Zhuohan Li committed over 3 years ago
Woosuk Kwon committed over 3 years ago
[BUG FIX] upgrade fschat version to 0.2.23 (#650)
YHPeter committed over 3 years ago
Add Falcon support (new) (#592)
Zhuohan Li committed over 3 years ago
Fix log message in scheduler (#652)
Lily Liu committed over 3 years ago
fix baichuan for different position embedding for 7b and 13b models (#643)
Song committed over 3 years ago
Qing committed over 3 years ago
Chaofan Lin committed over 3 years ago
fix ModuleNotFoundError (#599)
Fang li committed over 3 years ago
[Fix] fix import error of RayWorker (#604) (#605)
Xudong Zhang committed over 3 years ago
[Fix] Add model sequence length into model config (#575)
Zhuohan Li committed over 3 years ago
[Fix] Add chat completion Example and simplify dependencies (#576)
Zhuohan Li committed over 3 years ago
Add Baichuan-7B to README (#494)
Zhuohan Li committed over 3 years ago
fixed tensor parallel is not defined (#564)
MoeedDar committed over 3 years ago
Fix paged attention testing. (#495)
Tao Peng committed over 3 years ago
[Fix] Fix GPTBigcoder for distributed execution (#503)
Zhuohan Li committed over 3 years ago