A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
main
add option to shorten prompt print in log (#991)
leiwen83 committed about 3 years ago
Bump up the version to v0.1.7 (#1013)
Woosuk Kwon committed about 3 years ago
Use FP32 in RoPE initialization (#1004)
Woosuk Kwon committed about 3 years ago
Woosuk Kwon committed about 3 years ago
[Docs] Update installation page (#1005)
Woosuk Kwon committed about 3 years ago
fix: CUDA error when inferencing with Falcon-40B base model (#992)
Kyujin Cho committed about 3 years ago
Fix wrong dtype in PagedAttentionWithALiBi bias (#996)
Antoni Baum committed about 3 years ago
fix "tansformers_module" ModuleNotFoundError when load model with `trust_remote_code=True` (#871)
Jingru committed about 3 years ago
Bump up the version to v0.1.6 (#989)
Zhuohan Li committed about 3 years ago
Start background task in `AsyncLLMEngine.generate` (#988)
Antoni Baum committed about 3 years ago
Robert Irvine committed about 3 years ago
Bump up the version to v0.1.5 (#944)
Woosuk Kwon committed about 3 years ago
[FIX] Fix Alibi implementation in PagedAttention kernel (#945)
Zhuohan Li committed about 3 years ago
Enable safetensors loading for all models (#974)
Zhuohan Li committed about 3 years ago
Make `AsyncLLMEngine` more robust & fix batched abort (#969)
Antoni Baum committed about 3 years ago
Bum up transformers version (#976)
Woosuk Kwon committed about 3 years ago
Set torch default dtype in a context manager (#971)
Antoni Baum committed about 3 years ago
[BugFix] Implement RoPE for GPT-J (#941)
Woosuk Kwon committed about 3 years ago
Use queue for finished requests (#957)
Antoni Baum committed about 3 years ago
Clean up kernel unit tests (#938)
Woosuk Kwon committed about 3 years ago