A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 08:00 UTC
main
GPTJConfig has no attribute rotary. (#532)
leegohi04517 committed over 3 years ago
Add support for LLaMA-2 (#505)
Zhuohan Li committed over 3 years ago
Fix bad assert in initialize_cluster if PG already exists (#526)
Antoni Baum committed over 3 years ago
Ray placement group support (#397)
Antoni Baum committed over 3 years ago
fix: enable trust-remote-code in api server & benchmark. (#509)
Ricardo Lu committed over 3 years ago
support trust_remote_code in benchmark (#518)
WRH committed over 3 years ago
fix(ray_utils): ignore re-init error (#465)
Massimiliano Pronesti committed over 3 years ago
hotfix attn alibi wo head mapping (#496)
Song committed over 3 years ago
Merge pull request #498 from MoeedDar/main
Lily Liu committed over 3 years ago
Fixed old name reference for max_seq_len
MoeedDar committed over 3 years ago
Lily Liu committed over 3 years ago
Add support for baichuan (#365)
codethazine committed over 3 years ago
[Fix] Fix the condition of max_seq_len (#477)
Zhuohan Li committed over 3 years ago
[Doc] Add doc for running vLLM on the cloud (#426)
Zhanghao Wu committed over 3 years ago
Offload port selection to OS (#467)
Zhangir Azerbayev committed over 3 years ago
Zhuohan Li committed over 3 years ago
Fix the `KeyError` when loading bloom-based models (#441)
Wen Sun committed over 3 years ago
add vocab padding for LLama(Support WizardLM) (#411)
panda committed over 3 years ago
Fix handling of special tokens in decoding. (#418)
xcnick committed over 3 years ago
fix: freeze pydantic to v1 (#429)
Keming committed over 3 years ago