A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 16:00 UTC
main
[V1][Core] Remove unused speculative config from scheduler (#15818)
Mark McLoughlin committed over 1 year ago
Fix Transformers backend compatibility check (#15290)
Harry Mellor committed over 1 year ago
[V1] TPU CI - Add basic perf regression test (#15414)
Alexander Matveev committed over 1 year ago
fix: change GB to GiB in logging close #14979 (#15807)
yihong committed over 1 year ago
[V1][Spec Decode] Remove deprecated spec decode config params (#15466)
shangmingc committed over 1 year ago
[Bugfix] Check dimensions of multimodal embeddings in V1 (#15816)
Cyrus Leung committed over 1 year ago
Upgrade `transformers` to `v4.50.3` (#13905)
Harry Mellor committed over 1 year ago
[Bugfix] Fix missing return value in load_weights method of adapters.py (#15542)
Mrm committed over 1 year ago
[Bugfix] Fix Crashing When Loading Modules With Batchnorm Stats (#15813)
Alex Brooks committed over 1 year ago
[Model] Update support for NemotronNAS models (#15008)
Naveassaf committed over 1 year ago
[V1] Fully Transparent Implementation of CPU Offloading (#15354)
youkaichao committed over 1 year ago
fix: better install requirement for install in setup.py (#15796)
yihong committed over 1 year ago
Recommend developing with Python 3.12 in developer guide (#15811)
Harry Mellor committed over 1 year ago
[Feature][ROCm]Enable fusion pass for torch.compile on ROCm (#15050)
Charlie Fu committed over 1 year ago
[Benchmark] Update Vision Arena Dataset and HuggingFaceDataset Setup (#15748)
Jennifer Zhao committed over 1 year ago
[Misc] update the comments (#15780)
Chengyang LIU committed over 1 year ago
[V1][Scheduler] Avoid calling `_try_schedule_encoder_inputs` for every request (#15778)
Woosuk Kwon committed over 1 year ago
fix: Comments to English for better dev experience (#15768)
yihong committed over 1 year ago
[Bugfix] Added `embed_is_patch` mask for fuyu model (#15731)
kYLe committed over 1 year ago
fix: lint fix a ruff checkout syntax error (#15767)
yihong committed over 1 year ago