A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 03:30 UTC
main
[Misc] remove unused jinaai_serving_reranking (#18878)
Abirdcfly committed over 1 year ago
[Docs] Clean up tables in supported_models.md (#20552)
Michael Yao committed over 1 year ago
[XPU] log clean up for XPU platform (#20553)
Yan Ma committed over 1 year ago
Add docstrings to url_schemes.py to improve readability (#20545)
Michael Yao committed over 1 year ago
[XPU][CI] add v1/core test in xpu hardware ci (#20537)
Liangliang Ma committed over 1 year ago
[CI/Build] Enable phi2 lora test (#20540)
Jee Jee Li committed over 1 year ago
Yang Yang committed over 1 year ago
[Misc] add a tip for pre-commit (#20536)
Reid committed over 1 year ago
Implement OpenAI Responses API [1/N] (#20504)
Woosuk Kwon committed over 1 year ago
[Bugfix] Add `use_cross_encoder` flag to use correct activation in `ClassifierPooler` (#20527)
Cyrus Leung committed over 1 year ago
[BugFix][Spec Decode] Fix spec token ids in model runner (#20530)
Woosuk Kwon committed over 1 year ago
[V1] Support any head size for FlexAttention backend (#20467)
Cyrus Leung committed over 1 year ago
[V0 deprecation] Remove V0 CPU/XPU/TPU backends (#20412)
Woosuk Kwon committed over 1 year ago
[Misc] call the pre-defined func (#20518)
Reid committed over 1 year ago
[Benchmark] Add support for multiple batch size benchmark through CLI in `benchmark_moe.py` (#20516)
Brayden Zhong committed over 1 year ago
[Frontend] Support image object in llm.chat (#19635)
Flora Feng committed over 1 year ago
[TPU][Bugfix] fix the MoE OOM issue (#20339)
Chengji Yao committed over 1 year ago
[BugFix] Fix: ImportError when building on hopper systems (#20513)
Lucas Wilkinson committed over 1 year ago
[Bugfix] Fix missing per_act_token parameter in compressed_tensors_moe (#20509)
Lucia Fang committed over 1 year ago
[BUG] Fix #20484. Support empty sequence in cuda penalty kernel (#20491)
Vadim Gimpelson committed over 1 year ago