A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 00:30 UTC
main
afeldman-nm committed over 2 years ago
[Kernel] Enhance MoE benchmarking & tuning script (#4921)
Woosuk Kwon committed over 2 years ago
[Bugfix] Add warmup for prefix caching example (#5235)
Zhuohan Li committed over 2 years ago
[Bugfix] Support `prompt_logprobs==0` (#5217)
Toshiki Kataoka committed over 2 years ago
[FRONTEND] OpenAI `tools` support named functions (#5032)
Breno Faria committed over 2 years ago
New CI template on AWS stack (#5110)
Kevin H. Luu committed over 2 years ago
[Misc]: Implement CPU/GPU swapping in BlockManagerV2 (#3834)
Kaiyang Chen committed over 2 years ago
[CI/BUILD] enable intel queue for longer CPU tests (#4113)
Yuan committed over 2 years ago
[Kernel] Pass a device pointer into the quantize kernel for the scales (#5159)
Tyler Michael Smith committed over 2 years ago
[Core] Remove unnecessary copies in flash attn backend (#5138)
Antoni Baum committed over 2 years ago
[Core] Support image processor (#4197)
Cyrus Leung committed over 2 years ago
[Misc] Simplify code and fix type annotations in `conftest.py` (#5118)
Cyrus Leung committed over 2 years ago
[Kernel][ROCm][AMD] enable fused topk_softmax kernel for moe layer (#4927)
Divakar Verma committed over 2 years ago
[Frontend][OpenAI] Support for returning max_model_len on /v1/models response (#4643)
Avinash Raj committed over 2 years ago
Update test_ignore_eos (#4898)
Simon Mo committed over 2 years ago
[BugFix] Prevent `LLM.encode` for non-generation Models (#5184)
Robert Shaw committed over 2 years ago
[Bugfix]: Fix issues related to prefix caching example (#5177) (#5180)
Daniil Arapov committed over 2 years ago
[Bugfix] Remove deprecated @abstractproperty (#5174)
Zhuohan Li committed over 2 years ago
[Feature][Kernel] Support bitsandbytes quantization and QLoRA (#4776)
chenqianfzh committed over 2 years ago
[Bugfix] Fix call to init_logger in openai server (#4765)
Nadav Shmayovits committed over 2 years ago