A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
main
bump version to v0.5.0.post1 (#5522)
Simon Mo committed over 2 years ago
[Misc] Fix arg names in quantizer script (#5507)
Allen.Dou committed over 2 years ago
[Hardware][Intel] Support CPU inference with AVX2 ISA (#5452)
Jie Fu (傅杰) committed over 2 years ago
Add `cuda_device_count_stateless` (#5473)
Antoni Baum committed over 2 years ago
[Kernel] Disable CUTLASS kernels for fp8 (#5505)
Tyler Michael Smith committed over 2 years ago
[CI/Build] Disable test_fp8.py (#5508)
Tyler Michael Smith committed over 2 years ago
youkaichao committed over 2 years ago
Revert "[Core] Remove unnecessary copies in flash attn backend" (#5478)
Antoni Baum committed over 2 years ago
Seperate dev requirements into lint and test (#5474)
Antoni Baum committed over 2 years ago
[MISC] Remove FP8 warning (#5472)
Cody Yu committed over 2 years ago
[Kernel] Factor out epilogues from cutlass kernels (#5391)
Tyler Michael Smith committed over 2 years ago
[Doc] Update LLaVA docs (#5437)
Cyrus Leung committed over 2 years ago
[CI/Build] Simplify OpenAI server setup in tests (#5100)
Cyrus Leung committed over 2 years ago
[Misc] Add vLLM version getter to utils (#5098)
Cyrus Leung committed over 2 years ago
[Docs] Add 4th meetup slides (#5509)
Woosuk Kwon committed over 2 years ago
[Hardware][Intel] Optimize CPU backend and add more performance tips (#4971)
Li, Jiang committed over 2 years ago
[Kernel] Tune Qwen2MoE kernel configurations with tp2,4 (#5497)
wenyujin333 committed over 2 years ago
[CI/Build][REDO] Add is_quant_method_supported to control quantization test configurations (#5466)
Michael Goin committed over 2 years ago
[Kernel] `w4a16` support for `compressed-tensors` (#5385)
Dipika Sikka committed over 2 years ago
[Bugfix]if the content is started with ":"(response of ping), client should i… (#5303)
Wang, Yi committed over 2 years ago