A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 17:00 UTC
main
[Bugfix] Add missing attributes in mistral tokenizer (#8364)
Cyrus Leung committed about 2 years ago
[CI/Build] Excluding test_moe.py from AMD Kernels tests for investigation (#8373)
Alexey Kondratiev(AMD) committed about 2 years ago
[Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257)
Li, Jiang committed about 2 years ago
[Model][VLM] Add Qwen2-VL model support (#7905)
Yang Fan committed about 2 years ago
[Frontend] Create ErrorResponse instead of raising exceptions in run_batch (#8347)
Pooya Davoodi committed about 2 years ago
[model] Support for Llava-Next-Video model (#7559)
Yangshen⚡Deng committed about 2 years ago
[Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112)
Pavani Majety committed about 2 years ago
[Bugfix] Fix InternVL2 vision embeddings process with pipeline parallel (#8299)
Isotr0py committed about 2 years ago
[Misc] Skip loading extra bias for Qwen2-MOE GPTQ models (#8329)
Jee Jee Li committed about 2 years ago
[CI/Build][Kernel] Update CUTLASS to 3.5.1 tag (#8043)
Tyler Michael Smith committed about 2 years ago
[Core/Bugfix] pass VLLM_ATTENTION_BACKEND to ray workers (#8172)
William Lin committed about 2 years ago
[Bugfix] lookahead block table with cuda graph max capture (#8340)
Alexander Matveev committed about 2 years ago
Cody Yu committed about 2 years ago
[Misc] remove peft as dependency for prompt models (#8162)
Prashant Gupta committed about 2 years ago
[Spec Decode] Move ops.advance_step to flash attn advance_step (#8224)
Kevin Lin committed about 2 years ago
Fix ppc64le buildkite job (#8309)
sumitd2 committed about 2 years ago
[CI/Build] Enabling kernels tests for AMD, ignoring some of then that fail (#8130)
Alexey Kondratiev(AMD) committed about 2 years ago
[Frontend] Clean up type annotations for mistral tokenizer (#8314)
Cyrus Leung committed about 2 years ago
[CI/Build] enable ccache/scccache for HIP builds (#8327)
Daniele committed about 2 years ago
[Bugfix] Fix missing `post_layernorm` in CLIP (#8155)
Cyrus Leung committed about 2 years ago