A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 04:30 UTC
main
[CI] Expand test_guided_generate to test all backends (#11313)
Michael Goin committed almost 2 years ago
[Model] IBM Granite 3.1 (#11307)
Travis Johnson committed almost 2 years ago
[Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311)
Tyler Michael Smith committed almost 2 years ago
[V1] VLM - enable processor cache by default (#11305)
Alexander Matveev committed almost 2 years ago
[Bugfix] fix minicpmv test (#11304)
Joe Runde committed almost 2 years ago
[FIX] update openai version (#11287)
Kunshang Ji committed almost 2 years ago
[CI][Misc] Remove Github Action Release Workflow (#11274)
Simon Mo committed almost 2 years ago
[Bugfix] Fix broken phi3-v mm_processor_kwargs tests (#11263)
Isotr0py committed almost 2 years ago
[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995)
Dipika Sikka committed almost 2 years ago
[MISC][XPU]update ipex link for CI fix (#11278)
Yan Ma committed almost 2 years ago
[Bugfix] Fix guided decoding with tokenizer mode mistral (#11046)
Wallas Henrique committed almost 2 years ago
[Bugfix] Restore support for larger block sizes (#11259)
Konrad Zawora committed almost 2 years ago
[V1] Prefix caching for vision language models (#11187)
Cody Yu committed almost 2 years ago
[Bugfix] Set temperature=0.7 in test_guided_choice_chat (#11264)
Michael Goin committed almost 2 years ago
[Bugfix] Fix request cancellation without polling (#11190)
Joe Runde committed almost 2 years ago
[Misc] Allow passing logits_soft_cap for xformers backend (#11252)
Isotr0py committed almost 2 years ago
[Misc] Kernel Benchmark for `RMSNorm` (#11241)
Roger Wang committed almost 2 years ago
[V1][Core] Use weakref.finalize instead of atexit (#11242)
Tyler Michael Smith committed almost 2 years ago
[Platform] platform agnostic for EngineArgs initialization (#11225)
wangxiyuan committed almost 2 years ago
[V1][VLM] Proper memory profiling for image language models (#11210)
Roger Wang committed almost 2 years ago