A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 19:30 UTC
main
[Kernel] Fix deprecation function warnings squeezellm quant_cuda_kernel (#6901)
Tyler Michael Smith committed over 2 years ago
[Core] Reduce unnecessary compute when logprobs=None (#6532)
Peng Guanwen committed over 2 years ago
[Kernel] Tuned FP8 Kernels for Ada Lovelace (#6677)
Varun Sundar Rabindranath committed over 2 years ago
[Model] Initialize support for InternVL2 series models (#6514)
Isotr0py committed over 2 years ago
[Misc] Pass cutlass_fp8_supported correctly in fbgemm_fp8 (#6871)
Elsa Granger committed over 2 years ago
Add Nemotron to PP_SUPPORTED_MODELS (#6863)
Michael Goin committed over 2 years ago
[Kernel] Increase precision of GPTQ/AWQ Marlin kernel (#6795)
Alexander Matveev committed over 2 years ago
[TPU] Reduce compilation time & Upgrade PyTorch XLA version (#6856)
Woosuk Kwon committed over 2 years ago
[Docs] Add RunLLM chat widget (#6857)
Chenggang Wu committed over 2 years ago
[Model] Initial support for BLIP-2 (#5920)
Cyrus Leung committed over 2 years ago
[CI/Build][Doc] Update CI and Doc for VLM example changes (#6860)
Roger Wang committed over 2 years ago
[bugfix] make args.stream work (#6831)
Wang Ran (汪然) committed over 2 years ago
[Bugfix] Fix VLM example typo (#6859)
Roger Wang committed over 2 years ago
[Misc][VLM][Doc] Consolidate offline examples for vision language models (#6858)
Roger Wang committed over 2 years ago
[Bugfix] torch.set_num_threads() in multiproc_gpu_executor (#6802)
Travis Johnson committed over 2 years ago
[Doc] Add missing mock import to docs `conf.py` (#6834)
Harry Mellor committed over 2 years ago
[Hardware][TPU] Implement tensor parallelism with Ray (#5871)
Woosuk Kwon committed over 2 years ago
[Model] H2O Danube3-4b (#6451)
Joe committed over 2 years ago
[Bugfix][Model] Jamba assertions and no chunked prefill by default for Jamba (#6784)
tomeras91 committed over 2 years ago
[Doc] add VLLM_TARGET_DEVICE=neuron to documentation for neuron (#6844)
omrishiv committed over 2 years ago