A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 22:00 UTC
main
[Lora] Use safetensor keys instead of adapter_config.json to find unexpected modules. (#5909)
SangBin Cho committed over 2 years ago
[Frontend]: Support base64 embedding (#5935)
llmpros committed over 2 years ago
[ci][distributed] fix device count call
youkaichao committed over 2 years ago
[CI/Build] [3/3] Reorganize entrypoints tests (#5966)
Cyrus Leung committed over 2 years ago
[CI/Build] Reuse code for checking output consistency (#5988)
Cyrus Leung committed over 2 years ago
[CI/Build] Temporarily Remove Phi3-Vision from TP Test (#5989)
Roger Wang committed over 2 years ago
[Bugfix][CI/Build][Hardware][AMD] Install matching torchvision to fix AMD tests (#5949)
Matt Wong committed over 2 years ago
[ CI/Build ] LM Eval Harness Based CI Testing (#5838)
Robert Shaw committed over 2 years ago
[CI/Build] Add TP test for vision models (#5892)
Cyrus Leung committed over 2 years ago
[ CI/Build ] Added E2E Test For Compressed Tensors (#5839)
Robert Shaw committed over 2 years ago
[Kernel] Raise an exception in MoE kernel if the batch size is larger then 65k (#5939)
Cody Yu committed over 2 years ago
[Core] Optimize `SequenceStatus.is_finished` by switching to IntEnum (#5974)
Antoni Baum committed over 2 years ago
[Bugfix] Support `eos_token_id` from `config.json` (#5954)
Cyrus Leung committed over 2 years ago
[Misc] Update Phi-3-Vision Example (#5981)
Roger Wang committed over 2 years ago
[Bugfix] Fix precisions in Gemma 1 (#5913)
Woosuk Kwon committed over 2 years ago
[Kernel] Add punica dimensions for Granite 3b and 8b (#5930)
Joe Runde committed over 2 years ago
[Misc] Extend vLLM Metrics logging API (#5925)
William Lin committed over 2 years ago
[Bugfix] fix missing last itl in openai completions benchmark (#5926)
mcalman committed over 2 years ago
[Bugfix][TPU] Fix pad slot id (#5977)
Woosuk Kwon committed over 2 years ago
[Bugfix][TPU] Fix TPU sampler output (#5978)
Woosuk Kwon committed over 2 years ago