A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
main
Woosuk Kwon committed over 2 years ago
[CI/Build] Fix Args for `_get_logits_warper` in Sampler Test (#5922)
Roger Wang committed over 2 years ago
[BugFix] Fix `min_tokens` behaviour for multiple eos tokens (#5849)
Nick Hill committed over 2 years ago
[BugFix] Fix `MLPSpeculator` handling of `num_speculative_tokens` (#5876)
Nick Hill committed over 2 years ago
[doc][misc] add note for Kubernetes users (#5916)
youkaichao committed over 2 years ago
[Model][Bugfix] Implicit model flags and reenable Phi-3-Vision (#5896)
Cyrus Leung committed over 2 years ago
[CI/Build] [1/3] Reorganize entrypoints tests (#5526)
Cyrus Leung committed over 2 years ago
[Bugfix] Fix img_sizes Parsing in Phi3-Vision (#5888)
Roger Wang committed over 2 years ago
[Model] Add base class for LoRA-supported models (#5018)
Cyrus Leung committed over 2 years ago
[VLM][Bugfix] Make sure that `multi_modal_kwargs` is broadcasted properly (#5880)
xwjiang2010 committed over 2 years ago
[Doc] Add note about context length in Phi-3-Vision example (#5887)
Cyrus Leung committed over 2 years ago
[BugFix] Fix cuda graph for MLPSpeculator (#5875)
Nick Hill committed over 2 years ago
[Misc] Add example for LLaVA-NeXT (#5879)
Roger Wang committed over 2 years ago
[doc] update usage of env var to avoid conflict (#5873)
youkaichao committed over 2 years ago
Support CPU inference with VSX PowerPC ISA (#5652)
Chip Kerchner committed over 2 years ago
[Bugfix][TPU] Fix CPU cache allocation (#5869)
Woosuk Kwon committed over 2 years ago
[Hardware][TPU] Support parallel sampling & Swapping (#5855)
Woosuk Kwon committed over 2 years ago
[Frontend] Add tokenize/detokenize endpoints (#5054)
sasha0552 committed over 2 years ago
[Kernel] Adding bias epilogue support for `cutlass_scaled_mm` (#5560)
Luka Govedič committed over 2 years ago
[CI/Build] Refactor image test assets (#5821)
Cyrus Leung committed over 2 years ago