A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 01:45 UTC
main
[Misc]Add customized information for models (#4132)
Jee Li committed over 2 years ago
Allow user to define whitespace pattern for outlines (#4305)
Robert Caulk committed over 2 years ago
[Misc] fix typo in block manager (#4453)
Pastel! committed over 2 years ago
[Misc][Typo] type annotation fix (#4495)
harrywu committed over 2 years ago
[Core] Centralize GPU Worker construction (#4419)
Nick Hill committed over 2 years ago
Unable to find Punica extension issue during source code installation (#4494)
fuchen.ljl committed over 2 years ago
fix_tokenizer_snapshot_download_bug (#4493)
fuchen.ljl committed over 2 years ago
[Bugfix][Minor] Make ignore_eos effective (#4468)
Li, Jiang committed over 2 years ago
[Frontend] [Core] Tensorizer: support dynamic `num_readers`, update version (#4467)
Alpay Ariyak committed over 2 years ago
[Frontend] Support complex message content for chat completions endpoint (#3467)
Florian Greinacher committed over 2 years ago
[Kernel] Support Fp8 Checkpoints (Dynamic + Static) (#4332)
Robert Shaw committed over 2 years ago
[Doc] add visualization for multi-stage dockerfile (#4456)
Prashant Gupta committed over 2 years ago
[BugFix] fix num_lookahead_slots missing in async executor (#4165)
leiwen83 committed over 2 years ago
[Core]Refactor gptq_marlin ops (#4466)
Kunshang Ji committed over 2 years ago
[Bugfix][Kernel] Fix compute_type for MoE kernel (#4463)
Woosuk Kwon committed over 2 years ago
[Misc] Upgrade to `torch==2.3.0` (#4454)
Michael Goin committed over 2 years ago
[Core][Distributed] use cpu group to broadcast metadata in cpu (#4444)
youkaichao committed over 2 years ago
[CI] hotfix: soft fail neuron test (#4458)
Simon Mo committed over 2 years ago
[Kernel] Marlin Expansion: Support AutoGPTQ Models with Marlin (#3922)
Robert Shaw committed over 2 years ago
[mypy][5/N] Support all typing on model executor (#4427)
SangBin Cho committed over 2 years ago