A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 03:00 UTC
main
[Misc] Minor type annotation fix (#3716)
Woosuk Kwon committed over 2 years ago
[Test] Make model tests run again and remove --forked from pytest (#3631)
SangBin Cho committed over 2 years ago
Revert "bump version to v0.4.0" (#3708)
youkaichao committed over 2 years ago
bump version to v0.4.0 (#3705)
Simon Mo committed over 2 years ago
[Benchmark] Change mii to use persistent deployment and support tensor parallel (#3628)
Yile (Michael) Gu committed over 2 years ago
[Core] fix del of communicator (#3702)
youkaichao committed over 2 years ago
fix logging msg for block manager (#3701)
Simon Mo committed over 2 years ago
[Bugfix] Set enable_prefix_caching=True in prefix caching example (#3703)
Woosuk Kwon committed over 2 years ago
[Kernel] Add MoE Triton kernel configs for A100 40GB (#3700)
Woosuk Kwon committed over 2 years ago
[Core] Support multi-node inference(eager and cuda graph) (#3686)
Roy committed over 2 years ago
[CI] Add test case to run examples scripts (#3638)
Simon Mo committed over 2 years ago
fix benchmark format reporting in buildkite (#3693)
Simon Mo committed over 2 years ago
[Bugfix] Update neuron_executor.py to add optional vision_language_config (#3695)
Adam Boeglin committed over 2 years ago
[2/N] Chunked prefill data update (#3538)
SangBin Cho committed over 2 years ago
[Kernel] DBRX Triton MoE kernel H100 (#3692)
Roger Wang committed over 2 years ago
[Model] Add support for Qwen2MoeModel (#3346)
wenyujin333 committed over 2 years ago
[Core][Bugfix]Refactor block manager for better testability (#3492)
Cade Daniel committed over 2 years ago
[Kernel] Add Triton MoE kernel configs for DBRX on A100 (#3679)
Woosuk Kwon committed over 2 years ago
[CI/Build] update default number of jobs and nvcc threads to avoid overloading the system (#3675)
youkaichao committed over 2 years ago
[Model] Add support for xverse (#3610)
hxer7963 committed over 2 years ago