A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 02:00 UTC
main
[Model] Add cogagent model support vLLM (#11742)
sixgod committed almost 2 years ago
[Bugfix] Fix RobertaModel loading (#11940)
Nicolò Lucchesi committed almost 2 years ago
[Doc] Basic guide for writing unit tests for new models (#11951)
Cyrus Leung committed almost 2 years ago
[CI/Build] Move model-specific multi-modal processing tests (#11934)
Cyrus Leung committed almost 2 years ago
[Bugfix] fused_experts_impl wrong compute type for float32 (#11921)
shaochangxu committed almost 2 years ago
[Bugfix][SpecDecode] Adjust Eagle model architecture to align with intended design (#11672)
Sungjae Lee committed almost 2 years ago
[ci] fix broken distributed-tests-4-gpus (#11937)
youkaichao committed almost 2 years ago
[mypy] Fix mypy warnings in api_server.py (#11941)
Fred Reiss committed almost 2 years ago
[Bugfix] Check that number of images matches number of <|image|> tokens with mllama (#11939)
Travis Johnson committed almost 2 years ago
[Misc] Update benchmark_prefix_caching.py fixed example usage (#11920)
minmin committed almost 2 years ago
[Misc] Clean up debug code in Deepseek-V3 (#11930)
Isotr0py committed almost 2 years ago
[Hardware][CPU] Support MOE models on x86 CPU (#11831)
Li, Jiang committed almost 2 years ago
Doc fix in `benchmark_long_document_qa_throughput.py` (#11933)
Kuntai Du committed almost 2 years ago
[Docs] Fix docstring in `get_ip` function (#11932)
Kuntai Du committed almost 2 years ago
[Doc] Rename offline inference examples (#11927)
Harry Mellor committed almost 2 years ago
[platform] support custom torch.compile backend key (#11318)
wangxiyuan committed almost 2 years ago
[Doc] [1/N] Initial guide for merged multi-modal processor (#11925)
Cyrus Leung committed almost 2 years ago
[ci] Fix sampler tests (#11922)
youkaichao committed almost 2 years ago
Replace "online inference" with "online serving" (#11923)
Harry Mellor committed almost 2 years ago
[platform] support pytorch custom op pluggable (#11328)
wangxiyuan committed almost 2 years ago