A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 12, 09:45 UTC
main
[CPU] Update torch 2.9.1 for CPU backend (#29664)
Li, Jiang committed 12 months ago
Revert "Supress verbose logs from model_hosting_container_standards (… (#29335)
HappyAmazonian committed 12 months ago
[Optimization] Early return for `_apply_matches` and `_iter_placeholders` (#29668)
Cyrus Leung committed 12 months ago
[BugFix] Fix spec decoding max_tokens scheduling perf issue (#29542)
Nick Hill committed 12 months ago
[mypy] Pass type checking for `vllm/utils` and `vllm/v1/pool` (#29666)
Cyrus Leung committed 12 months ago
[Misc] Remove redundant attention var constants (#29650)
Cyrus Leung committed 12 months ago
[Docs] Add SPLADE and Ultravox models to supported models documentation (#29659)
Wilson Wu committed 12 months ago
[Rocm] Set VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS default is disabled (#28985)
杰兮 committed 12 months ago
[Doc] Reorganize benchmark docs (#29658)
Cyrus Leung committed 12 months ago
Fix parameter order in GPT-OSS weight loading function for non-MXFP4 weights (#29506)
Filipp Fisin committed 12 months ago
[BUGFIX] MistralTokenizer._call__ adds an invalid EOS token (#29607)
Julien Denize committed 12 months ago
[Doc] Improve abnormal information string (#29655)
maang-h committed 12 months ago
[Feature][Bench] Add pareto visualization (#29477)
rongfu.leng committed 12 months ago
Revert "[CPU]Update CPU PyTorch to 2.9.0 (#29589)" (#29647)
Cyrus Leung committed 12 months ago
Improve enable chunked_prefill & prefix_caching logic. (#26623)
wang.yuqi committed 12 months ago
[Multimodal][Speculative Decoding]Eagle3 mm support, enablement on qwen3vl (#29594)
EanWang211123 committed 12 months ago
[BugFix] Fix ValueError in NewRequestData repr methods (#29392)
maang-h committed 12 months ago
[Docs] Update supported models for Olmo 3 in tool calling documentation (#29411)
Wilson Wu committed 12 months ago
[LoRA] Support FusedMoE LoRA Triton kernel for mxfp4 (#28971)
Xin Yang committed 12 months ago
[CPU]Update CPU PyTorch to 2.9.0 (#29589)
scydas committed 12 months ago