A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Model] Add pp support for hunyuan (#24212)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Jiangyun Zhu committed
Sep 4, 2025 at 18:58 UTC
eafa8dcde63d625350ed618db4dd1cbcbaae77a1
2 files changed
+4
-4
docs/models/supported_models.md
+2
-2
@@ -365,8 +365,8 @@ th {
365
| `GraniteMoeSharedForCausalLM` | Granite MoE Shared | `ibm-research/moe-7b-1b-active-shared-experts` (test model) | ✅︎ | ✅︎ | ✅︎ |
366
| `GritLM` | GritLM | `parasail-ai/GritLM-7B-vllm`. | ✅︎ | ✅︎ | ✅︎ |
367
| `Grok1ModelForCausalLM` | Grok1 | `hpcai-tech/grok-1`. | ✅︎ | ✅︎ | ✅︎ |
368
-| `HunYuanDenseV1ForCausalLM` | Hunyuan-7B-Instruct-0124 | `tencent/Hunyuan-7B-Instruct-0124` | ✅︎ | | ✅︎ |
369
-| `HunYuanMoEV1ForCausalLM` | Hunyuan-80B-A13B | `tencent/Hunyuan-A13B-Instruct`, `tencent/Hunyuan-A13B-Pretrain`, `tencent/Hunyuan-A13B-Instruct-FP8`, etc. | ✅︎ | | ✅︎ |
368
+| `HunYuanDenseV1ForCausalLM` | Hunyuan-7B-Instruct-0124 | `tencent/Hunyuan-7B-Instruct-0124` | ✅︎ | ✅︎ | ✅︎ |
369
+| `HunYuanMoEV1ForCausalLM` | Hunyuan-80B-A13B | `tencent/Hunyuan-A13B-Instruct`, `tencent/Hunyuan-A13B-Pretrain`, `tencent/Hunyuan-A13B-Instruct-FP8`, etc. | ✅︎ | ✅︎ | ✅︎ |
370
| `HCXVisionForCausalLM` | HyperCLOVAX-SEED-Vision-Instruct-3B | `naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B` | | | ✅︎ |
371
| `InternLMForCausalLM` | InternLM | `internlm/internlm-7b`, `internlm/internlm-chat-7b`, etc. | ✅︎ | ✅︎ | ✅︎ |
372
| `InternLM2ForCausalLM` | InternLM2 | `internlm/internlm2-7b`, `internlm/internlm2-chat-7b`, etc. | ✅︎ | ✅︎ | ✅︎ |
vllm/model_executor/models/hunyuan_v1.py
+2
-2
@@ -56,7 +56,7 @@ from vllm.model_executor.model_loader.weight_utils import (
56
from vllm.model_executor.sampling_metadata import SamplingMetadata
57
from vllm.sequence import IntermediateTensors
58
59
-from .interfaces import SupportsLoRA
59
+from .interfaces import SupportsLoRA, SupportsPP
60
from .utils import (AutoWeightsLoader, PPMissingLayer, is_pp_missing_parameter,
61
make_layers)
62
@@ -841,7 +841,7 @@ class HunYuanModel(nn.Module):
841
return loaded_params
842
843
844
-class HunYuanV1Base(nn.Module, SupportsLoRA):
844
+class HunYuanV1Base(nn.Module, SupportsLoRA, SupportsPP):
845
packed_modules_mapping = {
846
"qkv_proj": [
847
"q_proj",