A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 04:45 UTC
[Docs] Improve frameworks/helm.md (#20113)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
Michael Yao committed
Jun 26, 2025 at 18:41 UTC
84c260caeb88d25840ec0653c0b978a46eae6a84
1 file changed
+64
-56
docs/deployment/frameworks/helm.md
+64
-56
@@ -5,9 +5,9 @@ title: Helm
5
6
A Helm chart to deploy vLLM for Kubernetes
7
8
-Helm is a package manager for Kubernetes. It will help you to deploy vLLM on k8s and automate the deployment of vLLM Kubernetes applications. With Helm, you can deploy the same framework architecture with different configurations to multiple namespaces by overriding variable values.
8
+Helm is a package manager for Kubernetes. It helps automate the deployment of vLLM applications on Kubernetes. With Helm, you can deploy the same framework architecture with different configurations to multiple namespaces by overriding variable values.
9
10
-This guide will walk you through the process of deploying vLLM with Helm, including the necessary prerequisites, steps for helm installation and documentation on architecture and values file.
10
+This guide will walk you through the process of deploying vLLM with Helm, including the necessary prerequisites, steps for Helm installation and documentation on architecture and values file.
11
12
## Prerequisites
13
@@ -16,17 +16,23 @@ Before you begin, ensure that you have the following:
16
- A running Kubernetes cluster
17
- NVIDIA Kubernetes Device Plugin (`k8s-device-plugin`): This can be found at [https://github.com/NVIDIA/k8s-device-plugin](https://github.com/NVIDIA/k8s-device-plugin)
18
- Available GPU resources in your cluster
19
-- S3 with the model which will be deployed
19
+- An S3 with the model which will be deployed
20
21
## Installing the chart
22
23
To install the chart with the release name `test-vllm`:
24
25
```bash
26
-helm upgrade --install --create-namespace --namespace=ns-vllm test-vllm . -f values.yaml --set secrets.s3endpoint=$ACCESS_POINT --set secrets.s3bucketname=$BUCKET --set secrets.s3accesskeyid=$ACCESS_KEY --set secrets.s3accesskey=$SECRET_KEY
26
+helm upgrade --install --create-namespace \
27
+ --namespace=ns-vllm test-vllm . \
28
+ -f values.yaml \
29
+ --set secrets.s3endpoint=$ACCESS_POINT \
30
+ --set secrets.s3bucketname=$BUCKET \
31
+ --set secrets.s3accesskeyid=$ACCESS_KEY \
32
+ --set secrets.s3accesskey=$SECRET_KEY
33
```
34
29
-## Uninstalling the Chart
35
+## Uninstalling the chart
36
37
To uninstall the `test-vllm` deployment:
38
@@ -39,57 +45,59 @@ chart **including persistent volumes** and deletes the release.
45
46
## Architecture
47
42
-
48
+
49
50
## Values
51
46
-| Key | Type | Default | Description |
47
-|--------------------------------------------|---------|----------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------|
48
-| autoscaling | object | {"enabled":false,"maxReplicas":100,"minReplicas":1,"targetCPUUtilizationPercentage":80} | Autoscaling configuration |
49
-| autoscaling.enabled | bool | false | Enable autoscaling |
50
-| autoscaling.maxReplicas | int | 100 | Maximum replicas |
51
-| autoscaling.minReplicas | int | 1 | Minimum replicas |
52
-| autoscaling.targetCPUUtilizationPercentage | int | 80 | Target CPU utilization for autoscaling |
53
-| configs | object | {} | Configmap |
54
-| containerPort | int | 8000 | Container port |
55
-| customObjects | list | [] | Custom Objects configuration |
56
-| deploymentStrategy | object | {} | Deployment strategy configuration |
57
-| externalConfigs | list | [] | External configuration |
58
-| extraContainers | list | [] | Additional containers configuration |
59
-| extraInit | object | {"pvcStorage":"1Gi","s3modelpath":"relative_s3_model_path/opt-125m", "awsEc2MetadataDisabled": true} | Additional configuration for the init container |
60
-| extraInit.pvcStorage | string | "50Gi" | Storage size of the s3 |
61
-| extraInit.s3modelpath | string | "relative_s3_model_path/opt-125m" | Path of the model on the s3 which hosts model weights and config files |
62
-| extraInit.awsEc2MetadataDisabled | boolean | true | Disables the use of the Amazon EC2 instance metadata service |
63
-| extraPorts | list | [] | Additional ports configuration |
64
-| gpuModels | list | ["TYPE_GPU_USED"] | Type of gpu used |
65
-| image | object | {"command":["vllm","serve","/data/","--served-model-name","opt-125m","--host","0.0.0.0","--port","8000"],"repository":"vllm/vllm-openai","tag":"latest"} | Image configuration |
66
-| image.command | list | ["vllm","serve","/data/","--served-model-name","opt-125m","--host","0.0.0.0","--port","8000"] | Container launch command |
67
-| image.repository | string | "vllm/vllm-openai" | Image repository |
68
-| image.tag | string | "latest" | Image tag |
69
-| livenessProbe | object | {"failureThreshold":3,"httpGet":{"path":"/health","port":8000},"initialDelaySeconds":15,"periodSeconds":10} | Liveness probe configuration |
70
-| livenessProbe.failureThreshold | int | 3 | Number of times after which if a probe fails in a row, Kubernetes considers that the overall check has failed: the container is not alive |
71
-| livenessProbe.httpGet | object | {"path":"/health","port":8000} | Configuration of the Kubelet http request on the server |
72
-| livenessProbe.httpGet.path | string | "/health" | Path to access on the HTTP server |
73
-| livenessProbe.httpGet.port | int | 8000 | Name or number of the port to access on the container, on which the server is listening |
74
-| livenessProbe.initialDelaySeconds | int | 15 | Number of seconds after the container has started before liveness probe is initiated |
75
-| livenessProbe.periodSeconds | int | 10 | How often (in seconds) to perform the liveness probe |
76
-| maxUnavailablePodDisruptionBudget | string | "" | Disruption Budget Configuration |
77
-| readinessProbe | object | {"failureThreshold":3,"httpGet":{"path":"/health","port":8000},"initialDelaySeconds":5,"periodSeconds":5} | Readiness probe configuration |
78
-| readinessProbe.failureThreshold | int | 3 | Number of times after which if a probe fails in a row, Kubernetes considers that the overall check has failed: the container is not ready |
79
-| readinessProbe.httpGet | object | {"path":"/health","port":8000} | Configuration of the Kubelet http request on the server |
80
-| readinessProbe.httpGet.path | string | "/health" | Path to access on the HTTP server |
81
-| readinessProbe.httpGet.port | int | 8000 | Name or number of the port to access on the container, on which the server is listening |
82
-| readinessProbe.initialDelaySeconds | int | 5 | Number of seconds after the container has started before readiness probe is initiated |
83
-| readinessProbe.periodSeconds | int | 5 | How often (in seconds) to perform the readiness probe |
84
-| replicaCount | int | 1 | Number of replicas |
85
-| resources | object | {"limits":{"cpu":4,"memory":"16Gi","nvidia.com/gpu":1},"requests":{"cpu":4,"memory":"16Gi","nvidia.com/gpu":1}} | Resource configuration |
86
-| resources.limits."nvidia.com/gpu" | int | 1 | Number of gpus used |
87
-| resources.limits.cpu | int | 4 | Number of CPUs |
88
-| resources.limits.memory | string | "16Gi" | CPU memory configuration |
89
-| resources.requests."nvidia.com/gpu" | int | 1 | Number of gpus used |
90
-| resources.requests.cpu | int | 4 | Number of CPUs |
91
-| resources.requests.memory | string | "16Gi" | CPU memory configuration |
92
-| secrets | object | {} | Secrets configuration |
93
-| serviceName | string | Service name | |
94
-| servicePort | int | 80 | Service port |
95
-| labels.environment | string | test | Environment name |
52
+The following table describes configurable parameters of the chart in `values.yaml`:
53
+
54
+| Key | Type | Default | Description |
55
+|-----|------|---------|-------------|
56
+| autoscaling | object | {"enabled":false,"maxReplicas":100,"minReplicas":1,"targetCPUUtilizationPercentage":80} | Autoscaling configuration |
57
+| autoscaling.enabled | bool | false | Enable autoscaling |
58
+| autoscaling.maxReplicas | int | 100 | Maximum replicas |
59
+| autoscaling.minReplicas | int | 1 | Minimum replicas |
60
+| autoscaling.targetCPUUtilizationPercentage | int | 80 | Target CPU utilization for autoscaling |
61
+| configs | object | {} | Configmap |
62
+| containerPort | int | 8000 | Container port |
63
+| customObjects | list | [] | Custom Objects configuration |
64
+| deploymentStrategy | object | {} | Deployment strategy configuration |
65
+| externalConfigs | list | [] | External configuration |
66
+| extraContainers | list | [] | Additional containers configuration |
67
+| extraInit | object | {"pvcStorage":"1Gi","s3modelpath":"relative_s3_model_path/opt-125m", "awsEc2MetadataDisabled": true} | Additional configuration for the init container |
68
+| extraInit.pvcStorage | string | "1Gi" | Storage size of the s3 |
69
+| extraInit.s3modelpath | string | "relative_s3_model_path/opt-125m" | Path of the model on the s3 which hosts model weights and config files |
70
+| extraInit.awsEc2MetadataDisabled | boolean | true | Disables the use of the Amazon EC2 instance metadata service |
71
+| extraPorts | list | [] | Additional ports configuration |
72
+| gpuModels | list | ["TYPE_GPU_USED"] | Type of gpu used |
73
+| image | object | {"command":["vllm","serve","/data/","--served-model-name","opt-125m","--host","0.0.0.0","--port","8000"],"repository":"vllm/vllm-openai","tag":"latest"} | Image configuration |
74
+| image.command | list | ["vllm","serve","/data/","--served-model-name","opt-125m","--host","0.0.0.0","--port","8000"] | Container launch command |
75
+| image.repository | string | "vllm/vllm-openai" | Image repository |
76
+| image.tag | string | "latest" | Image tag |
77
+| livenessProbe | object | {"failureThreshold":3,"httpGet":{"path":"/health","port":8000},"initialDelaySeconds":15,"periodSeconds":10} | Liveness probe configuration |
78
+| livenessProbe.failureThreshold | int | 3 | Number of times after which if a probe fails in a row, Kubernetes considers that the overall check has failed: the container is not alive |
79
+| livenessProbe.httpGet | object | {"path":"/health","port":8000} | Configuration of the kubelet http request on the server |
80
+| livenessProbe.httpGet.path | string | "/health" | Path to access on the HTTP server |
81
+| livenessProbe.httpGet.port | int | 8000 | Name or number of the port to access on the container, on which the server is listening |
82
+| livenessProbe.initialDelaySeconds | int | 15 | Number of seconds after the container has started before liveness probe is initiated |
83
+| livenessProbe.periodSeconds | int | 10 | How often (in seconds) to perform the liveness probe |
84
+| maxUnavailablePodDisruptionBudget | string | "" | Disruption Budget Configuration |
85
+| readinessProbe | object | {"failureThreshold":3,"httpGet":{"path":"/health","port":8000},"initialDelaySeconds":5,"periodSeconds":5} | Readiness probe configuration |
86
+| readinessProbe.failureThreshold | int | 3 | Number of times after which if a probe fails in a row, Kubernetes considers that the overall check has failed: the container is not ready |
87
+| readinessProbe.httpGet | object | {"path":"/health","port":8000} | Configuration of the kubelet http request on the server |
88
+| readinessProbe.httpGet.path | string | "/health" | Path to access on the HTTP server |
89
+| readinessProbe.httpGet.port | int | 8000 | Name or number of the port to access on the container, on which the server is listening |
90
+| readinessProbe.initialDelaySeconds | int | 5 | Number of seconds after the container has started before readiness probe is initiated |
91
+| readinessProbe.periodSeconds | int | 5 | How often (in seconds) to perform the readiness probe |
92
+| replicaCount | int | 1 | Number of replicas |
93
+| resources | object | {"limits":{"cpu":4,"memory":"16Gi","nvidia.com/gpu":1},"requests":{"cpu":4,"memory":"16Gi","nvidia.com/gpu":1}} | Resource configuration |
94
+| resources.limits."nvidia.com/gpu" | int | 1 | Number of GPUs used |
95
+| resources.limits.cpu | int | 4 | Number of CPUs |
96
+| resources.limits.memory | string | "16Gi" | CPU memory configuration |
97
+| resources.requests."nvidia.com/gpu" | int | 1 | Number of GPUs used |
98
+| resources.requests.cpu | int | 4 | Number of CPUs |
99
+| resources.requests.memory | string | "16Gi" | CPU memory configuration |
100
+| secrets | object | {} | Secrets configuration |
101
+| serviceName | string | "" | Service name |
102
+| servicePort | int | 80 | Service port |
103
+| labels.environment | string | test | Environment name |