The best way to control Claude Code, Codex and more from any computer or phone

main
mdx 276 lines 9.51 KB
Raw
1 ---
2 title: HyperPod
3 description: Provision SageMaker HyperPod clusters — Slurm or EKS orchestrated — and run ML workloads on them with sbatch, raw manifests, or effectful Jobs with task governance.
4 ---
5
6 **SageMaker HyperPod** is AWS's persistent fleet for ML training
7 and inference: a cluster of accelerated instances with automatic
8 health checks, faulty-node replacement, and deep health checks
9 for GPUs. Where ECS and EKS run application containers, HyperPod
10 runs distributed training — and stays up between jobs.
11
12 The HyperPod primitives:
13
14 - A **Cluster** is the fleet. It is orchestrated by **Slurm**
15 (the default) or by an **EKS** cluster you attach it to.
16 - An **instance group** names a set of identical instances — a
17 controller group, a worker group — each with an instance type,
18 count, execution role, and lifecycle script.
19 - A **lifecycle script** (`on_create.sh` in S3) runs on every
20 node when it boots. This is where real clusters install
21 schedulers, mount FSx, and wire observability.
22 - **Task governance** (EKS orchestration only) arbitrates the
23 fleet between teams: a scheduler policy sets priority classes,
24 and per-team compute quotas reserve capacity.
25
26 Alchemy models these with
27 [`Cluster`](/providers/aws/sagemaker/cluster),
28 [`ClusterSchedulerConfig`](/providers/aws/sagemaker/clusterschedulerconfig),
29 and [`ComputeQuota`](/providers/aws/sagemaker/computequota), and
30 Kubernetes workloads opt onto HyperPod nodes by referencing their
31 attributes (the group's `nodeSelector`, the quota's governed
32 `namespace` and `queueName`) from plain
33 [`Kubernetes.Deployment`](/providers/kubernetes/deployment) and
34 [`Kubernetes.Job`](/providers/kubernetes/job) props.
35
36 ## Choose an orchestrator
37
38 | | Slurm (default) | EKS (`orchestrator: { Eks }`) |
39 | --------------- | ------------------------------------- | ---------------------------------------------------- |
40 | Workloads | `sbatch` on the login node (over SSM) | Kubernetes objects — `Deployment`, `Job`, `Manifest` |
41 | High-level DX | — | `Kubernetes.Job` / `Deployment` via HyperPod attributes |
42 | Governance | Slurm accounting | `ClusterSchedulerConfig` + `ComputeQuota` (Kueue) |
43 | Extra plumbing | Lifecycle bucket + script | EKS cluster + the HyperPod dependencies Helm chart |
44
45 Pick Slurm for classic HPC-style training queues. Pick EKS when
46 your platform is Kubernetes or you want typed workloads and task
47 governance.
48
49 ## Create a Slurm cluster
50
51 A Slurm cluster needs three things: an S3 bucket holding the
52 lifecycle script, an execution role the nodes assume, and the
53 cluster itself. The script must exist in S3 before the cluster
54 creates — a deploy-time `Action` uploads it, and the data flow
55 (bucket → script → cluster) orders the deploy:
56
57 ```typescript
58 const bucket = yield* AWS.S3.Bucket("LifecycleScripts", {
59 forceDestroy: true,
60 });
61
62 const script = yield* UploadLifecycleScript({
63 bucketName: bucket.bucketName,
64 });
65
66 const role = yield* AWS.IAM.Role("HyperPodInstanceRole", {
67 assumeRolePolicyDocument: {
68 Version: "2012-10-17",
69 Statement: [
70 {
71 Effect: "Allow",
72 Principal: { Service: "sagemaker.amazonaws.com" },
73 Action: ["sts:AssumeRole"],
74 },
75 ],
76 },
77 managedPolicyArns: [
78 "arn:aws:iam::aws:policy/AmazonSageMakerClusterInstanceRolePolicy",
79 ],
80 });
81
82 const cluster = yield* AWS.SageMaker.Cluster("TrainingCluster", {
83 instanceGroups: {
84 controller: {
85 InstanceType: "ml.t3.medium",
86 InstanceCount: 1,
87 ExecutionRole: role.roleArn,
88 LifeCycleConfig: {
89 SourceS3Uri: script.sourceS3Uri,
90 OnCreate: script.onCreate,
91 },
92 },
93 },
94 });
95 ```
96
97 Instance groups reconcile in place: change a count or add a group
98 and the cluster updates; remove a group from `instanceGroups` and
99 it is deleted from the cluster. Changing the VPC replaces the
100 cluster. Provisioning takes ~5 minutes for small CPU groups and
101 10–25 minutes for large GPU groups.
102
103 The full working stack is
104 [`examples/aws-hyperpod`](https://github.com/alchemy-run/alchemy/tree/main/examples/aws-hyperpod).
105
106 ## Submit Slurm jobs
107
108 Slurm has no remote submission API — jobs are submitted on the
109 cluster. Every node is an SSM target named
110 `sagemaker-cluster:<cluster-id>_<instance-group>-<instance-id>`:
111
112 ```sh
113 aws sagemaker list-cluster-nodes --cluster-name <clusterName>
114 aws ssm start-session \
115 --target sagemaker-cluster:<cluster-id>_controller-<instance-id>
116 # then, on the node:
117 sbatch --nodes=4 train.sbatch
118 ```
119
120 ## Create an EKS-orchestrated cluster
121
122 Under EKS orchestration, HyperPod nodes join an EKS cluster you
123 provide and workloads are ordinary Kubernetes objects. The API
124 enforces a few constraints, all encoded in the example:
125
126 - The EKS cluster must use the **`API`** (or `API_AND_CONFIG_MAP`)
127 authentication mode — EKS's own CONFIG_MAP default is rejected.
128 - HyperPod trails the newest Kubernetes version; pin one it
129 supports (1.28–1.35 today).
130 - `LifeCycleConfig` is required for EKS-orchestrated instance
131 groups too.
132 - The **HyperPod dependencies Helm chart**
133 ([aws/sagemaker-hyperpod-cli](https://github.com/aws/sagemaker-hyperpod-cli))
134 must be installed on the EKS cluster before the HyperPod
135 cluster attaches — SageMaker validates it. The example fetches
136 the chart with an `Action` and applies it with
137 [`Kubernetes.HelmChart`](/providers/kubernetes/helmchart).
138
139 ```typescript
140 const eks = yield* AWS.EKS.Cluster("Orchestrator", {
141 roleArn: eksRole.roleArn,
142 version: "1.34",
143 resourcesVpcConfig: { subnetIds: network.privateSubnetIds },
144 accessConfig: {
145 authenticationMode: "API",
146 bootstrapClusterCreatorAdminPermissions: true,
147 },
148 });
149
150 const hyperpod = yield* AWS.SageMaker.Cluster("HyperPod", {
151 orchestrator: { Eks: { ClusterArn: eks.clusterArn } },
152 vpcConfig: {
153 SecurityGroupIds: [clusterSecurityGroupId],
154 Subnets: network.privateSubnetIds,
155 },
156 instanceGroups: {
157 workers: {
158 InstanceType: "ml.t3.medium",
159 InstanceCount: 1,
160 ExecutionRole: role.roleArn,
161 LifeCycleConfig: {
162 SourceS3Uri: script.sourceS3Uri,
163 OnCreate: script.onCreate,
164 },
165 },
166 },
167 });
168 ```
169
170 HyperPod nodes must live in **private subnets** — the
171 [`Network`](/providers/aws/ec2/network) helper with `nat:
172 "single"` builds a suitable VPC.
173
174 ## Run workloads on HyperPod nodes
175
176 HyperPod nodes are ordinary EKS nodes carrying well-known labels.
177 [`Kubernetes.Deployment`](/providers/kubernetes/deployment) and
178 [`Kubernetes.Job`](/providers/kubernetes/job) opt onto them with
179 each instance group's `nodeSelector` attribute — health-checked
180 nodes of that group. The instance-group keys carry through to the
181 cluster's attributes as types, so the reference is typed per key
182 (a typo'd group name is a compile error) and the workload is
183 connected to the fleet through the resource graph:
184
185 ```typescript
186 const train = yield* Kubernetes.Job(
187 "Train",
188 {
189 cluster: eks,
190 main: import.meta.url,
191 podTemplate: {
192 spec: {
193 nodeSelector: hyperpod.instanceGroups.workers.nodeSelector,
194 },
195 },
196 },
197 Effect.gen(function* () {
198 return {
199 run: Effect.gen(function* () {
200 // training / eval logic; bindings land IAM on pod identity
201 }),
202 };
203 }),
204 );
205 ```
206
207 For anything the typed surface doesn't cover — a Kubeflow
208 `PyTorchJob`, a custom operator — apply a raw
209 [`Kubernetes.Manifest`](/providers/kubernetes/manifest) pinned by
210 the node labels directly:
211
212 ```typescript
213 nodeSelector: {
214 "sagemaker.amazonaws.com/node-health-status": "Schedulable",
215 "sagemaker.amazonaws.com/instance-group-name": "workers",
216 },
217 ```
218
219 ## Task governance
220
221 Task governance arbitrates the fleet between teams. It ships as
222 the `amazon-sagemaker-hyperpod-taskgovernance` EKS add-on (Kueue
223 under the hood), a **scheduler policy** with priority classes,
224 and per-team **compute quotas**:
225
226 ```typescript
227 yield* AWS.EKS.Addon("TaskGovernance", {
228 clusterName: eks.clusterName,
229 addonName: "amazon-sagemaker-hyperpod-taskgovernance",
230 });
231
232 // One policy per cluster — a second create fails with the typed
233 // ClusterSchedulerConfigAlreadyExists error.
234 const policy = yield* AWS.SageMaker.ClusterSchedulerConfig("Scheduler", {
235 clusterArn: hyperpod.clusterArn,
236 schedulerConfig: {
237 PriorityClasses: [
238 { Name: "inference", Weight: 100 },
239 { Name: "training", Weight: 75 },
240 ],
241 FairShare: "Enabled",
242 },
243 });
244
245 // Creates the hyperpod-ns-research namespace + its Kueue queue.
246 const quota = yield* AWS.SageMaker.ComputeQuota("ResearchQuota", {
247 clusterArn: hyperpod.clusterArn,
248 computeQuotaTarget: { TeamName: "research", FairShareWeight: 10 },
249 computeQuotaConfig: {
250 ComputeQuotaResources: [{ InstanceType: "ml.t3.medium", Count: 1 }],
251 },
252 });
253 ```
254
255 A workload submits through governance by referencing the quota's
256 attributes — the governed namespace and the team's Kueue queue —
257 which also orders the workload after the quota:
258
259 ```typescript
260 namespace: quota.namespace, // hyperpod-ns-research
261 labels: {
262 [AWS.SageMaker.KUEUE_QUEUE_NAME_LABEL]: quota.queueName,
263 [AWS.SageMaker.KUEUE_PRIORITY_CLASS_LABEL]: "training-priority",
264 },
265 ```
266
267 ## Where next
268
269 - [`examples/aws-hyperpod`](https://github.com/alchemy-run/alchemy/tree/main/examples/aws-hyperpod)
270 — both orchestrators, every workload tier, task governance
271 - [EKS](/aws/compute/eks) — the Kubernetes surface HyperPod
272 workloads ride on
273 - [`Cluster`](/providers/aws/sagemaker/cluster),
274 [`ClusterSchedulerConfig`](/providers/aws/sagemaker/clusterschedulerconfig),
275 [`ComputeQuota`](/providers/aws/sagemaker/computequota) — API
276 reference