@cryptotaxi247 / SquadScope
Code Commits Issues Pulls Sessions CI/CD
main
main
md 164 lines 5.6 KB
Rendered Raw
1 # KEDA External Scaler for GitHub Issue-Driven Agent Autoscaling
2
3 > Scale agent pods to zero when idle, up when work arrives — driven by GitHub Issues.
4
5 ## Overview
6
7 When running Squad on Kubernetes, agent pods sit idle when no work exists. [KEDA](https://keda.sh) (Kubernetes Event-Driven Autoscaler) solves this for queue-based workloads, but GitHub Issues isn't a native KEDA trigger.
8
9 The `keda-copilot-scaler` is a KEDA External Scaler (gRPC) that bridges this gap:
10 1. Polls GitHub API for issues matching specific labels (e.g., `squad:copilot`)
11 2. Reports queue depth as a KEDA metric
12 3. Handles rate limits gracefully (Retry-After, exponential backoff)
13 4. Supports composite scaling decisions
14
15 ## Quick Start
16
17 ### Prerequisites
18 - Kubernetes cluster with KEDA v2.x installed
19 - GitHub personal access token (PAT) with `repo` scope
20 - Helm 3.x
21
22 ### 1. Install the Scaler
23
24 ```bash
25 helm install keda-copilot-scaler oci://ghcr.io/tamirdresher/keda-copilot-scaler \
26 --namespace squad-scaler --create-namespace \
27 --set github.owner=YOUR_ORG \
28 --set github.repo=YOUR_REPO \
29 --set github.token=YOUR_TOKEN
30 ```
31
32 Or with Kustomize:
33 ```bash
34 kubectl apply -k https://github.com/tamirdresher/keda-copilot-scaler/deploy/kustomize
35 ```
36
37 ### 2. Create a ScaledObject
38
39 ```yaml
40 apiVersion: keda.sh/v1alpha1
41 kind: ScaledObject
42 metadata:
43 name: picard-scaler
44 namespace: squad
45 spec:
46 scaleTargetRef:
47 name: picard-deployment
48 minReplicaCount: 0 # Scale to zero when idle
49 maxReplicaCount: 3
50 pollingInterval: 30 # Check every 30 seconds
51 cooldownPeriod: 300 # Wait 5 minutes before scaling down
52 triggers:
53 - type: external
54 metadata:
55 scalerAddress: keda-copilot-scaler.squad-scaler.svc.cluster.local:6000
56 owner: your-org
57 repo: your-repo
58 labels: squad:copilot # Only count issues with this label
59 threshold: "1" # Scale up when >= 1 issue exists
60 ```
61
62 ### 3. Verify
63
64 ```bash
65 # Check the scaler is running
66 kubectl get pods -n squad-scaler
67
68 # Check ScaledObject status
69 kubectl get scaledobject picard-scaler -n squad
70
71 # Watch scaling events
72 kubectl get events -n squad --watch
73 ```
74
75 ## Scaling Behavior
76
77 | Open Issues | Target Replicas | Behavior |
78 |------------|----------------|----------|
79 | 0 | 0 | Scale to zero — save resources |
80 | 1–3 | 1 | Single agent handles work |
81 | 4–10 | 2 | Scale up for parallel processing |
82 | 10+ | 3 (max) | Maximum parallelism |
83
84 The threshold and max replicas are configurable per ScaledObject.
85
86 ## Rate Limit Awareness
87
88 The scaler tracks GitHub API rate limits:
89 - Reads `X-RateLimit-Remaining` from API responses
90 - Backs off when quota is low (< 100 remaining)
91 - Reports rate limit metrics as secondary KEDA triggers
92 - Never exhausts API quota from polling
93
94 ## Integration with Squad
95
96 ### Machine Capabilities (#514)
97
98 Combine with machine capability labels for intelligent scheduling:
99
100 ```yaml
101 # Only scale pods on GPU-capable nodes
102 spec:
103 template:
104 spec:
105 nodeSelector:
106 node.squad.dev/gpu: "true"
107 triggers:
108 - type: external
109 metadata:
110 labels: squad:copilot,needs:gpu
111 ```
112
113 ### Cooperative Rate Limiting (#515)
114
115 The scaler exposes rate limit metrics that feed into the cooperative rate limiting system:
116 - Current `X-RateLimit-Remaining` value
117 - Predicted time to exhaustion (from predictive circuit breaker)
118 - Can return 0 target replicas when rate limited → pods scale to zero
119
120 ## Architecture
121
122 ```
123 GitHub API KEDA Kubernetes
124 ┌──────────┐ ┌──────────┐ ┌──────────────┐
125 │ Issues │◄── poll ──►│ Scaler │──metrics─►│ HPA / KEDA │
126 │ (REST) │ │ (gRPC) │ │ Controller │
127 └──────────┘ └──────────┘ └──────┬───────┘
128 │
129 scale up/down
130 │
131 ┌──────▼───────┐
132 │ Agent Pods │
133 │ (0–N replicas)│
134 └──────────────┘
135 ```
136
137 ## Configuration Reference
138
139 | Parameter | Default | Description |
140 |-----------|---------|-------------|
141 | `github.owner` | — | Repository owner |
142 | `github.repo` | — | Repository name |
143 | `github.token` | — | GitHub PAT with `repo` scope |
144 | `github.labels` | `squad:copilot` | Comma-separated label filter |
145 | `scaler.port` | `6000` | gRPC server port |
146 | `scaler.pollInterval` | `30s` | GitHub API polling interval |
147 | `scaler.rateLimitThreshold` | `100` | Stop polling below this remaining |
148
149 ## Source & Contributing
150
151 - **Repository:** [tamirdresher/keda-copilot-scaler](https://github.com/tamirdresher/keda-copilot-scaler)
152 - **License:** MIT
153 - **Language:** Go
154 - **Tests:** 51 passing (unit + integration)
155 - **CI:** GitHub Actions
156
157 The scaler is maintained as a standalone project. PRs and issues welcome.
158
159 ## References
160
161 - [KEDA External Scalers](https://keda.sh/docs/latest/concepts/external-scalers/) — KEDA documentation
162 - [Squad on AKS](https://github.com/tamirdresher/squad-on-aks) — Full Kubernetes deployment example
163 - [Machine Capabilities](machine-capabilities.md) — Capability-based routing (#514)
164 - [Cooperative Rate Limiting](cooperative-rate-limiting.md) — Multi-agent rate management (#515)