main
md 520 lines 18.8 KB
Rendered Raw
1 ---
2 title: Deployment
3 description: Production deployment guide for Portal relay servers.
4 priority: P1
5 ---
6
7 <div class="not-prose mb-8 rounded-lg border border-blue-200 bg-blue-50 px-4 py-3 text-sm text-blue-800 dark:border-blue-800 dark:bg-blue-950/30 dark:text-blue-300">
8 <strong>Advanced Documentation</strong> - This page covers production relay deployment for operators.
9 </div>
10
11 # Portal Relay Deployment Guide
12
13 Portal supports two deployment profiles:
14
15 - API-only relay: one `portal` image exposes relay API paths and tunnel ingress
16 directly. This is documented in [Self-Hosting](/self-hosting).
17 - Full Portal edge: `nginx`, `portal`, `portal-frontend`, and `portal-api`
18 provide one browser-facing HTTPS origin with dashboard, presentation API, and
19 wildcard tunnel routing.
20
21 This guide covers the full Portal edge profile and is the source of truth for
22 how the split relay, frontend, and presentation API are expected to be deployed
23 together.
24
25 ## 1. Production Topology
26
27 The production deployment has four roles:
28
29 | Role | Service or image | Publicly exposed | Owns |
30 |---|---|---|---|
31 | Public edge | `nginx` | yes, `443/tcp` | Public TLS termination, path routing, wildcard SNI passthrough |
32 | Relay | `portal`, `ghcr.io/gosuda/portal` | no direct public API port | Relay API, wallet auth, policy enforcement, tunnel ingress |
33 | Static frontend | `portal-frontend`, `ghcr.io/gosuda/portal-frontend` | no direct public port | SPA assets |
34 | Presentation API | `portal-api`, `ghcr.io/gosuda/portal-api` | no direct public port | Frontend-owned state, policy composition, service status, thumbnails |
35
36 Traffic should flow through one public HTTPS origin:
37
38 ```text
39 Browser
40 -> https://portal.example.com
41 -> nginx public TLS edge
42 -> portal-frontend for SPA routes and assets
43 -> portal for /sdk/*, /discovery*, /v1/sign, and /api/* relay paths
44 -> portal-api for /ui/* presentation API paths
45
46 Tunnel clients and public app visitors
47 -> https://*.portal.example.com
48 -> nginx TCP passthrough
49 -> portal SNI listener
50 ```
51
52 ### Public Routing
53
54 | Public request | nginx behavior | Upstream |
55 |---|---|---|
56 | `portal.example.com/`, `/admin`, SPA assets | Terminate TLS, HTTP proxy | `portal-frontend:8080` |
57 | `/sdk/*`, `/discovery*`, `/v1/sign` | Terminate TLS, HTTP proxy | `portal:4017` over HTTPS |
58 | `/api/*` | Terminate TLS, HTTP proxy | `portal:4017` over HTTPS |
59 | `/ui/*` | Terminate TLS, HTTP proxy | `portal-api:8081` |
60 | `*.portal.example.com` | Raw TCP passthrough with `ssl_preread` | `portal` SNI listener |
61
62 The root relay host needs HTTP path routing, so it is not TCP-passthrough. Wildcard app hosts need TCP passthrough, so nginx must not terminate TLS for them.
63
64 ### Migration From Embedded Frontend
65
66 Older deployments could run only `ghcr.io/gosuda/portal` because the relay served frontend assets. Current production deployment separates that into:
67
68 - `portal` for relay API and tunnel ingress
69 - `portal-frontend` for static SPA assets
70 - `portal-api` for frontend-owned dynamic behavior
71 - `nginx` as the public TLS edge
72
73 Operators upgrading from the embedded frontend must deploy all three Portal images and route them through nginx. `PORTAL_URL` remains the browser-facing HTTPS origin, for example `https://portal.example.com`; do not set it to `localhost` or an internal Docker hostname for a public relay.
74
75 ### Security Boundary
76
77 To keep the same practical security level as the embedded frontend deployment:
78
79 - Public users reach the dashboard only through `https://portal.example.com`.
80 - `portal:4017`, `portal-frontend:8080`, and `portal-api:8081` are not exposed directly to the internet.
81 - Root-host relay protocol paths (`/sdk/*`, `/discovery*`, `/v1/sign`) and relay JSON API paths (`/api/*`) are HTTP reverse-proxied by nginx to the relay API upstream, while `/ui/*` presentation paths go to `portal-api`.
82 - Wildcard app hosts are TCP-passthrough to the relay SNI listener.
83 - The nginx browser certificate and the relay API certificate are separate operational concerns unless you intentionally share the same certificate files.
84
85 It is fine for nginx to terminate public TLS and then proxy to the relay API over HTTPS internally. That is two TLS legs. TCP passthrough is only for wildcard tunnel app hosts.
86
87 ## 2. Prerequisites
88
89 You need:
90
91 - A public domain, for example `portal.example.com`.
92 - A public Linux server with a static public IPv4.
93 - Docker and Docker Compose.
94 - DNS `A` records for the relay host and wildcard host:
95
96 ```text
97 portal.example.com -> <server-ip>
98 *.portal.example.com -> <server-ip>
99 ```
100
101 If you use Cloudflare, keep these records `DNS only`. Proxied records break the raw wildcard TCP passthrough path.
102
103 Open only the public ports that match the topology:
104
105 | Port | Required | Purpose |
106 |---|---|---|
107 | `80/tcp` | optional | HTTP to HTTPS redirect in a front nginx |
108 | `443/tcp` | yes | Public nginx edge for dashboard, relay API path routing, and wildcard TCP passthrough |
109 | `WIREGUARD_PORT/udp` | when `DISCOVERY=true` | Relay discovery WireGuard transport |
110 | `SNI_PORT/udp` | when UDP transport is enabled | QUIC tunnel ingress |
111 | `MIN_PORT-MAX_PORT/udp` | when UDP lease transport is enabled | Public UDP lease ports |
112 | `MIN_PORT-MAX_PORT/tcp` | when raw TCP lease transport is enabled | Public raw TCP lease ports |
113
114 Keep these ports private or loopback-only in the recommended topology:
115
116 | Port | Owner |
117 |---|---|
118 | `4017/tcp` | `portal` relay API |
119 | `8080/tcp` | `portal-frontend` static server |
120 | `8081/tcp` | `portal-api` presentation API |
121
122 Certificate files are also split by owner:
123
124 | Certificate | Location | Used by |
125 |---|---|---|
126 | Browser-facing HTTPS certificate | Edge-specific certificate path | nginx instance that terminates `portal.example.com` |
127 | Relay API and SNI certificate | `./.portal-certs/fullchain.pem`, `./.portal-certs/privatekey.pem` | `portal` unless managed ACME is configured |
128
129 Portal-managed ACME can manage the relay certificate and relay DNS records. If a separate front nginx only TCP-passthroughs Portal hostnames, it does not need Portal certificate material; the Portal nginx or relay edge behind it still terminates the Portal root host.
130
131 ## 3. Deploy the Recommended Stack
132
133 Deploy the Portal services with your own Compose, systemd, or orchestration
134 manifest. If another nginx already owns public `443/tcp`, use the nginx
135 passthrough example only for SNI routing to the Portal nginx:
136
137 ```bash
138 cp <repo>/docs/static/examples/nginx-proxy/nginx.conf ./nginx.conf
139 ```
140
141 Replace `portal.example.com` and the `portal_nginx` upstream in `nginx.conf`.
142 The example does not deploy Portal containers, write Portal `.env` files, or
143 reload Portal services.
144
145 ### Configure `.env`
146
147 Minimal production baseline:
148
149 ```bash
150 PORTAL_URL=https://portal.example.com
151 BOOTSTRAPS=
152 DISCOVERY=true
153 IDENTITY_PATH=/portal-certs
154
155 API_PORT=4017
156 SNI_PORT=443
157 WIREGUARD_PORT=51820
158 MIN_PORT=0
159 MAX_PORT=0
160 UDP_ENABLED=false
161 TCP_ENABLED=false
162
163 ACME_DNS_PROVIDER=
164 ENS_GASLESS_ENABLED=false
165
166 TRUST_PROXY_HEADERS=true
167 TRUSTED_PROXY_CIDRS=
168
169 LANDING_PAGE_ENABLED=false
170 ```
171
172 `API_PORT` defaults to `4017`. Keep `SNI_PORT=443` because this is the public
173 SNI port advertised to tunnel clients. If a separate nginx sits in front of
174 Portal's own nginx, it should TCP-passthrough `portal.example.com` and
175 `*.portal.example.com` instead of proxying Portal API paths itself.
176
177 If the relay joins public discovery, set `BOOTSTRAPS` to at least one reachable relay URL and keep `WIREGUARD_PORT/udp` open.
178
179 Set `ADMIN_TOKEN` to a long random value before exposing the admin UI or policy API.
180
181 Leave `TRUSTED_PROXY_CIDRS` empty for the default private and loopback proxy ranges. Set it only when you need a stricter proxy source allowlist.
182
183 ### Prepare Certificates and State
184
185 Create the state directories:
186
187 ```bash
188 mkdir -p ./.portal-certs/frontend-state ./certs
189 sudo chown 65532:65532 ./.portal-certs
190 chmod 755 ./.portal-certs
191 ```
192
193 Place the nginx browser certificate here:
194
195 ```text
196 ./certs/fullchain.pem
197 ./certs/privkey.pem
198 ```
199
200 In manual relay certificate mode, also place the relay certificate here before startup:
201
202 ```text
203 ./.portal-certs/fullchain.pem
204 ./.portal-certs/privatekey.pem
205 ```
206
207 You may use the same certificate material for nginx and the relay when it covers both `portal.example.com` and `*.portal.example.com`; keep the filenames expected by each service.
208
209 When `ACME_DNS_PROVIDER` is configured, Portal can create and renew the relay certificate under `IDENTITY_PATH`. That does not remove nginx's need for its own browser-facing certificate under `./certs`.
210
211 ### Start and Verify
212
213 Start the stack:
214
215 ```bash
216 docker compose up -d
217 ```
218
219 Verify the public edge:
220
221 ```bash
222 curl -I https://portal.example.com
223 docker compose ps
224 ```
225
226 Expected service names in the recommended stack:
227
228 - `nginx`
229 - `portal`
230 - `portal-frontend`
231 - `portal-api`
232
233 If `https://portal.example.com` loads the dashboard and tunnel app hosts under `*.portal.example.com` reach the relay, the topology is correct.
234
235 ## 4. Certificate and DNS Automation
236
237 Choose one certificate and DNS mode for the relay.
238
239 | Mode | `ACME_DNS_PROVIDER` | Relay cert source | DNS automation |
240 |---|---|---|---|
241 | Manual certificate | empty | `IDENTITY_PATH/fullchain.pem` and `IDENTITY_PATH/privatekey.pem` | none |
242 | Manual certificate plus gasless DNS | DNSSEC-capable provider | manual files | ENS TXT and DNSSEC automation |
243 | Managed ACME | supported provider | Portal-managed ACME DNS-01 | root/wildcard A records, ECH HTTPS records, relay cert renewal |
244
245 Supported provider values:
246
247 | Provider | Required environment | ENS gasless support |
248 |---|---|---|
249 | `cloudflare` | `CLOUDFLARE_TOKEN` | yes |
250 | `gcloud` | Google ADC, optionally `GCP_PROJECT_ID`, `GCP_MANAGED_ZONE`, `GOOGLE_APPLICATION_CREDENTIALS` | yes |
251 | `route53` | AWS credentials or instance role, optionally `AWS_HOSTED_ZONE_ID` | yes, needs an active KSK or `AWS_DNSSEC_KMS_KEY_ARN` |
252 | `vultr` | `VULTR_API_KEY` | yes |
253 | `hetzner` | `HETZNER_API_TOKEN` | no |
254 | `njalla` | `NJALLA_TOKEN` | no |
255
256 For `gcloud` with a service account file under Docker Compose, mount the file and point `GOOGLE_APPLICATION_CREDENTIALS` at the in-container path:
257
258 ```yaml
259 services:
260 portal:
261 environment:
262 GOOGLE_APPLICATION_CREDENTIALS: /run/secrets/gcp-dns.json
263 volumes:
264 - ./.portal-certs:/portal-certs
265 - ./gcp-dns.json:/run/secrets/gcp-dns.json:ro
266 ```
267
268 ### ENS Gasless Automation
269
270 ENS gasless DNS import is optional and not required for normal relay operation.
271
272 Enable it only when you need ENS-aware clients to resolve Portal domains through gasless DNSSEC import:
273
274 ```bash
275 ACME_DNS_PROVIDER=cloudflare
276 CLOUDFLARE_TOKEN=cf_xxxxxxxxxxxxxxxxx
277 ENS_GASLESS_ENABLED=true
278 ```
279
280 Operational notes:
281
282 - ENS gasless requires `ACME_DNS_PROVIDER`.
283 - Portal writes `ENS1 0x238A8F792dFA6033814B18618aD4100654aeef01 <address>` TXT records.
284 - The base domain uses the relay identity address; lease hostnames use each lease identity address.
285 - Provider-side DNSSEC automation is not the same as registrar-side DS publication.
286 - If the provider returns a `DS` record or reports DNSSEC as pending, publish the DS record at your registrar and wait for parent-zone propagation.
287 - Keep `ENS_GASLESS_ENABLED=false` unless you intentionally use this feature.
288
289 Verification checklist:
290
291 ```bash
292 dig +short DS portal.example.com
293 dig +short TXT portal.example.com
294 ```
295
296 Provider DNSSEC should be active, and the TXT response should include the `ENS1 ...` value.
297
298 ## 5. Optional UDP and Raw TCP Transport
299
300 UDP transport and raw TCP lease transport are disabled by default.
301
302 Open these ports in your cloud security group or host firewall only when the matching feature is enabled:
303
304 - `WIREGUARD_PORT/udp` when discovery is enabled.
305 - `SNI_PORT/udp` when UDP tunnel ingress is enabled.
306 - `MIN_PORT-MAX_PORT/udp` when UDP lease transport is enabled.
307 - `MIN_PORT-MAX_PORT/tcp` when raw TCP lease transport is enabled.
308
309 Example with `MIN_PORT=40000`, `MAX_PORT=40009`, and `SNI_PORT=443`:
310
311 ```bash
312 sudo ufw allow 51820/udp
313 sudo ufw allow 443/udp
314 sudo ufw allow 40000:40009/udp
315 sudo ufw allow 40000:40009/tcp
316 ```
317
318 Configure the shared lease range in `.env`:
319
320 ```bash
321 MIN_PORT=40000
322 MAX_PORT=40009
323 UDP_ENABLED=true
324 TCP_ENABLED=true
325 ```
326
327 When using bridge networking, publish the same range in `docker-compose.yaml`:
328
329 ```yaml
330 ports:
331 - "${WIREGUARD_PORT:-51820}:${WIREGUARD_PORT:-51820}/udp"
332 - "${SNI_PORT:-443}:${SNI_PORT:-443}/udp"
333 - "${MIN_PORT:-40000}-${MAX_PORT:-40009}:${MIN_PORT:-40000}-${MAX_PORT:-40009}/udp"
334 - "${MIN_PORT:-40000}-${MAX_PORT:-40009}:${MIN_PORT:-40000}-${MAX_PORT:-40009}"
335 ```
336
337 UDP and raw TCP use the same numeric range independently, so the same number can be allocated once for UDP and once for TCP.
338
339 After startup, enable UDP or raw TCP policy in the admin UI and set any lease limits you want to enforce.
340
341 For better QUIC performance on Linux:
342
343 ```bash
344 sudo sysctl -w net.core.rmem_max=7500000
345 sudo sysctl -w net.core.wmem_max=7500000
346 ```
347
348 Persist those values in `/etc/sysctl.conf` or a file under `/etc/sysctl.d/` if needed.
349
350 ## 6. Frontend Presentation API
351
352 `portal-api` is a small TypeScript service owned by the frontend deployment. It keeps frontend-specific behavior out of the Go relay.
353
354 It owns:
355
356 - `/ui/state` composition with frontend-owned fields.
357 - `/ui/policy/*` composition, while relay-enforced policy changes are still forwarded to `portal`.
358 - `/ui/service/status`, derived from relay state for quick-start UI checks.
359 - `/ui/thumbnail/<hostname>`, when optional screenshot generation is enabled.
360 - The landing-page flag persisted at `PORTAL_FRONTEND_STATE_PATH`; a Compose deployment can store it under `./.portal-certs/frontend-state/state.json`.
361
362 The Go relay remains the owner of authentication, policy enforcement, lease state, tunnel ingress, install scripts, and discovery paths.
363
364 ### Custom Frontend
365
366 To attach your own dashboard frontend, replace only the `portal-frontend`
367 service image. Keep the service name `portal-frontend` and serve plain HTTP on
368 port `8080` so the existing nginx route for SPA paths can continue to point at
369 `portal-frontend:8080`.
370
371 The custom frontend should use same-origin browser requests and leave these
372 paths owned by the Portal services:
373
374 | Path | Owner |
375 |---|---|
376 | `/ui/*` | `portal-api` presentation API |
377 | `/api/*` | `portal` relay API |
378 | `/sdk/*`, `/discovery*`, `/v1/sign` | `portal` relay protocols |
379 | `*.portal.example.com` | `portal` SNI listener |
380
381 ```bash
382 cp <repo>/docs/static/examples/custom-frontend/docker-compose.override.yaml ./docker-compose.override.yaml
383 docker compose up -d portal-frontend
384 ```
385
386 If the automated release updater should also track your custom frontend image,
387 set `IMAGES` to include that image in addition to the Portal release-track
388 images.
389
390 ### Thumbnail Screenshots
391
392 Generated thumbnails are optional and disabled by default. Without this feature, apps without a custom thumbnail simply use the default card background.
393
394 To enable generated thumbnails:
395
396 1. Uncomment the `headless-shell` service in `docker-compose.yaml`.
397 2. Add `headless-shell` to `portal-api.depends_on`.
398 3. Set `HEADLESS_SHELL_URL=ws://headless-shell:9222`.
399 4. Restart with `docker compose up -d`.
400
401 Expected log when a thumbnail is captured:
402
403 ```text
404 thumbnail captured hostname=myapp.portal.example.com size=36209
405 ```
406
407 Disable the feature by removing `HEADLESS_SHELL_URL` and stopping the `headless-shell` container.
408
409 ## 7. Automated Release Updates
410
411 Auto-update should follow a published release tag, not `latest`. The `latest`
412 image tag tracks default-branch image builds, so using it can update production
413 on a `main` merge before the GitHub Release and tunnel binaries are published.
414
415 Production deployments should follow the v2 release track directly.
416 Auto-update must pull all production images from that same release track:
417
418 - `ghcr.io/gosuda/portal:2`
419 - `ghcr.io/gosuda/portal-frontend:2`
420 - `ghcr.io/gosuda/portal-api:2`
421
422 The auto-update example watches those image digests, pulls the changed official
423 images, recreates the Portal services, and reloads the Portal nginx:
424
425 ```bash
426 cp <repo>/docs/static/examples/auto-update/watch_and_deploy.sh ./watch_and_deploy.sh
427 chmod +x watch_and_deploy.sh
428 ```
429
430 The example watcher reads Portal images from Docker Compose when available,
431 falls back to the v2 image set, and updates `portal`, `portal-frontend`, and
432 `portal-api` when any watched release tag changes.
433
434 Systemd example:
435
436 ```bash
437 sudo tee /etc/systemd/system/portal-watcher.service << 'EOF'
438 [Unit]
439 Description=Portal Docker Image Watcher
440 After=network-online.target docker.service
441 Wants=network-online.target
442 Requires=docker.service
443
444 [Service]
445 Type=simple
446 User=opc
447 WorkingDirectory=<path-to-project>
448 ExecStart=/bin/bash <path-to-project>/watch_and_deploy.sh
449 Restart=always
450 RestartSec=10
451 Environment=INTERVAL=60
452
453 [Install]
454 WantedBy=multi-user.target
455 EOF
456
457 sudo systemctl daemon-reload
458 sudo systemctl enable --now portal-watcher
459 ```
460
461 Adjust `User` and paths to match your server. The service user must be able to run Docker.
462
463 Monitor it with:
464
465 ```bash
466 sudo systemctl status portal-watcher
467 sudo journalctl -u portal-watcher -f
468 ```
469
470 ## 8. Troubleshooting
471
472 ### `4017` Shows Only API
473
474 That is expected. `4017/tcp` is the relay API, not the dashboard. Use `https://portal.example.com` through nginx for the production UI.
475
476 ### Frontend Logs Show Binary TLS Bytes and `400`
477
478 Logs like `"\x16\x03\x01..." 400` mean a client sent HTTPS to the plain HTTP `portal-frontend:8080` listener. Do not expose `8080` publicly. Put nginx with TLS in front of it.
479
480 ### Relay Logs Show `tls: unknown certificate`
481
482 This usually means a browser or proxy hit the relay API certificate directly instead of the public nginx certificate, or an upstream proxy tried to verify the relay's internal certificate. In the recommended topology, public browsers verify the Portal HTTPS edge certificate, while the edge proxies to the relay API over internal HTTPS.
483
484 ### Root Host Works but Wildcard Apps Fail
485
486 Check that `portal.example.com` is HTTP-proxied after TLS termination and that `*.portal.example.com` is TCP-passthrough to the relay SNI listener. Do not terminate TLS for wildcard app hosts in nginx.
487
488 ### Discovery Announce Is Rejected as Local-Only
489
490 Public discovery rejects `PORTAL_URL` hosts such as `localhost`, `127.0.0.1`, `::1`, or other local-only names. Set `PORTAL_URL` to a publicly reachable HTTPS hostname.
491
492 ### Docker DNS Resolution Fails
493
494 If logs show `discover bootstraps failed`, `sync dns records`, or `lookup <host> on 127.0.0.11:53: write: operation not permitted`, Docker is usually using the wrong host resolver config.
495
496 On Linux hosts with `systemd-resolved`, point `/etc/resolv.conf` at the upstream resolver list and restart Docker:
497
498 ```bash
499 sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
500 sudo systemctl restart docker
501 docker compose up -d
502 ```
503
504 Verify from the container:
505
506 ```bash
507 docker run --rm --network container:portal busybox nslookup api4.ipify.org
508 ```
509
510 ### Ports Are Blocked
511
512 Confirm the required public ports are open:
513
514 ```bash
515 sudo ufw allow 443/tcp
516 sudo ufw allow 51820/udp
517 sudo ufw status
518 ```
519
520 Only add UDP and raw TCP lease ranges when those transports are enabled.