@samitouri / QOSAMI-X402Tunnel / commits / 6e3831da

fix AGENTS and docs

Kim committed Mar 6, 2026 at 14:43 UTC 6e3831dad944d083bfbd699b5ae0889588cb1a32
7 files changed +277 -208
AGENTS.md
+50 -54
@@ -6,91 +6,85 @@ This file is a high-signal rulebook for future agents.
6 Include only constraints that are expensive to rediscover from quick code search.
7
8 Source of truth for architecture decisions: `docs/adr/README.md` and linked ADRs.
9 +Descriptive docs under `docs/` should match current code paths.
10
11 ## Non-Negotiable Architecture Invariants
12
13 1. **Raw TCP reverse-connect is the canonical transport.**
13 - - Why: ADR-0001 and ADR-0002 accepted this to keep NAT-friendly behavior and reduce protocol complexity.
14 + - Why: keeps backend connectivity NAT-friendly and avoids parallel transport stacks.
15
16 2. **Do not introduce websocket or legacy compatibility paths unless a new ADR supersedes ADR-0002.**
17 - Why: dual transport paths increase security and test surface and reintroduce drift.
18
18 -3. **Derive routing hostnames from full portal root host in `PORTAL_URL` (supports non-apex), not apex extraction.**
19 - - Why: prevents SNI/public URL mismatches in non-apex deployments (ADR-0001).
19 +3. **Derive routing hostnames from the full portal root host in `PORTAL_URL` (supports non-apex), not apex extraction.**
20 + - Why: prevents SNI/public URL mismatches in non-apex deployments.
21
21 -4. **Keep explicit root-domain fallback behavior through SNI no-route handling to admin/API listener.**
22 - - Why: preserves intended control-plane vs tenant routing split (ADR-0001).
22 +4. **Keep explicit root-domain fallback behavior through SNI no-route handling to the admin/API listener.**
23 + - Why: preserves the intended split between root-host control-plane traffic and tenant subdomain traffic.
24
24 -5. **All leases require TLS=true.** The register endpoint rejects `TLS=false`. The `RegisterRequest.TLS` field exists but non-TLS leases are not permitted.
25 - - Why: enforces end-to-end transport security for all tenant routes.
26 -
27 -## Security and Anti-Abuse Invariants
28 -
29 -1. **Admin-managed policy is authoritative for runtime security controls.**
30 - - Why: ADR-0003 requires central policy ownership to avoid endpoint-local drift.
31 -
32 -2. **Do not rely on single-endpoint checks for abuse controls.**
33 - - Why: ADR-0003 expects enforcement across critical ingress paths (registration and reverse admission class paths).
34 -
35 -3. **Reverse connection authorization must remain lease-token validated before bridge/forwarding.**
36 - - Why: prevents unauthorized tunnel attachment (ADR-0003).
37 -
38 -4. **`/sdk/connect` must share the same policy source as `/sdk/register`.**
39 - - Why: registration and reverse admission must apply identical IP-ban + token checks in one enforcement pipeline.
25 +5. **All leases require `TLS=true`.** The register endpoint rejects `TLS=false`.
26 + - Why: all tenant routes are expected to stay on the TLS passthrough path.
27
28 ## TLS and Identity Invariants
29
43 -1. **Relay terminates admin/API TLS directly and also exposes `/v1/sign` as the keyless signer for tenant passthrough TLS.** The relay still does not terminate tenant traffic; it peeks ClientHello for SNI and bridges raw encrypted bytes, while the backend/tunnel endpoint performs the handshake using a `RemoteSigner`.
44 - - Why: preserves SNI passthrough data flow while centralizing certificate signing behind the relay keyless endpoint.
30 +1. **Relay terminates admin/API TLS on the root host and also exposes `/v1/sign` for tenant-side keyless signing.**
31 + - Relay still does not terminate tenant TLS. It peeks ClientHello for SNI and bridges raw encrypted bytes.
32 + - SDK/tunnel endpoints terminate tenant TLS locally with a keyless-backed signer that calls the relay.
33
46 -2. **/sdk/* control-plane auth is token.** Admission order is IP ban -> Lease -> Token.
47 - - Admin/API TLS listener does not request client certificates.
48 - - Keyless TLS (RemoteSigner for /v1/sign) remains independent and is still used for admin/API TLS termination.
49 - - Why: removes browser client-cert prompt side effects while keeping centralized token/IP policy enforcement.
34 +2. **`/sdk/connect`, `/sdk/renew`, and `/sdk/unregister` are authorized by lease existence plus reverse token.**
35 + - `/sdk/register` requires the caller to provide a reverse token for later use, but registration itself is not separately authenticated by that token.
36
51 -3. **All relay URLs must be `https://`.** `NormalizeRelayAPIURL` rejects non-HTTPS. SDK and tunnel hard-fail on `http://`.
52 - - Why: enforces transport security without opt-out.
37 +3. **All relay URLs must be `https://`.**
38 + - Why: SDK and tunnel are expected to hard-fail on insecure relay URLs.
39
54 -4. **`keyless_tls/` is published to `github.com/gosuda/keyless_tls`** and vendored as a directory for co-development. Root `go.mod` pins a specific pseudo-version or tag — no `replace` directive.
55 - - Why: enables external consumers while keeping co-development convenient. Pin version after pushing upstream changes.
40 +4. **HTTP/2 is intentionally disabled on the admin/API TLS listener.**
41 + - Why: `/sdk/connect` depends on HTTP/1.1 hijacking semantics.
42
57 -## SNI Routing Invariants
43 +## Reverse Session Protocol
44
59 -1. **SNI wildcard matching is one-level only.** `sni.Router.GetRoute()` checks `*.parent.example.com` for `foo.parent.example.com` — not arbitrary depth.
60 - - Why: matches RFC TLS wildcard semantics.
45 +1. **SNI wildcard matching is one-level only.**
46 + - `*.parent.example.com` matches `foo.parent.example.com`, not arbitrary depth.
47
62 -2. **Protocol markers on reverse TCP connections:** `0x00` = keepalive, `0x01` = non-TLS start, `0x02` = TLS passthrough activation. The SNI router does true TLS passthrough — it peeks the ClientHello for routing, then bridges the raw encrypted connection without terminating TLS.
63 - - Why: binary protocol, not discoverable from HTTP-layer code.
48 +2. **Protocol markers on reverse TCP connections are still meaningful protocol state.**
49 + - `0x00` = idle keepalive
50 + - `0x02` = TLS passthrough activation
51 + - Why: SDK waits on these bytes before switching an idle reverse session into a claimed tenant TLS session.
52
65 -3. **HTTP/2 is intentionally disabled on the admin HTTP server** (`TLSNextProto: make(…)`).
66 - - Why: the server hijacks connections for `/sdk/connect`; HTTP/2 multiplexing breaks hijack semantics.
53 +3. **`/sdk/connect` must remain HTTP/1.1 only.**
54 + - Why: hijacking and long-lived reverse sessions depend on HTTP/1.1 connection ownership.
55
56 ## API Response Contract
57
70 -1. **All HTTP responses use the `APIEnvelope` wrapper:** `{ ok: bool, data?: any, error?: { code, message } }` (defined in `types/api.go`). Write responses through `writeAPIData()`, `writeAPIOK()`, or `writeAPIError()` helpers — never raw JSON.
71 - - Why: cross-cutting contract across all endpoints; inconsistent envelopes break SDK and frontend parsing.
58 +1. **All JSON control-plane responses use the `APIEnvelope` wrapper:** `{ ok: bool, data?: any, error?: { code, message } }` (defined in `types/api.go`).
59 + - Write responses through `writeAPIData()`, `writeAPIOK()`, or `writeAPIError()`.
60 + - Admin HTML pages, tunnel script/binary responses, and other non-JSON endpoints are exceptions.
61
62 ## Shared Types Package
63
75 -1. **`types/` is reserved for shared wire/public types and protocol constants only.**
76 - - Allowed: request/response DTOs, shared metadata structs, protocol marker/header/path constants.
64 +1. **`types/` is reserved for shared wire/public types and cross-package constants only.**
65 + - Allowed: request/response DTOs, shared metadata structs, protocol markers, shared headers, shared public path constants.
66 - Not allowed: relay runtime state, broker/session state, server config, SDK lifecycle state, generic helpers.
67
79 -2. **Do not import `portal` from `cmd/*` or `sdk` just to reach shared DTOs or protocol constants.**
68 +2. **Shared control-plane and public route constants that cross package boundaries belong in `types/paths.go`.**
69 + - Examples: `/sdk/*`, `/v1/sign`, `/healthz`, `/admin`, `/admin/leases`, `/tunnel`.
70 +
71 +3. **Relay-local frontend asset paths stay local to `cmd/relay-server`.**
72 + - Why: filenames like `favicon.svg` or `portal.jpg` are frontend serving details, not cross-package API contract.
73 +
74 +4. **Do not import `portal` from `cmd/*` or `sdk` just to reach shared DTOs or constants.**
75 - Why: `portal` is relay runtime code; shared public shapes belong in `types/`.
76
77 ## Operational Truths (CI-Aligned, Minimal)
78
84 -1. **CI verification commands:** `make vet`, `make lint`, `make test`, `make vuln`. These are the enforced checks in `.github/workflows/ci.yml`. Note: `make tidy` is a local maintenance/pre-release step, not part of CI.
85 -
86 -2. **`make build-server` does NOT call `make build-frontend`.** If called alone, `//go:embed dist/*` will be stale or empty. The Dockerfile calls both explicitly in order.
87 - - Why: prevents silent broken builds with missing frontend assets.
79 +1. **CI verification commands:** `make vet`, `make lint`, `make test`, `make vuln`.
80 + - `make tidy` is a local maintenance step, not part of CI.
81
89 -3. **`admin_settings.json` persists in the process CWD**, not in `KEYLESS_DIR`. State is lost on container restart unless CWD is a mounted volume.
90 - - Why: prevents state-loss surprises in production.
82 +2. **`make build-server` does not build the frontend first.**
83 + - Why: `cmd/relay-server/dist/*` is embed input; build the frontend explicitly before packaging the relay binary.
84
92 -4. **`onLeaseDeleted` has dual registration.** `portal/relay.go` registers one callback; `cmd/relay-server/main.go` overwrites it with a broader one (adds IP/BPS cleanup). The outer callback supersedes.
93 - - Why: coupling hazard — modifying either registration without understanding both breaks cleanup.
85 +3. **ACME management keeps both root and wildcard DNS A records in sync for non-localhost deployments.**
86 + - Certificates and keys live under `KEYLESS_DIR` as `fullchain.pem` and `privatekey.pem`.
87 + - Localhost uses the development certificate path instead of Cloudflare-managed ACME.
88
89 ## Change Discipline
90
@@ -102,16 +96,18 @@ Source of truth for architecture decisions: `docs/adr/README.md` and linked ADRs
96
97 ## Go Conventions
98
105 -**Imports:** stdlib → external → internal (blank-line separated), local prefix `github.com/gosuda`. **Concurrency:** `errgroup.Group` over `WaitGroup` · `errgroup.SetLimit` for bounded work · `context.WithTimeout` over `time.After` in loops (timer leak) · no bare `go func()` — creator owns lifecycle. **I/O:** `os.Root` (Go 1.24+) for directory-scoped file operations.
99 +**Imports:** stdlib -> external -> internal (blank-line separated), local prefix `gosuda.org/portal`.
100 +**Concurrency:** `errgroup.Group` over `WaitGroup`; `errgroup.SetLimit` for bounded work; `context.WithTimeout` over `time.After` in loops; no bare `go func()` without owned lifecycle.
101 +**I/O:** prefer directory-scoped operations when touching filesystem trees.
102
103 ---
104
105 ## Agent Behavior
106
111 -**Verbalized sampling:** Before changes, sample 3–5 intent hypotheses (rank by likelihood, note one weakness each), assess coupling (structural/temporal/semantic), and tidy-first when coupling is high. Ask the human when trade-offs exist.
107 +**Verbalized sampling:** Before changes, sample 3-5 intent hypotheses (rank by likelihood, note one weakness each), assess coupling (structural/temporal/semantic), and tidy-first when coupling is high. Ask the human when trade-offs exist.
108
109 **Project rules:**
110 - No backward compatibility unless explicitly requested.
111 - No wrapper functions without demonstrated value.
116 -- Consolidate changes in one pass — do not stack minimal patches.
112 +- Consolidate changes in one pass; do not stack minimal patches.
113 - Run tests only when requested, before handoff, or for high-risk changes.
docs/adr/0003-security-and-anti-abuse-hardening.md
+4 -1
@@ -1,6 +1,6 @@
1 # ADR 0003: Security and Anti-Abuse Hardening
2
3 -- Status: `Accepted`
3 +- Status: `Deprecated`
4 - Date: `2026-03-03`
5 - Owners: `Portal maintainers`
6
@@ -8,6 +8,9 @@
8
9 Portal accepts unauthenticated internet traffic on relay/admin edges while managing long-lived reverse tunnel sessions. Abuse controls and security boundaries must be first-class or operational risk rises quickly.
10
11 +This ADR captured a broader policy/bans/admin-settings model than the current runtime now implements.
12 +Keep it as historical context only. Current descriptive docs and `AGENTS.md` are the source for the simplified runtime behavior until a replacement ADR is written.
13 +
14 ## Decision
15
16 - Treat admin-authenticated controls (approval, settings, bans) as authoritative for runtime policy.
docs/adr/README.md
+1 -1
@@ -13,7 +13,7 @@ This directory is the source of truth for major Portal architecture decisions.
13
14 - [0001 - Raw TCP Reverse Connect and ACME TLS for Portal Root](./0001-raw-tcp-reverse-connect-and-autocert-tls.md) (`Accepted`)
15 - [0002 - Remove WebSocket and Legacy Compatibility Paths](./0002-remove-websocket-and-legacy-compatibility.md) (`Accepted`)
16 -- [0003 - Security and Anti-Abuse Hardening](./0003-security-and-anti-abuse-hardening.md) (`Accepted`)
16 +- [0003 - Security and Anti-Abuse Hardening](./0003-security-and-anti-abuse-hardening.md) (`Deprecated`)
17
18 ## Authoring Notes
19
docs/architecture.md
+125 -81
@@ -2,144 +2,188 @@
2
3 ## Overview
4
5 -Portal is a relay network for publishing local services on public subdomains.
6 -The relay is both the control plane and routing plane. Service backends connect outward to the relay (NAT-friendly), and clients connect to the relay domain.
5 +Portal publishes local services on public subdomains through a relay.
6 +Backends connect outward to the relay, the relay routes inbound client traffic by SNI, and tenant TLS remains end-to-end between the browser and the SDK/tunnel endpoint.
7
8 High-level path:
9
10 ```text
11 Client (Browser)
12 - -> Relay (:443 SNI router or :4017 HTTP/API)
13 - -> Reverse tunnel connection
14 - -> Local service (App/Tunnel host)
12 + -> Relay SNI listener (:443 by default)
13 + -> Claimed reverse session
14 + -> SDK / portal-tunnel
15 + -> Local service
16 ```
17
17 -## Connection Responsibilities
18 +## Connection Model
19
19 -- Conn #1 (`browser -> app`) is the tenant-facing data plane.
20 - - Data-plane TLS behavior remains end-to-end between client and app/tunnel host.
21 - - Relay forwards tenant traffic and does not replace app identity policy.
22 -- Conn #2 (`relay -> tunnel`) is the control plane.
23 - - `/sdk/register`, `/sdk/connect`, `/sdk/renew`, and `/sdk/unregister` use token-based control-plane admission.
24 - - Control-plane admission order is strict and deterministic: `IP -> Lease -> Token`.
25 - - Client certificates are not part of runtime control-plane admission.
20 +Portal has two distinct network roles:
21 +
22 +- **Control-plane HTTP requests**
23 + - `POST /sdk/register`
24 + - `POST /sdk/renew`
25 + - `POST /sdk/unregister`
26 + - `GET /sdk/domain`
27 +- **Reverse session connection**
28 + - `GET /sdk/connect?lease_id=...`
29 + - HTTP/1.1 only
30 + - hijacked into a long-lived raw TCP session
31 + - starts idle in the relay broker, then becomes the tenant data path when claimed
32 +
33 +That distinction matters because `/sdk/connect` stops being ordinary HTTP once hijacked.
34
35 ## Core Components
36
37 ### Relay Server (`cmd/relay-server`)
38
31 -- API/Admin server on `--adminport` (default `:4017`)
32 -- SNI router on `--sni-port` (default `:443`)
33 -- SDK registry endpoints under `/sdk/*`
34 -- Keyless signer endpoint at `/v1/sign` (when signer is configured)
39 +- Admin/API TLS listener on `--adminport` (default `:4017`)
40 +- SNI listener on `--sni-port` (default `:443`)
41 +- Public frontend routes under `/`, `/app`, `/assets/*`
42 +- Minimal admin surface at `/admin` and `/admin/leases`
43 +- Tunnel bootstrap routes at `/tunnel` and `/tunnel/bin/*`
44 +- Keyless signer endpoint at `/v1/sign`
45
46 ### Relay Core (`portal/`)
47
38 -- `LeaseManager`: lease registration, renew TTL, ban/list policy
39 -- `ReverseHub`: authenticated reverse connection pool per lease
40 -- `sni.Router`: TCP listener that peeks SNI and routes to lease backends
41 -- `acme` + `keyless`: ACME provisioning and remote signing support
42 -
43 -Anti-abuse policy is driven from admin-managed state and applied consistently for both registration and reverse admission.
48 +- `Server`: owns listeners, lease registry, API handlers, and shutdown lifecycle
49 +- `routeTable`: exact + single-label wildcard hostname lookup
50 +- `leaseBroker`: per-lease ready queue for reverse sessions
51 +- `reverseSession`: idle keepalive + activation state machine for one reverse TCP connection
52 +- `acme`: Cloudflare DNS sync + certificate provisioning/renewal for the relay root host and wildcard
53 +- `keyless`: admin/API TLS attach helpers and tenant-side signer integration
54
55 ### SDK (`sdk/`)
56
47 -- `Client`: bootstrap relay URLs and optional TLS/keyless setup
48 -- `Listener`: relay-backed `net.Listener` used by apps/tunnel clients
49 -- Shared API types and paths (`/sdk/register`, `/sdk/renew`, etc.)
57 +- `Client`: validates relay URL, owns HTTP client and raw TLS dial config
58 +- `Listener`: registers a lease, maintains `readyTarget` reverse sessions, renews lease TTL, and yields accepted tenant TLS connections
59 +- Tenant TLS is created automatically through the relay keyless signer; callers do not provide a local self-signed fallback path
60
61 ### Tunnel (`cmd/portal-tunnel`)
62
53 -- CLI proxy for existing local apps without code changes
54 -- Registers lease via SDK and forwards relay traffic to local `--host`
63 +- Registers a lease through the SDK
64 +- Accepts claimed tenant connections from the relay
65 +- Proxies raw TCP to a local `--host`
66 +- Returns an HTTP 503 response when the local target is unavailable
67
68 ## Transport Model
69
58 -### Raw reverse transport (`lease.TLS=true` only)
70 +### Raw reverse transport (`TLS=true` only)
71
60 -1. Relay requires registered leases to use TLS.
61 -2. Backend opens a raw TCP reverse connection via `GET /sdk/connect?lease_id=...`.
62 -3. Clients connect via HTTPS to relay SNI port.
63 -4. Relay resolves route by SNI and acquires a reverse connection from `ReverseHub`.
64 -5. Tunnel-side listener performs TLS handshake (keyless-backed signer), while relay forwards raw TCP transparently.
65 -6. No alternate control-plane transport is supported; relay/tunnel transport stays raw TCP reverse-connect only.
72 +1. SDK/tunnel registers a lease with `POST /sdk/register`.
73 +2. SDK opens one or more reverse sessions with `GET /sdk/connect?lease_id=...`.
74 +3. Relay hijacks each `/sdk/connect` request and places the connection in the per-lease broker ready queue.
75 +4. While idle, the relay writes `0x00` keepalive markers.
76 +5. A browser connects to the relay SNI listener.
77 +6. Relay extracts SNI from ClientHello, resolves a lease, and claims one ready reverse session.
78 +7. Relay writes `0x02` to activate that session.
79 +8. SDK/tunnel receives `0x02`, starts tenant TLS locally using the relay-backed keyless signer, and the relay bridges raw encrypted bytes end-to-end.
80
67 -Result: the relay handles SNI-based routing and transparent raw TCP forwarding, preserving end-to-end TLS where applicable.
81 +Result: the relay decides routing, but tenant TLS termination still happens at the SDK/tunnel side.
82
83 ## Control Plane Flow
84
85 ### 1. Register
86
73 -- App/tunnel posts to `POST /sdk/register` with:
74 - - `lease_id`
87 +- `POST /sdk/register`
88 +- JSON envelope response
89 +- Caller provides:
90 - `name`
76 - - `metadata`
77 - - `tls`
91 - `reverse_token`
79 -- Relay stores lease and (TLS only) registers SNI route.
80 -- Route hostnames are generated from normalized lease + normalized `PORTAL_URL` host (extract host from URL without scheme/port/path); path segments are ignored, so `https://portal.example.com:8443/admin` and `https://portal.example.com` both map to `portal.example.com`.
81 -- `/sdk/register` admission uses strict order: `IP -> Lease -> Token`.
92 + - `tls=true`
93 + - optional `hostnames`
94 + - optional `metadata`
95 + - optional `ttl_seconds`
96 +- If no hostname is supplied, relay derives one from `name + root host`
97 +- `PORTAL_URL` is normalized to its host component only; path/query segments are ignored for routing
98
99 ### 2. Reverse Connect
100
85 -- Backend opens a raw TCP reverse connection to `GET /sdk/connect` and streams traffic over that long-lived connection
86 - - `/sdk/connect` applies strict admission order before hijacking:
87 - - `IP -> Lease -> Token`
88 -- `X-Portal-Reverse-Token` is validated at HTTP precheck, then validated again in `ReverseHub` with centralized policy callbacks before the connection is pooled.
89 -- Connection is pooled in `ReverseHub` only after token/IP checks pass.
101 +- `GET /sdk/connect?lease_id=...`
102 +- Requires HTTP/1.1
103 +- Requires `X-Portal-Token` header with the lease reverse token
104 +- Relay validates:
105 + - lease exists and is not expired
106 + - reverse token matches the registered lease token
107 +- After hijack, the connection becomes a broker-managed reverse session
108
109 ### 3. Renew
110
93 -- Backend sends `POST /sdk/renew` keepalive.
94 -- `/sdk/renew` requires `reverse_token`.
95 -- Relay refreshes lease TTL and keeps route state current.
111 +- `POST /sdk/renew`
112 +- Requires `lease_id` + `reverse_token`
113 +- Extends lease TTL
114 +- Resets a previously dropped broker back to active
115
116 ### 4. Unregister
117
99 -- Backend sends `POST /sdk/unregister`.
100 -- `/sdk/unregister` validates normalized `lease_id` and token before deletion.
101 -- Relay removes lease, route, and reverse pool.
118 +- `POST /sdk/unregister`
119 +- Requires `lease_id` + `reverse_token`
120 +- Removes the lease, routes, and ready reverse sessions
121
103 -## Admin Lease ID Contract
122 +## Routing Behavior
123
105 -- `/admin/leases` returns plain lease IDs in `Peer`.
106 -- `/admin/leases/banned` returns plain lease IDs (`[]string`).
107 -- Base64URL encoding is used only in admin action path segments (`/admin/leases/{encodedLeaseID}/{action}`).
124 +Route lookup order:
125
109 -## Routing Behavior
126 +1. Exact hostname match
127 +2. Single-label wildcard match (`*.example.com`)
128 +3. Root-host fallback to the admin/API listener
129 +
130 +Notes:
131 +
132 +- Wildcards are one level only.
133 +- The exact root host is never served by the wildcard route.
134 +- For non-apex `PORTAL_URL` values such as `https://portal.example.com:8443/admin`, public lease hosts become `<lease>.portal.example.com`.
135
111 -`sni.Router` route lookup order:
136 +## Admin and Frontend Surface
137
113 -1. Exact host match
114 -2. Single-label wildcard (`*.example.com`)
115 -3. No-route handler (used for exact portal-root host fallback to admin/API listener on the `PORTAL_URL` root host)
138 +Current relay-served public routes:
139
117 -Note: wildcard does not match the portal root host itself (`example.com` or `portal.example.com`), so exact root-host matches trigger fallback to admin/API listener.
118 -`PORTAL_URL` is normalized to its host component (scheme/port/path removed), so non-apex values such as `https://portal.example.com:8443/admin` become `<lease>.portal.example.com` and exact host matches still resolve through no-route fallback.
140 +- `/`
141 +- `/app`
142 +- `/app/*`
143 +- `/assets/*`
144 +- `/admin`
145 +- `/admin/leases`
146 +- `/tunnel`
147 +- `/tunnel/bin/*`
148 +- `/healthz`
149 +- `/v1/sign`
150 +- `/sdk/*`
151 +
152 +The admin surface is intentionally small in the current Go runtime: an HTML index plus a JSON lease list.
153 +
154 +## Shared Contract Surface
155 +
156 +Cross-package public contract lives in:
157 +
158 +- `types/api.go`
159 + - API envelope
160 + - shared request/response DTOs
161 + - lease metadata
162 + - reverse marker/header constants
163 +- `types/paths.go`
164 + - shared `/sdk/*`, admin, health, tunnel, and signer paths
165 +
166 +Relay-local frontend asset filenames stay in `cmd/relay-server`, not `types/`.
167
168 ## Keyless and Certificates
169
122 -- Relay keyless materials are stored in `KEYLESS_DIR`:
170 +- Relay admin/API TLS uses the certificate in `KEYLESS_DIR`
171 - `fullchain.pem`
172 - `privatekey.pem`
125 -- If materials are missing and Cloudflare token is configured, relay can provision via ACME DNS-01.
126 -- SDK/tunnel TLS mode uses keyless signer workflow and `/v1/sign` for remote signatures.
173 +- For non-localhost deployments, ACME DNS-01 uses Cloudflare and keeps:
174 + - root host A record
175 + - wildcard host A record
176 + - relay certificate renewal
177 +- SDK/tunnel fetches the relay certificate chain, verifies it covers tenant hostnames, and uses `/v1/sign` for remote signatures during tenant TLS handshakes
178
179 ## Design Properties
180
130 -- Reverse-only backend connectivity (no inbound port required on the app host)
131 -- Per-lease reverse token authorization
132 -- Separation of control plane (`/sdk/*`) and data plane (SNI + raw TCP forwarding)
133 -- Single relay/tunnel transport policy: raw TCP reverse-connect only
134 -- Control-plane identity policy: token-based admission with deterministic order
135 -- Unified lease abstraction for routing, metadata, and lifecycle
136 -- Shared anti-abuse path: admin-managed bans and lease authorization are enforced both in SDK registration and reverse admission
137 -
138 -## Breaking-Change Upgrade Expectations
139 -
140 -- Control-plane admission is token-only.
141 -- Tunnels/SDK clients must provide valid lease tokens for `/sdk/*` admission.
142 -- Client certificates are not required for `/sdk/*` admission.
181 +- Reverse-only backend connectivity
182 +- One canonical raw TCP reverse transport
183 +- SNI-based routing with root-host fallback
184 +- End-to-end tenant TLS with relay-backed keyless signing
185 +- Per-lease reverse token authorization for reverse session lifecycle
186 +- Lease-local reverse session ownership through `leaseBroker`
187
188 ## ADRs
189
docs/deployment.md
+32 -17
@@ -25,7 +25,7 @@ You need:
25
26 Cloudflare Dashboard -> `DNS` -> `Records`:
27
28 -- Record 1 (apex)
28 +- Record 1 (root host)
29 - Type: `A`
30 - Name: `@`
31 - Content: `<server-ip>`
@@ -41,20 +41,13 @@ Expected records:
41 - `example.com -> <server-ip>`
42 - `*.example.com -> <server-ip>`
43
44 -If you deploy on a non-apex host (for example, `PORTAL_URL=https://portal.example.com:8443`), create host-scoped records:
44 +If you deploy on a non-apex host (for example, `PORTAL_URL=https://portal.example.com:8443`), create host-scoped records instead:
45
46 - `portal.example.com -> <server-ip>`
47 - `*.portal.example.com -> <server-ip>`
48
49 -Portal normalizes `PORTAL_URL` to its host for routing, so public service hosts become `<lease>.portal.example.com`.
50 -Requests to the exact `PORTAL_URL` host (for example, `portal.example.com`) are not wildcard-matched; the router uses no-route fallback and forwards them to the admin/API listener.
51 -Relay/tunnel traffic for reverse admission stays raw TCP on `/sdk/connect`.
52 -
53 -### 2.4 Control-Plane Admission (Token-Only)
54 -
55 -- `/sdk/register`, `/sdk/connect`, `/sdk/renew`, and `/sdk/unregister` use token-based admission.
56 -- Control-plane admission order is fixed: `IP -> Lease -> Token`.
57 -- Clients without valid lease token are rejected.
49 +Portal derives public lease hostnames from the normalized `PORTAL_URL` host.
50 +Requests to the exact root host are not served by the wildcard route; they fall back to the admin/API listener.
51
52 ### 2.3 Create Cloudflare API Token
53
@@ -71,9 +64,31 @@ Scope:
64
65 Save this token for `CLOUDFLARE_TOKEN`.
66
74 -## 3. Run Relay Server
67 +## 3. Relay Runtime Behavior
68 +
69 +### 3.1 Control Plane and Reverse Sessions
70 +
71 +- `/sdk/register` creates a lease and stores the caller-provided reverse token.
72 +- `/sdk/connect` requires:
73 + - `lease_id` query parameter
74 + - `X-Portal-Token` header
75 + - HTTP/1.1
76 +- `/sdk/renew` and `/sdk/unregister` require `lease_id` + `reverse_token`.
77 +- `/sdk/connect` is hijacked into a long-lived reverse TCP session after validation.
78 +
79 +### 3.2 Certificates and DNS Maintenance
80 +
81 +- Relay certificates live in `KEYLESS_DIR`:
82 + - `fullchain.pem`
83 + - `privatekey.pem`
84 +- On non-localhost deployments, ACME DNS-01 uses the Cloudflare token to:
85 + - ensure root and wildcard A records point to the current public IP
86 + - provision the relay certificate
87 + - keep DNS and certificate state refreshed over time
88 +
89 +## 4. Run Relay Server
90
76 -### 3.1 Create `.env` at repository root
91 +### 4.1 Create `.env` at repository root
92
93 ```bash
94 PORTAL_URL=https://example.com
@@ -84,18 +99,18 @@ KEYLESS_DIR=/etc/portal/keyless
99 CLOUDFLARE_TOKEN=cf_xxxxxxxxxxxxxxxxx
100 ```
101
87 -For non-apex deployments, set `PORTAL_URL` and `BOOTSTRAP_URIS` to the same non-apex host value (for example, `https://portal.example.com:8443`). Keep any path segments only for dashboard use, not for routing.
102 +For non-apex deployments, set `PORTAL_URL` and `BOOTSTRAP_URIS` to the same non-apex host value (for example, `https://portal.example.com:8443`).
103 `PORTAL_URL` path/query segments are ignored for route derivation; only the host component is used.
104
90 -### 3.2 Start Relay
105 +### 4.2 Start Relay
106
107 ```bash
108 docker compose up
109 ```
110
96 -## 4. Troubleshooting
111 +## 5. Troubleshooting
112
98 -### 4.1 Ports blocked
113 +### 5.1 Ports blocked
114
115 Required inbound ports:
116
docs/glossary.md
+48 -39
@@ -4,79 +4,88 @@ Key terms used in Portal.
4
5 ## Portal / Relay
6
7 -The central server that handles lease registration, routing, and reverse-connection brokering.
8 -In TLS passthrough mode, it routes transport and does not terminate app payload TLS.
9 -All backend-to-relay ingress uses a long-lived raw TCP reverse-connect channel (`/sdk/connect`).
7 +The central server that handles lease registration, SNI routing, reverse-session brokering, admin/API TLS, and keyless signing.
8 +It does not terminate tenant TLS.
9
10 ## App (Service Publisher)
11
13 -A backend service connected to Portal through Tunnel or Native SDK.
14 -An app publishes one or more leases and serves traffic from local services.
12 +A backend service connected to Portal through `portal-tunnel` or the native Go SDK.
13 +An app publishes one or more leases and serves traffic from a local process.
14
15 ## Client (Service Consumer)
16
18 -A browser or external caller that accesses a published service through relay-managed domains.
17 +A browser or external caller that accesses a published service through relay-managed hostnames.
18
20 -## Conn #1 (Data Plane)
19 +## Control-Plane Request
20
22 -Tenant-facing traffic path between browser/client and app/tunnel endpoint.
23 -This connection keeps existing data-plane TLS behavior.
21 +An ordinary HTTPS request to the relay admin/API listener, such as:
22
25 -## Conn #2 (Control Plane)
23 +- `/sdk/register`
24 +- `/sdk/renew`
25 +- `/sdk/unregister`
26 +- `/sdk/domain`
27
27 -Relay-to-tunnel control path used by `/sdk/register`, `/sdk/connect`, `/sdk/renew`, and `/sdk/unregister`.
28 -This connection uses token-based admission with order `IP -> Lease -> Token`.
28 +These use the JSON API envelope contract.
29
30 -## Tunnel
30 +## Reverse Session
31
32 -The CLI publisher path (`cmd/portal-tunnel`).
33 -It forwards relay traffic to an existing local host/port without app code changes.
34 -
35 -## Native SDK
36 -
37 -The Go integration path (`sdk/`).
38 -It provides relay-backed listener APIs and lease metadata control for direct integration.
32 +A long-lived raw TCP connection opened through `GET /sdk/connect?lease_id=...`.
33 +The relay hijacks it, keeps it idle in a lease broker, and later claims it for one tenant TLS passthrough connection.
34
35 ## Lease
36
37 Portal's routing and advertisement unit.
43 -Each lease maps to one public endpoint and includes identity, name, metadata, TLS flag, and reverse token.
38 +Each lease has an ID, display name, hostnames, metadata, expiry, reverse token, and a broker of ready reverse sessions.
39
40 ## Lease Name
41
47 -The human-readable identifier used for subdomain routing (for example, `myapp` -> `myapp.example.com`).
42 +The human-readable identifier used to derive a hostname (for example, `myapp` -> `myapp.example.com`) when no explicit hostname is supplied.
43 +
44 +## Lease Broker
45 +
46 +The relay-side owner of reverse session state for one lease.
47 +It manages the ready queue, claim/wakeup behavior, drop/stop lifecycle, and idle keepalive policy.
48
49 ## Reverse Token
50
51 -A per-lease secret used to authenticate reverse connections (`/sdk/connect`) from backend to relay.
52 -Token validation is a required admission stage after lease and policy checks.
51 +A per-lease secret supplied at registration time and later required by:
52
54 -## ReverseHub
53 +- `/sdk/connect`
54 +- `/sdk/renew`
55 +- `/sdk/unregister`
56
56 -Relay-side pool of authenticated reverse connections keyed by lease ID.
57 -It supplies raw TCP reverse connections for TLS SNI forwarding.
57 +It authorizes reverse-session and lease-lifecycle operations.
58
59 -## SNI Router
59 +## Route Table
60
61 -The TCP router on relay SNI port (default `443`) that selects lease routes by TLS SNI.
62 -Exact matches on the portal root host (derived from `PORTAL_URL` host) are intentionally routed via no-route fallback to the admin/API listener.
61 +The relay hostname map that resolves exact and single-label wildcard matches to a lease ID.
62 +
63 +## SNI Routing
64 +
65 +The relay reads ClientHello to extract the requested hostname, chooses a lease route, and then bridges the original encrypted TLS stream without terminating it.
66
67 ## Keyless TLS
68
66 -A mode where the backend performs TLS while using the relay signer endpoint (`/v1/sign`) for remote signing.
67 -This avoids distributing private keys to every backend host.
69 +A mode where the SDK/tunnel terminates tenant TLS locally while delegating private-key signing to the relay `/v1/sign` endpoint.
70 +This keeps the relay out of the tenant data plane while avoiding direct private-key distribution to every backend host.
71
72 ## ACME DNS-01
73
71 -Certificate issuance/renewal method used with a Cloudflare DNS API token when keyless materials are missing.
74 +The relay certificate issuance and renewal path for non-localhost deployments.
75 +It uses a Cloudflare DNS API token to provision the root and wildcard certificate coverage used by the relay.
76
73 -## Base Domain
77 +## Base Domain / Root Host
78
75 -The host extracted from `PORTAL_URL` (scheme, port, and path removed) and used to build service subdomains.
76 -For non-apex values such as `https://portal.example.com:8443/admin`, the base host is `portal.example.com`.
77 -The same host is used for exact-match SNI fallback, which routes root-host requests to the admin/API listener.
79 +The host extracted from `PORTAL_URL` after removing scheme, port, path, query, and fragment.
80 +For `https://portal.example.com:8443/admin`, the root host is `portal.example.com`.
81
82 ## Admin/API Server
83
81 -The relay HTTP server (default `:4017`) serving admin UI and control endpoints such as `/sdk/*`, `/admin`, and `/healthz`.
82 -It also receives root-domain fallback traffic from SNI when no more specific lease route is found.
84 +The relay HTTPS listener (default `:4017`) serving:
85 +
86 +- `/sdk/*`
87 +- `/admin`
88 +- `/admin/leases`
89 +- `/healthz`
90 +- `/v1/sign`
91 +- frontend root/app routes through root-host fallback
frontend/AGENTS.md
+17 -15
@@ -5,31 +5,33 @@ High-signal constraints for the relay-server frontend. Only items expensive to r
5 ## Frontend-Backend Contracts (Manually Synced)
6
7 1. **SSR data shape is a 3-way contract.**
8 - Go `leaseRow` (`cmd/relay-server/utils.go`) ↔ TS `ServerData` (`src/hooks/useSSRData.ts`) ↔ `<script id="__SSR_DATA__">` injection (`cmd/relay-server/frontend.go`).
8 + Go `leaseRow` (`cmd/relay-server/frontend.go`) -> TS `ServerData` (`src/hooks/useSSRData.ts`) -> `<script id="__SSR_DATA__">` injection (`cmd/relay-server/frontend.go`).
9 - Why: no shared schema or codegen. Field drift silently breaks SSR hydration. The script tag ID `__SSR_DATA__` is hardcoded in all three locations.
10
11 2. **API path constants require dual maintenance.**
12 - Go definitions in `types/api.go`, TS duplicates in `src/lib/apiPaths.ts`.
13 - - Why: no codegen. A path mismatch produces silent 404s on same-origin requests.
12 + Go definitions live in `types/paths.go`; TS duplicates live in `src/lib/apiPaths.ts`.
13 + - Why: no codegen. A path mismatch produces same-origin 404s.
14 + - Current Go runtime only serves `/admin` and `/admin/leases` on the admin surface. Extra TS admin paths need matching backend work or they will 404.
15
16 3. **API envelope shape must match across Go and TS.**
16 - All responses use `{ ok, data?, error?: { code, message } }`. Go helpers (`utils.go`) and TS parser (`src/lib/apiClient.ts`) must agree. TS treats a missing boolean `ok` field as `invalid_envelope`.
17 - - Why: backend responses that skip the envelope (e.g., raw middleware errors) throw `invalid_envelope` instead of displaying the error.
17 + All JSON control-plane responses use `{ ok, data?, error?: { code, message } }`.
18 + Go shape is `types.APIEnvelope` in `types/api.go`; Go writers live in `portal/api.go`; TS parser lives in `src/lib/apiClient.ts`.
19 + - Why: backend responses that skip the envelope surface as `invalid_envelope` in the frontend.
20
19 -4. **Lease ID encoding uses base64url without padding.**
20 - Frontend encodes via `btoa` + character replacement (`src/lib/apiPaths.ts`). Backend two-pass decodes: `base64.URLEncoding` then `base64.RawURLEncoding` (`cmd/relay-server/utils.go`).
21 - - Why: standard base64 in either direction → 400 errors. `btoa` is browser-only — not portable to Node/SSR without polyfill.
22 -
23 -5. **Build output renames `index.html` to `portal.html`.**
24 - Vite plugin `rename-index` (`vite.config.ts`) performs this post-build. Go backend serves `portal.html`, not `index.html`. The rename is skipped when `VITEST` env is set.
21 +4. **Build output renames `index.html` to `portal.html`.**
22 + Vite plugin `rename-index` (`vite.config.ts`) performs this post-build. Go backend serves `portal.html`, not `index.html`. The rename is skipped when `VITEST` is set.
23 - Why: any tooling or script assuming `index.html` post-build will fail.
24
27 -6. **OG metadata placeholders must match between HTML and Go.**
28 - `index.html` (becomes `portal.html`) contains `[%OG_TITLE%]`, `[%OG_DESCRIPTION%]`, `[%OG_IMAGE_URL%]`. Server-side substitution happens in `cmd/relay-server/frontend.go`.
29 - - Why: renaming a placeholder in one place without the other → raw uninjected strings in responses.
25 +5. **OG metadata placeholders must match between HTML and Go.**
26 + `index.html` (renamed to `portal.html`) contains `[%OG_TITLE%]`, `[%OG_DESCRIPTION%]`, `[%OG_IMAGE_URL%]`. Server-side substitution happens in `cmd/relay-server/frontend.go`.
27 + - Why: renaming a placeholder in one place without the other leaves raw placeholder strings in production HTML.
28 +
29 +6. **Frontend admin action helpers currently outpace the Go runtime.**
30 + `src/lib/apiPaths.ts` still contains action routes such as `/admin/leases/{id}/{action}` and `/admin/settings/*`.
31 + - Why: do not assume those routes exist in the current Go relay unless you implement the backend in the same change.
32
33 ## Frontend Conventions
34
35 1. **Do not use `useCallback` in new code.**
36 React Compiler (`babel-plugin-react-compiler`, enabled in `vite.config.ts`) handles memoization automatically.
35 - - Why: manual `useCallback` is redundant with the compiler and adds noise. Existing usage in `useAdmin.ts` and `ServerListView.tsx` is legacy — remove when touching those files.
37 + - Why: manual `useCallback` is redundant with the compiler and adds noise.