Fast and extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS

caddyhttp, reverseproxy: experimental WebTransport passthrough (#7669)

* caddy: pass ListenQUIC options as a struct Collapse the TLS config, packet-conn wrappers, and Allow0RTT arguments into ListenQUICOptions so the signature matches Listen's ctx/portOffset/ ListenConfig prefix. Zero-value opts keep the historical default of allowing 0-RTT. The API remains experimental. * caddyhttp: advertise WebTransport in HTTP/3 SETTINGS Add github.com/quic-go/webtransport-go dep (built on quic-go, same maintainers) and call webtransport.ConfigureHTTP3Server on the http3.Server. This advertises WebTransport enablement in SETTINGS, enables HTTP/3 DATAGRAMs, and stashes the *quic.Conn in each request's context — a prerequisite for a later WebTransport-aware handler or reverse-proxy transport to call webtransport.Server.Upgrade. Also enable QUIC stream reset partial delivery, required by webtransport-go. No user-visible behavior change: clients that don't speak WebTransport ignore the extra SETTINGS, and no handler yet calls Upgrade. Extract the http3.Server construction into buildHTTP3Server so the SETTINGS assertions can be unit-tested without a live UDP listener. * caddyhttp: add UnwrapResponseWriterAs helper Go's type assertion `x.(T)` does not follow Unwrap() http.ResponseWriter chains. Caddy wraps the writer multiple times (logging recorder, intercept, encode, etc.), so code that needs interfaces implemented only by the raw writer owned by the HTTP server — for example the http3.Settingser/HTTPStreamer interfaces that webtransport.Server.Upgrade type-asserts — cannot see through those wrappers. UnwrapResponseWriterAs walks the Unwrap() chain and returns the first writer that satisfies the requested interface (or the zero value if none do). Mirrors the traversal http.ResponseController performs internally. Used by upcoming WebTransport handler and reverse-proxy transport. * caddyhttp: add terminating WebTransport handler Introduces http.handlers.webtransport, an EXPERIMENTAL handler that terminates an incoming WebTransport session on top of Caddy's HTTP/3 server and echoes bytes on each bidirectional stream. Primary use case is as a test upstream for the forthcoming WebTransport reverse-proxy transport; it also serves as the minimal proof that the server-side WebTransport wiring works end-to-end. Plumbing changes: * caddyhttp.Server gains a *webtransport.Server field alongside h3server. It's built in buildWebTransportServer(), wrapping the existing http3.Server. Exposed via WebTransportServer() any on the Server, so the caddyhttp public API does not name the upstream webtransport-go type (per AGENTS.md). * serveHTTP3 now runs a custom accept loop (serveH3AcceptLoop) that dispatches each accepted QUIC connection to webtransport.Server.ServeQUICConn instead of http3.Server.ServeListener. The WebTransport server transparently forwards non-WT streams to the underlying http3 request handler (cost: one varint peek per stream), so behavior for non-WT clients is unchanged. * ListenQUIC enables EnableDatagrams and EnableStreamResetPartialDelivery on the QUIC listener config. These are capability bits negotiated during the QUIC handshake and are prerequisites for any WebTransport session; they do not force usage so non-WT H3 traffic is unaffected. * Stop path closes wtServer after h3server.Shutdown to clean up any remaining WebTransport session state. The handler uses caddyhttp.UnwrapResponseWriterAs to reach the naked http3.Settingser/HTTPStreamer writer through Caddy's wrapping chain before calling webtransport.Server.Upgrade. Includes unit tests for request-shape detection plus an integration test (caddytest/integration/webtransport_test.go) that spins up a Caddy HTTP/3 server with the handler, dials it with a real webtransport.Dialer, and asserts end-to-end bidirectional-stream echo. * reverseproxy: add WebTransport upstream dialer helper dialUpstreamWebTransport is a thin wrapper around webtransport.Dialer.Dial that sets the QUIC config flags WebTransport requires (EnableDatagrams, EnableStreamResetPartialDelivery) and forwards request headers on the Extended CONNECT. Intended as an internal building block for the upcoming WebTransport reverse-proxy transport; not yet wired into ServeHTTP. Unit-tested against an in-process webtransport.Server with a freshly minted self-signed certificate. Covers: successful dial, header forwarding, and connection-refused against an unbound port. * reverseproxy: add WebTransport session pump runWebTransportPump bridges two WebTransport sessions so every bidirectional stream, unidirectional stream, and datagram opened on one side is mirrored on the other. Uses six goroutines (bidi both ways, uni both ways, datagrams both ways) and blocks until both sessions end. Close propagation: when either session ends, the peer is closed via CloseWithError. The code/message are read from the closing session's stored close state (by probing AcceptStream with a short timeout), since Receive{Datagram,UniStream} return the underlying stream error rather than the SessionError and can win the propagation race. Close propagation is best-effort for client-initiated close through a Dialer-dedicated QUIC conn: webtransport-go tears down the QUIC connection immediately after CloseWithError, so the upstream may observe a QUIC ApplicationError before the WT_CLOSE_SESSION capsule is parsed. The pump still closes the peer session; only the specific error code may not survive. Not yet wired into ServeHTTP. Tests: topology of client -> frontend -> upstream where frontend runs the pump. Exercises bidi both ways, uni client-to-upstream, datagram round-trip, CloseWithError propagation both ways, and a basic goroutine-leak check. * reverseproxy: wire WebTransport pump into ServeHTTP Extends the http reverse-proxy transport with a webtransport boolean that opts the upstream into WebTransport passthrough. Must be combined with versions: ["3"]; WebTransport rides on HTTP/3 exclusively. When enabled, Handler.ServeHTTP detects Extended CONNECT with :protocol=webtransport early — before any of the normal round-trip machinery — and branches to serveWebTransport, which: 1. Pulls the *webtransport.Server off caddyhttp.Server (via WebTransportServer()) and errors out cleanly if HTTP/3 isn't enabled on the frontend. 2. Picks a single upstream through the configured load-balancer. No retries: a failed dial closes the client session and returns. 3. Walks the response-writer Unwrap() chain to reach the raw http3 writer and calls webtransport.Server.Upgrade to terminate the incoming session. 4. Uses dialUpstreamWebTransport to open a session to the selected upstream, forwarding request headers on the Extended CONNECT. 5. Runs runWebTransportPump between the two sessions and blocks until both close. The transport's wtTLSConfig is built at Provision time from the existing TLS config (same path h3Transport already uses) and reused for every session. Tests: adds TestWebTransport_ReverseProxyEndToEnd which spins up a single Caddy instance with two HTTP/3 servers — one proxy on :9443, one terminating echo upstream on :9444 — and drives a real webtransport.Dialer through the proxy to assert end-to-end bidirectional-stream echo. * reverseproxy: add webtransport Caddyfile subdirective Adds a `webtransport` subdirective to the `transport http {}` block of reverse_proxy that sets the new WebTransport bool on the transport. Takes no arguments; exclusivity with versions 3 is enforced at Provision time so parse order doesn't matter. Example: reverse_proxy https://backend:9443 { transport http { versions 3 webtransport tls_insecure_skip_verify } } Includes a Caddyfile-to-JSON adapt test round-tripping the new subdirective. * reverseproxy: apply standard request preparation to WebTransport CONNECT The WebTransport proxy path previously bypassed the request-preparation pipeline that normal reverse-proxy traffic runs through. Reuse it so `header_up`, `X-Forwarded-For`/`Host`/`Proto`, `Via`, `Rewrite`, the `{http.reverse_proxy.upstream.*}` placeholders, dynamic upstreams, `countFailure`, and the `{http.reverse_proxy.duration{_ms}}` timing placeholder all behave the same as on the regular path. Retries, `handle_response`, and response-header ops are intentionally not run here — a WebTransport session has no HTTP response body to post-process and is not idempotent. Integration test exercises the header-forwarding contract end-to-end through a standalone (non-Caddy) WebTransport upstream so the forwarded Extended CONNECT can be inspected. * reverseproxy: dial WebTransport upstream before upgrading client Reorder serveWebTransport so the upstream is dialed first. If the upstream is unreachable or refuses the CONNECT, a proper 5xx is returned from the handler — the client's Dial() surfaces the real status instead of a successful upgrade followed by an opaque session close. Also apply `h.Headers.Response` (gated by `Require`, if configured) against the upstream response status/headers; the ops run on the client-visible response headers, which webtransport.Server.Upgrade flushes with the 200 OK. If the client-side upgrade fails after the upstream dial succeeded, close the upstream session cleanly. Integration test drives a dial to an unbound loopback port and asserts the client sees a 5xx status instead of a bare session close. * reverseproxy: track WebTransport sessions in upstream in-flight counters Bracket the pump's lifetime with Host.countRequest(±1) and incInFlightRequest/decInFlightRequest so WT sessions participate in the same accounting as the normal proxy path: - MaxRequests gating (Upstream.Full) now blocks WT sessions past the cap, instead of silently failing open. - LeastConn / FirstAvailable selection sees WT load, instead of seeing busy upstreams as idle. - Admin /reverse_proxy/upstreams reports WT sessions under num_requests. Integration test holds an upstream session open via a standalone WT server, polls the admin API to assert num_requests increments during the session and drops back to 0 after close. * reverseproxy: extract shared upstream-selection helpers The WebTransport proxy path in serveWebTransport duplicated the dynamic-upstream-fallback block and the {http.reverse_proxy.upstream.*} replacer-variable block from proxyLoopIteration. Francis flagged this as a maintenance burden in review of #7669. Extract two helpers: * resolveUpstreams(r) returns the candidate upstream set — dynamic when configured (with provisioning + fallback-on-error), static otherwise. Caller runs the LB selection policy, since the two call sites diverge on how selection failure is reported (retry loop vs. fast 502 for long-lived WT sessions). * setUpstreamReplacerVars(repl, up, di) publishes the seven placeholders describing the selected upstream. Both are used by proxyLoopIteration and serveWebTransport with identical semantics to the inlined code they replace. No behavior change for either path. * reverseproxy: inline WebTransport protocol const and writer interface Francis pointed out in review of #7669 that importing the whole modules/caddyhttp/webtransport package solely to pull in one constant and one interface wasn't worthwhile. Move both into webtransport_transport.go as unexported identifiers (webtransportProtocol, webtransportWriter). This removes reverseproxy's dependency on the caddywt package and clears the way for moving the echo handler itself out of the production module tree. No behavior change. * caddyhttp: move WebTransport echo handler to integration tests Francis pointed out in review of #7669 that the echo handler — which exists solely as a test upstream for the WebTransport reverse-proxy tests — should not be a full-fledged module registered in every Caddy binary. Mirroring the mockdns_test.go pattern, move it into a _test.go file under caddytest/integration/. The module ID http.handlers.webtransport is now registered only when the integration test binary is built, which is when caddytest/integration/webtransport_test.go references it by ID string in its JSON configs. Production Caddy builds no longer include it. Changes: * New file: caddytest/integration/webtransport_echo_test.go — contains the WebTransportEcho handler, its types and interface guards, the isWebTransportEchoUpgrade helper, and the unit tests that used to live in the deleted package's handler_test.go. * Deleted: modules/caddyhttp/webtransport/ (handler.go + handler_test.go). * Removed the blank import from modules/caddyhttp/standard/imports.go. The Protocol const and Writer interface that this package used to export were inlined into reverseproxy's own files in a preceding commit, so nothing else depends on the deleted package. * caddyhttp, reverseproxy: gate WebTransport behind enable_webtransport server flag steadytao raised an architectural concern in review of #7669: the PR put experimental WebTransport handling directly into Caddy's core HTTP/3 accept path, so every HTTP/3 deployment paid for the feature whether or not they used it. Collapse the enablement surface to a single server-level opt-in that matches Caddy's existing precedent for protocol-level features (`protocols`, `allow_0rtt`, `enable_full_duplex`), and detect the request shape at the handler the same way `reverse_proxy` detects a WebSocket upgrade today — no per-handler config flag. Core HTTP/3 path changes (modules/caddyhttp/server.go): * New `EnableWebTransport bool` field on Server, marked EXPERIMENTAL. * buildHTTP3Server now only calls webtransport.ConfigureHTTP3Server and sets EnableStreamResetPartialDelivery when the flag is true. When false, the constructed http3.Server is bit-for-bit identical to the pre-WebTransport implementation. * wtServer is constructed only when the flag is true. * serveH3AcceptLoop falls back to http3.Server.ServeListener when the flag is false — no varint peek, no per-connection dispatch. Caddyfile wiring (caddyconfig/httpcaddyfile/serveroptions.go): * New `enable_webtransport` global server option, modeled on `enable_full_duplex`. Reverse-proxy simplifications (modules/caddyhttp/reverseproxy/): * Removed HTTPTransport.WebTransport field and its Provision-time exclusivity check (no longer needed; H3 is validated separately). * Removed the `webtransport` Caddyfile subdirective under `transport http { }` — this neutralizes the prior commit that introduced it. * Removed Handler.webtransportEnabled cache. ServeHTTP now branches on isWebTransportExtendedConnect(r) alone, matching how the WebSocket upgrade branch works. * serveWebTransport gains fail-fast guards with clear errors when the parent server has enable_webtransport=false or when the handler's transport does not include HTTP/3. Tests: * Existing TestServer_BuildHTTP3ServerEnablesWebTransport now sets EnableWebTransport=true explicitly; new TestServer_BuildHTTP3ServerWithoutWebTransport locks in the regression guard that flag-off produces the pre-PR http3.Server. * Integration tests updated: enable_webtransport: true added to every H3 server block; "webtransport": true dropped from the reverse_proxy transport JSON (auto-detected now). * Caddyfile adapt test for the deleted `webtransport` subdirective is removed; `enable_webtransport` is added to the existing global_server_options_single adapt test alongside its peers. No runtime behavior change when enable_webtransport is false. Diff against master on the core HTTP/3 path is effectively zero in that configuration. * caddyhttp: micro-benchmark HTTP/3 server construction with/without WebTransport Adds a hermetic pair of benchmarks on buildHTTP3Server to provide quantitative evidence for the claim that deployments with enable_webtransport=false pay no cost for the WebTransport feature. Results on Apple M4, go1.25, -count=5: BenchmarkBuildHTTP3Server_WebTransportOff ~70 ns/op 392 B/op 3 allocs/op BenchmarkBuildHTTP3Server_WebTransportOn ~144 ns/op 600 B/op 6 allocs/op The Off path is about half the cost on every dimension, confirming that the work skipped when the flag is false is the webtransport.ConfigureHTTP3Server call plus EnableStreamResetPartialDelivery. Absolute cost is a one-time per-server setup so either branch is negligible in practice, but the asymmetry locks in a regression guard: a future refactor that accidentally re-enables the WT configuration unconditionally would show up as a jump in the Off numbers. This benchmark does not exercise the per-stream dispatch cost inside webtransport.Server.ServeQUICConn — that would require a full QUIC setup to measure in isolation and is follow-up work. * reverseproxy: collapse WebTransport handler into main proxy loop The WT path duplicated upstream resolution, LB selection, header ops, replacer vars, and in-flight counters. Route WT through the shared ServeHTTP -> proxyLoopIteration -> reverseProxy flow and swap RoundTrip for a small webTransportHijack that only does WT-specific work (writer unwrap, upstream dial, client upgrade, pump). Rename roundtripSucceededError -> terminalError. The existing name described when it was emitted (after a successful round-trip); the new name describes its contract with the retry loop (stop looping, propagate error unchanged). The WebTransport upgrade case is a second natural caller for that same signal. Comes with two behavior improvements that fall out of the collapse: - WT upstream dial failures now surface as DialError, so the loop can fail over across upstreams like normal proxies (today: 502). - Passive health checks apply to WT dials (dial-failure countFailure and UnhealthyLatency on dial duration) via the shared path. Addresses reviewer feedback that the duplicated setup was a maintenance risk. * reverseproxy: inline upstream-resolution helpers back into the proxy loop The earlier #7669 revision extracted resolveUpstreams and setUpstreamReplacerVars to share between the WebTransport path and the normal proxy path. A later commit collapsed the WebTransport path into the main proxy loop, so the helpers became single-callers — they no longer earn their abstraction. Inline both back into proxyLoopIteration. The dynamic-upstream block absorbs the CachingUpstreamSource cache-reset logic added by #7662, which was preserved through the rebase. No behavior change. * caddyhttp: clarify WebTransportServer doc covers both disabled paths WebTransportServer returns nil for two distinct configurations: EnableWebTransport=false on the server, and HTTP/3 not in use. The prior doc only mentioned the latter. Spell out both, per review. * reverseproxy: drop nil-logger guard in runWebTransportPump Production and test callers both pass a non-nil *zap.Logger. The defensive nil check silently swapped in a no-op logger, which can mask a real bug at the call site. Document the precondition in the doc comment instead, per review. * reverseproxy: use sync.WaitGroup.Go in WebTransport pump Replace the wg.Add(6) + six "go func() { defer wg.Done(); fn() }()" calls with sync.WaitGroup.Go (Go 1.25+, the project's minimum version). Same number of goroutines; less boilerplate. * caddyhttp: drop defensive checks in UnwrapResponseWriterAs Per review, the nil-pointer guard and the next == w self-reference guard defended against shapes that no in-tree ResponseWriter wrapper produces. Drop both, simplifying the loop. Also remove the self-reference test that exercised the now-removed guard. * reverseproxy: use r.URL.RequestURI() to build the WebTransport upstream URL Replace buildWebTransportUpstreamURL with an inline call to r.URL.RequestURI() at the single call site. RequestURI handles the empty-path → "/" fixup and query suffixing already, and uses EscapedPath so encoded path segments (e.g. %2F) are preserved on the upstream — fixing a small correctness gap where the previous helper read r.URL.Path (decoded). Aligns with how Caddy's normal proxy path lets stdlib render the outbound request line via URL.RequestURI semantics. * reverseproxy: use wg.Go in spliceBidi * reverseproxy: expand SNI placeholders on the WebTransport upstream dial The normal HTTP/3 path expands placeholders in tls_server_name at dial time via a custom h3Transport.Dial hook (#7737). The WebTransport path dials through its own webtransport.Dialer and bypasses that hook, so a placeholder like {http.request.host} reached the QUIC ClientHello literally. Expand it in webTransportHijack before dialing, cloning the shared TLS config so concurrent sessions are unaffected. Adds an integration test asserting the upstream observes the resolved SNI rather than the literal "{...}" string. * reverseproxy: bump webtransport-go to v0.11.0 (WebTransport draft-15) webtransport-go v0.11.0 implements draft-15, which renames the Extended CONNECT :protocol token from "webtransport" to "webtransport-h3" (the server keeps accepting the legacy token for older clients). Detect both tokens in the reverse-proxy upgrade check and in the test echo handler. Pulls quic-go v0.59.1 -> v0.60.0, required by webtransport-go v0.11.0; its only breaking change is requiring Go 1.25 (already met), with no API changes affecting Caddy core. Also picks up the v0.11.0 fix for a stream-close race on concurrent session close, relevant to the long-lived pump. * caddyhttp: log WebTransport QUIC connection serve errors * caddy: gate QUIC WebTransport capabilities behind an explicit ListenQUIC parameter * caddyhttp: fall back to plain HTTP/3 when a reused QUIC listener lacks WebTransport * caddyhttp: enable WebTransport via a webtransport config object Replace the enable_webtransport bool with a presence-gated WebTransportConfig object so later options can be added without renaming the key. JSON is "webtransport": {}, Caddyfile is webtransport (optional empty block). * reverseproxy: use origReq for the client WebTransport upgrade Upgrade is the client-facing session: CheckOrigin and the CONNECT shape must come from the downstream request. The upstream-directed clone remains used for the dial URL, header_up, and SNI expansion. * reverseproxy: relay WebTransport application protocol negotiation Parse WT-Available-Protocols from the downstream request, set it on the upstream Dialer, and copy WT-Protocol from the upstream 200 onto the client response. The shared webtransport.Server does not select a protocol of its own, so MOQ/MediaMTX clients receive the upstream's choice instead of WT_ALPN_ERROR. * caddytest: skip origin check on standalone WebTransport backends header_up Host rewrites used by proxy tests would 400 the upstream Upgrade via CheckOrigin. Test backends are not browsers. * caddytest: cover MediaMTX Host-rewrite + moqt-19 through the WT proxy Both of elee's #7669 reports fire together: header_up rewrites Host (Upgrade must use origReq) and the client offers moqt-19 (the proxy must relay WT-Available-Protocols / WT-Protocol). A bidi echo after negotiation proves the session is usable, not just headers. * caddyhttp: break self-referential Unwrap loops A wrapper whose Unwrap returns itself would spin forever. Restore the cycle-break mholt applied on #7669, without bringing back the nil-pointer guard Weidi asked to drop. * caddyhttp: close WebTransport sessions before HTTP/3 shutdown ServeQUICConn waits for active WT sessions, and with the default eternal grace period HTTP/3 Shutdown would hang until every client disconnected. Close the WebTransport server first so stop/reload can drain. * caddyhttp: serve plain HTTP/3 when a reused listener lacks WebTransport A pooled QUIC listener created without DATAGRAMs cannot upgrade WT sessions. Falling back on the WT-configured http3.Server still advertised SETTINGS the socket cannot honor. Build a separate plain HTTP/3 server for that listener and assert it omits WT SETTINGS. * caddyhttp: visit Unwrap writers by pointer identity Comparing writers with == panics on uncomparable types and only catches a one-node cycle. Track pointer identity, bound the walk, and add tests for an uncomparable self-unwrap and an A→B→A cycle. RecordHijackedStatus lets Upgrade write through the naked HTTP/3 writer while the access-log recorder still observes the 200. * reverseproxy: honor prepared Host, header_down, and WT bookkeeping Dialer.Dial always sets Host from the URL, so header_up Host never reached the upstream. Dial the real address via DialAddr and put the prepared Host on the CONNECT. Parse WT-Available-Protocols from the prepared request so header_up applies to negotiation. Copy filtered upstream response headers before header_down, record the hijacked 200 for access logs, and run the same latency / circuit-breaker / passive-health path as a normal round-trip. * caddytest: cover WT reload, access-log status, Host, and header ops Reload with an eternal grace period must close an in-flight session. Access logs must record 200 after Upgrade. header_up Host and WT-Available-Protocols must reach the upstream; header_down and non-hop upstream headers must reach the client. * reverseproxy: build WebTransport against quic-go v0.63 webtransport-go v0.13.0 renames Dialer to Transport. v0.11.1 does not compile against quic-go v0.63. * reverseproxy: close the upstream body when a WebTransport dial fails Dial can return a response with an error when the peer rejects the CONNECT. Close that body before the error is returned. * caddyhttp: close WebTransport sessions without closing the HTTP/3 server webtransport.Server.Close also closes the shared HTTP/3 server and aborts every QUIC connection before the grace period applies. Close the registered sessions, then let HTTP/3 shutdown drain ordinary requests. * chore: modernise Signed-off-by: Zen Dodd <mail@steadytao.com> * reverseproxy: preserve WT-Available-Protocols across upstream retries The proxy loop reuses the request and only recopies headers when header ops are configured. Deleting the offer on that map hid it from the next upstream. * caddyhttp: close WebTransport sessions when the grace period ends Eternal grace leaves an active session until the peer ends it. A configured grace period closes it at the deadline, while the socket is still up, so the close capsule can be delivered. * reverseproxy: use terminalError when refusing an Incremental response The proxy-loop collapse renamed roundtripSucceededError to terminalError. Master's Incremental refusal still returned the old type, so the rebased package did not compile. * caddyhttp: don't let idle HTTP/3 conns pin WebTransport shutdown - Count in-flight requests on ServeQUICConn connections and close idle ones when shutdown starts - Delay that close by a second so a just-finished response can be read - Retain plain HTTP/3 fallbacks for listeners without datagrams and shut them down with the same grace period * reverseproxy: wait for WebTransport stream workers - Join bidi and uni splice workers after the accept loops return - Cancel the opposite copy on a hard splice error --------- Signed-off-by: Zen Dodd <mail@steadytao.com> Co-authored-by: tomholford <tomholford@users.noreply.github.com> Co-authored-by: Zen Dodd <mail@steadytao.com>