@marcel91200 / caddy Mirror · read-only / commits / 4f45ded7
Fork 0

Fast and extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS

Mirrored from https://github.com/Slazee/caddy.git

Synced Nov 11, 03:30 UTC

caddyconfig: unify Caddyfile lexer/parser/formatter (token-based formatter) (#7880)

* docs: add Caddyfile lexer/parser/formatter unification design spec Design for unifying the Caddyfile lexer/parser and the standalone formatter by making the token pipeline drive formatting: raw-source-span capture, comment tokens, separator-kind metadata, follow-imports and (default-off) braced-wrap modes. Includes adversarial-review hardening and empirically-found intentional improvements over the legacy formatter. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: add phased implementation plan for Caddyfile formatter unification Task-by-task TDD plan: Phase 1 (token model + formatter rewrite + fuzz invariants + legacy oracle), Phase 2 (follow-imports mode + caddy fmt --imports), Phase 3 (default-off braced-wrap). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: vendor legacy Caddyfile formatter as differential oracle Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: add format-mode Token fields and accessors Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: add Lex and LexOptions; Tokenize delegates to Lex Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: capture verbatim raw source spans in format mode Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: emit comment tokens in format mode Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: record token separator kind in format mode Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: split structural braces in format mode Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: rewrite Format as a token-based renderer (core layout) Replace the rune-by-rune state machine with a renderer that consumes the format-mode token stream (Lex with Comments+Raw). Adds FormatWithOptions and FormatOptions{WrapUnbracedSite}; Format keeps its signature. Comment placement, continuations, empty-block expansion, and the import/}-glue rules land in Tasks 9-11; the six dependent TestFormatter cases are temporarily skipped with annotations. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: token renderer comments (I1/I3) and continuations (D1) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: token renderer empty blocks (I2) and post-} break (I4) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: import-brace exception, drop nesting cap, drop < quirk Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: fix formatter idempotency/semantic bugs, retire legacy oracle Fix two fuzz-found bugs plus several related idempotency issues in the token-based Caddyfile formatter, and retire the legacy byte-scanner formatter used as a test oracle. Bug B (formatter.go): a stray "}" at nesting 0 was glued to an unquoted predecessor because precededBySpace is unreliable after an unquoted token. Only suppress the separating space for an inline close brace when the previous token is quoted (same guard as comment gluing), so "0 }" no longer merges into "0}". Bug A (lexer.go): an empty-bodied heredoc hitting EOF with no closing marker was silently dropped, breaking Format idempotency. Surface it as an "incomplete heredoc" error ONLY in format mode (opts.Comments || opts.Raw); Tokenize/Parse keep the exact prior silent-drop behavior. Format then falls back to preserving the trimmed input. Additional format-mode-only fixes surfaced by property fuzzing: correct line-break accounting for empty-bodied heredocs and continuation-framed quoted tokens; normalize the raw slice / clear continuation on split and standalone structural braces; guard the address/comment/brace fold so a next-line "{" is not folded onto a comment trailing an opening brace; and a fallback + idempotency backstop in FormatWithOptions for degenerate tokens (ambiguous heredoc openers, unterminated/escaped quotes, dangling escapes, messy/leading line continuations). The parse path is unchanged. Property fuzz targets (formatter_test.go): keep FuzzFormatIdempotent, FuzzFormatNoPanic, and a redesigned FuzzFormatSemanticPreserve that asserts format-mode token-text preservation (robust to the parser's lenient grouping of ambiguous brace layouts). Add regression tests TestFormatStrayCloseBraceKeepsTokens and TestFormatIncompleteHeredocIdempotent. Remove FuzzFormatParity, containsParityExclusion, maxNesting and delete formatter_legacy_test.go (legacyFormat oracle + TestLegacyOracleMatchesCurrent). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: seed corpus and FormattingDifference stability tests Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: extract pure import-glob resolution helper Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: import discovery for follow-imports formatting Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: FormatImports formats root plus imported files Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * cmd: add 'caddy fmt --imports' to format imported files Adds a --imports flag to 'caddy fmt' that calls caddyfile.FormatImports to format the root Caddyfile and all files it references via import directives. Rejects --imports with stdin (no source directory to resolve against). With --overwrite each file is written back; otherwise each file's output is printed to stdout with a '# <path>' header, with optional per-file diff via --diff. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: detect single unbraced site block Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: implement default-off unbraced->braced site wrapping Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: make WrapUnbracedSite decision on normalized stream (idempotency fix) FormatWithOptions wrapped before formatting, deciding on the raw token stream. isSingleUnbracedSite uses source-line gaps, but formatTokens removes blank lines and folds dangling braces, so a shape read as "not a single site" on pass 1 could read as a single unbraced site on pass 2 and get wrapped, breaking Format(Format(x)) == Format(x). Decide the wrap on the already-normalized stream instead: normalize, re-lex, and wrap only if isSingleUnbracedSite holds there. tokens becomes the wrapped stream so the backstop still compares like-for-like; once wrapped the output is braced, so no double-wrap on re-runs. Format (zero options) still never wraps. Adds a regression reproducer and FuzzWrapUnbracedSiteIdempotent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: satisfy golangci-lint (rangeint + De Morgan) Modernize three index loops to range-over-int and simplify a negated boolean per staticcheck QF1001. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: modernize remaining index loops (rangeint) Convert the remaining for-i index loops to range-over-int per golangci-lint modernize. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * caddyfile: fix formatter and import correctness issues (#7881) * caddyfile: fix formatter and import correctness issues * noot * noot * caddyfile: document parser-driven import discovery * test: make formatter import tests portable (#7882) * caddyfile: propagate import observer into named route blocks Named route blocks expand their imports through a separate parser built in expandImportsInBlock, which copied only definedSnippets and importGraph. The importObserver hook was left nil, so FormatImports never saw files imported from inside a &(name) { ... } block and "caddy fmt --imports" silently skipped them. Carry the observer into that parser and cover it with a discovery test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * caddyfile: detect end-of-input-swallowing tokens from token shape hasUnformattableToken documented an unterminated quote/backtick as a shape that cannot be rendered idempotently, but never implemented the check. The condition was only caught dynamically by trailingNewlineChangesTokens, which returns early for input that already ends in a newline. That early return is wrong for a token that swallowed the newline: once Format's mandatory trailing newline sits inside an unterminated quote, appending another one no longer changes the stream, so the second pass rendered what the first pass had preserved verbatim. Format(`"""`) => `"""` (fallback, preserved) Format(`"""` + newline) => `"" "` (rendered, space inserted) Decide it from the token instead: a non-quoted token whose verbatim source ends in a newline swallowed it, since a newline otherwise terminates a token. This also covers an escaped quote ("\"") running to end-of-input, which has the same shape. A lone "\r" is kept in raw but does not terminate a token, so it is deliberately excluded; heredoc tokens carry wasQuoted == '<'. Master's rune-based formatter was idempotent on these inputs, so this restores parity. Found by FuzzFormatIdempotent; the inputs are added as portable seeds and table cases rather than testdata/fuzz files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * caddyfile: stop Format from mutating its input buffer The three fallback paths returned append(bytes.TrimSpace(input), '\n'). TrimSpace returns a subslice sharing the caller's backing array, so the append wrote the newline into the caller's buffer. os.ReadFile returns a slice with one byte of spare capacity, which is exactly what the append needs, so this reached real callers: $ printf 'foo "unterminated \n\n' > Caddyfile $ caddy fmt --diff foo "unterminated <- reported unchanged; its trailing - whitespace was in fact stripped cmd/commandfuncs.go compares input against output after calling Format, and by then input had been overwritten to match. The same aliasing corrupted FuzzFormatIdempotent, which reported failures whose recorded inputs did not reproduce. Copy into a fresh buffer instead, and assert both non-mutation and non-aliasing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * caddyfile: preserve repeated import expansions during formatting Deduplicate formatted files without skipping imports with different arguments, block substitutions, or lexical aliases. Track active physical-file ancestry to prevent symlink cycles, including snippet and block expansion contexts. Add regression coverage and align the upstream glued-brace expectation with the token-based formatter's literal-token behavior. Validation: go test -race -short ./..., go build ./..., and golangci-lint run --timeout 10m all pass. AI assistance: OpenAI Codex (GPT-6) assisted with implementation and testing. * docs: remove formatter implementation plan and design spec * caddyfile: require whitespace before line continuations Preserve attached trailing backslashes as literal text, including Windows root paths. Require a space or tab before an unquoted continuation backslash, and preserve significant leading whitespace in formatter fallback output. Add lexer, formatter, CRLF, adapter, and fuzz regression coverage. Validated with race tests, build, lint, formatter fuzzing, and the 165-example forum corpus. AI assistance: OpenAI Codex (GPT-6) assisted with implementation, regression tests, and validation. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: a <a@tuxpa.in>

Francis Lavoie committed Oct 7, 2026 at 17:14 UTC 4f45ded75556f6645075ddb8724a6d79cc1e5870
13 files changed +3146 -394
caddyconfig/caddyfile/format_imports.go new
+209
@@ -0,0 +1,209 @@
1 +// Copyright 2015 Matthew Holt and The Caddy Authors
2 +//
3 +// Licensed under the Apache License, Version 2.0 (the "License");
4 +// you may not use this file except in compliance with the License.
5 +// You may obtain a copy of the License at
6 +//
7 +// http://www.apache.org/licenses/LICENSE-2.0
8 +//
9 +// Unless required by applicable law or agreed to in writing, software
10 +// distributed under the License is distributed on an "AS IS" BASIS,
11 +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 +// See the License for the specific language governing permissions and
13 +// limitations under the License.
14 +
15 +package caddyfile
16 +
17 +import (
18 + "fmt"
19 + "io"
20 + "os"
21 + "path/filepath"
22 +
23 + "github.com/caddyserver/caddy/v2"
24 +)
25 +
26 +type importedFile struct {
27 + path string
28 + content []byte
29 + info os.FileInfo
30 +}
31 +
32 +// discoverImportedFileContents returns files reached by imports which the parser
33 +// actually executes, in discovery order and excluding rootFile. Driving the real
34 +// parser is important here: import behavior depends on declaration order, snippet
35 +// expansion, import arguments, block substitutions, environment expansion, and
36 +// whether an import appears in active configuration. Reimplementing those rules
37 +// in a separate token scanner caused FormatImports to disagree with adaptation.
38 +//
39 +// The parser calls importObserver after opening and reading each imported file,
40 +// but before inserting its tokens. The observer retains those exact bytes for
41 +// formatting so the file is not reopened later. Each physical file is retained
42 +// once, but repeated imports still expand with their own arguments. Only imports
43 +// of an active physical ancestor are skipped, preventing symlink cycles.
44 +// os.FileInfo values are retained to verify that each canonical pathname still
45 +// identifies the file whose contents were parsed.
46 +func discoverImportedFileContents(rootFile string, rootInput []byte, rootInfo os.FileInfo) ([]importedFile, error) {
47 + // Compare file identity rather than path spelling: lexical paths do not catch
48 + // imports such as dir/link/dir/link/Caddyfile through a symlink cycle.
49 + seen := []os.FileInfo{rootInfo}
50 + var files []importedFile
51 +
52 + // allTokens performs environment replacement in-place, so preserve the bytes
53 + // which FormatImports will subsequently pass to the formatter.
54 + inputCopy := append([]byte(nil), rootInput...)
55 + tokens, err := allTokens(rootFile, inputCopy)
56 + if err != nil {
57 + return nil, err
58 + }
59 + rootAncestry := &importAncestor{info: rootInfo}
60 + for i := range tokens {
61 + tokens[i].importAncestry = rootAncestry
62 + }
63 + // Construct the parser exactly as parse() does. The import graph remains
64 + // responsible for logical snippet/import cycles; the observer below adds
65 + // physical-file identity handling for symlink aliases.
66 + p := parser{
67 + Dispenser: NewDispenser(tokens),
68 + importGraph: importGraph{
69 + nodes: make(map[string]struct{}),
70 + edges: make(adjacency),
71 + },
72 + }
73 + p.importObserver = func(path string, info os.FileInfo, content []byte, ancestry *importAncestor) (bool, error) {
74 + for ancestor := ancestry; ancestor != nil; ancestor = ancestor.parent {
75 + if os.SameFile(ancestor.info, info) {
76 + return true, nil
77 + }
78 + }
79 + // Deduplicate output without suppressing argument-dependent descendants
80 + // or imports resolved relative to a different lexical alias.
81 + for _, previous := range seen {
82 + if os.SameFile(previous, info) {
83 + return false, nil
84 + }
85 + }
86 +
87 + // Store one stable, absolute spelling for callers, but verify it still
88 + // resolves to the descriptor which supplied content before retaining it.
89 + canonical, err := canonicalPath(path)
90 + if err != nil {
91 + return false, err
92 + }
93 + currentInfo, err := os.Stat(canonical)
94 + if err != nil {
95 + return false, err
96 + }
97 + if !os.SameFile(info, currentInfo) {
98 + return false, fmt.Errorf("imported file changed while being read")
99 + }
100 +
101 + seen = append(seen, info)
102 + files = append(files, importedFile{
103 + path: canonical,
104 + // The parser owns its read buffer; keep an independent copy for the
105 + // formatting pass and avoid a second pathname-based read.
106 + content: append([]byte(nil), content...),
107 + info: info,
108 + })
109 + return false, nil
110 + }
111 +
112 + if _, err := p.parseAll(); err != nil {
113 + return nil, err
114 + }
115 + return files, nil
116 +}
117 +
118 +// canonicalPath returns an absolute path with symlinks resolved. Canonical paths
119 +// make the FormattedFile API consistent for the root and every imported file;
120 +// physical deduplication still uses os.SameFile because spelling alone is not a
121 +// portable file-identity test.
122 +func canonicalPath(path string) (string, error) {
123 + abs, err := caddy.FastAbs(path)
124 + if err != nil {
125 + return "", err
126 + }
127 + return filepath.EvalSymlinks(abs)
128 +}
129 +
130 +// FormattedFile is a file path paired with its formatted contents.
131 +type FormattedFile struct {
132 + Path string
133 + Content []byte
134 +}
135 +
136 +// FormatImports reads filename, formats it (honoring opts), then discovers all
137 +// files reached by active import directives and formats each one independently
138 +// at nesting-0 baseline. WrapUnbracedSite is always disabled for imported files
139 +// because they are fragments, not standalone Caddyfiles.
140 +//
141 +// All returned paths are absolute, symlink-resolved paths. The root file is
142 +// always first, followed by imported files in parser discovery order.
143 +func FormatImports(filename string, opts FormatOptions) ([]FormattedFile, error) {
144 + // Keep the lexical absolute path for parser-relative import resolution, while
145 + // using the canonical path for opening and returning the root file. This
146 + // matches adaptation when the supplied Caddyfile itself is reached by symlink.
147 + rootLexicalPath, err := caddy.FastAbs(filename)
148 + if err != nil {
149 + return nil, err
150 + }
151 + rootPath, err := canonicalPath(filename)
152 + if err != nil {
153 + return nil, err
154 + }
155 + // Reading and statting one open descriptor binds rootBytes and rootInfo to the
156 + // same physical file. Separate ReadFile and Stat calls would permit a pathname
157 + // replacement between the two operations.
158 + root, err := os.Open(rootPath)
159 + if err != nil {
160 + return nil, err
161 + }
162 + rootInfo, err := root.Stat()
163 + if err != nil {
164 + root.Close()
165 + return nil, err
166 + }
167 + rootBytes, err := io.ReadAll(root)
168 + closeErr := root.Close()
169 + if err != nil {
170 + return nil, err
171 + }
172 + if closeErr != nil {
173 + return nil, closeErr
174 + }
175 +
176 + discovered, err := discoverImportedFileContents(rootLexicalPath, rootBytes, rootInfo)
177 + if err != nil {
178 + return nil, err
179 + }
180 +
181 + results := make([]FormattedFile, 0, 1+len(discovered))
182 + results = append(results, FormattedFile{Path: rootPath, Content: FormatWithOptions(rootBytes, opts)})
183 + for _, file := range discovered {
184 + results = append(results, FormattedFile{
185 + Path: file.path,
186 + Content: FormatWithOptions(file.content, FormatOptions{}),
187 + })
188 + }
189 +
190 + // Formatting can take long enough for a pathname to be replaced. Before
191 + // returning writable paths to the caller, ensure every one still identifies
192 + // the physical file whose bytes were parsed and formatted. The command layer
193 + // performs another check immediately before atomic replacement.
194 + for i, result := range results {
195 + currentInfo, err := os.Stat(result.Path)
196 + if err != nil {
197 + return nil, err
198 + }
199 + originalInfo := rootInfo
200 + if i > 0 {
201 + originalInfo = discovered[i-1].info
202 + }
203 + if !os.SameFile(originalInfo, currentInfo) {
204 + return nil, fmt.Errorf("file changed while formatting: %s", result.Path)
205 + }
206 + }
207 +
208 + return results, nil
209 +}
caddyconfig/caddyfile/format_imports_test.go new
+485
@@ -0,0 +1,485 @@
1 +// Copyright 2015 Matthew Holt and The Caddy Authors
2 +//
3 +// Licensed under the Apache License, Version 2.0 (the "License");
4 +// you may not use this file except in compliance with the License.
5 +// You may obtain a copy of the License at
6 +//
7 +// http://www.apache.org/licenses/LICENSE-2.0
8 +//
9 +// Unless required by applicable law or agreed to in writing, software
10 +// distributed under the License is distributed on an "AS IS" BASIS,
11 +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 +// See the License for the specific language governing permissions and
13 +// limitations under the License.
14 +
15 +package caddyfile
16 +
17 +import (
18 + "os"
19 + "path/filepath"
20 + "runtime"
21 + "strings"
22 + "testing"
23 +)
24 +
25 +func writeFile(t *testing.T, path, content string) {
26 + t.Helper()
27 + if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
28 + t.Fatal(err)
29 + }
30 +}
31 +
32 +func discoverImportedFiles(rootFile string, rootInput []byte) ([]string, error) {
33 + rootPath, err := canonicalPath(rootFile)
34 + if err != nil {
35 + return nil, err
36 + }
37 + rootInfo, err := os.Stat(rootPath)
38 + if err != nil {
39 + return nil, err
40 + }
41 + files, err := discoverImportedFileContents(rootFile, rootInput, rootInfo)
42 + if err != nil {
43 + return nil, err
44 + }
45 + paths := make([]string, len(files))
46 + for i, file := range files {
47 + paths[i] = file.path
48 + }
49 + return paths, nil
50 +}
51 +
52 +func TestDiscoverImportedFiles(t *testing.T) {
53 + dir := t.TempDir()
54 + root := filepath.Join(dir, "Caddyfile")
55 + writeFile(t, root, "import sites/*.caddy\nimport mysnip\n")
56 + os.MkdirAll(filepath.Join(dir, "sites"), 0o755)
57 + writeFile(t, filepath.Join(dir, "sites", "a.caddy"), "(mysnip) {\n\trespond 200\n}\nlocalhost {\n\timport mysnip\n}\n")
58 + rootInput, _ := os.ReadFile(root)
59 + files, err := discoverImportedFiles(root, rootInput)
60 + if err != nil {
61 + t.Fatal(err)
62 + }
63 + // sites/a.caddy is discovered; "mysnip" is a snippet (defined in a.caddy), not a file
64 + wantPath, err := canonicalPath(filepath.Join(dir, "sites", "a.caddy"))
65 + if err != nil {
66 + t.Fatal(err)
67 + }
68 + want := []string{wantPath}
69 + if len(files) != 1 || files[0] != want[0] {
70 + t.Errorf("got %v, want %v", files, want)
71 + }
72 +}
73 +
74 +func TestFormatImports(t *testing.T) {
75 + dir := t.TempDir()
76 + root := filepath.Join(dir, "Caddyfile")
77 + writeFile(t, root, "import sites/a.caddy\n")
78 + os.MkdirAll(filepath.Join(dir, "sites"), 0o755)
79 + // deliberately messy imported file to prove it gets formatted at baseline 0
80 + writeFile(t, filepath.Join(dir, "sites", "a.caddy"), "localhost {\nrespond 200\n}\n")
81 + results, err := FormatImports(root, FormatOptions{})
82 + if err != nil {
83 + t.Fatal(err)
84 + }
85 + if len(results) != 2 {
86 + t.Fatalf("got %d results, want 2", len(results))
87 + }
88 + // results[0] is the root; results[1] is the imported file, formatted
89 + var imported *FormattedFile
90 + for i := range results {
91 + if strings.HasSuffix(results[i].Path, "a.caddy") {
92 + imported = &results[i]
93 + }
94 + }
95 + if imported == nil {
96 + t.Fatal("imported file not in results")
97 + }
98 + want := "localhost {\n\trespond 200\n}\n"
99 + if string(imported.Content) != want {
100 + t.Errorf("imported formatted = %q, want %q", imported.Content, want)
101 + }
102 +}
103 +
104 +func TestDiscoverIgnoresNonDirectiveImport(t *testing.T) {
105 + dir := t.TempDir()
106 + root := filepath.Join(dir, "Caddyfile")
107 + writeFile(t, root, "localhost {\n\tbasic_auth / import password\n}\n")
108 + rootInput, _ := os.ReadFile(root)
109 + files, err := discoverImportedFiles(root, rootInput)
110 + if err != nil {
111 + t.Fatal(err)
112 + }
113 + if len(files) != 0 {
114 + t.Errorf("got %v, want none ('import' here is an argument)", files)
115 + }
116 +}
117 +
118 +func TestDiscoverImportBeforeSnippetDeclarationIsFile(t *testing.T) {
119 + dir := t.TempDir()
120 + root := filepath.Join(dir, "Caddyfile")
121 + writeFile(t, root, "import foo\n\n(foo) {\n\trespond ok\n}\n")
122 + writeFile(t, filepath.Join(dir, "foo"), "localhost {\n\trespond from-file\n}\n")
123 +
124 + rootInput, err := os.ReadFile(root)
125 + if err != nil {
126 + t.Fatal(err)
127 + }
128 + files, err := discoverImportedFiles(root, rootInput)
129 + if err != nil {
130 + t.Fatal(err)
131 + }
132 + if len(files) != 1 || files[0] != mustCanonicalPath(t, filepath.Join(dir, "foo")) {
133 + t.Fatalf("got %v, want foo file", files)
134 + }
135 +}
136 +
137 +func TestDiscoverImportInsideNamedRoute(t *testing.T) {
138 + dir := t.TempDir()
139 + root := filepath.Join(dir, "Caddyfile")
140 + writeFile(t, root, "&(foo) {\n\timport handler\n}\n\nlocalhost {\n\tinvoke foo\n}\n")
141 + writeFile(t, filepath.Join(dir, "handler"), "respond \"hi\"\n")
142 +
143 + rootInput, err := os.ReadFile(root)
144 + if err != nil {
145 + t.Fatal(err)
146 + }
147 + files, err := discoverImportedFiles(root, rootInput)
148 + if err != nil {
149 + t.Fatal(err)
150 + }
151 + // Named route blocks expand their imports through a separate parser; it must
152 + // still report to the observer or --imports silently skips the file.
153 + if len(files) != 1 || files[0] != mustCanonicalPath(t, filepath.Join(dir, "handler")) {
154 + t.Fatalf("got %v, want handler file", files)
155 + }
156 +}
157 +
158 +func TestDiscoverRecursiveImportArgs(t *testing.T) {
159 + dir := t.TempDir()
160 + root := filepath.Join(dir, "Caddyfile")
161 + selector := filepath.Join(dir, "selector.caddy")
162 + child := filepath.Join(dir, "child.caddy")
163 + writeFile(t, root, "import selector.caddy child.caddy\n")
164 + writeFile(t, selector, "import {args[0]}\n")
165 + writeFile(t, child, "localhost {\n\trespond ok\n}\n")
166 +
167 + results, err := FormatImports(root, FormatOptions{})
168 + if err != nil {
169 + t.Fatal(err)
170 + }
171 + want := []string{mustCanonicalPath(t, root), mustCanonicalPath(t, selector), mustCanonicalPath(t, child)}
172 + if len(results) != len(want) {
173 + t.Fatalf("got %d results, want %d: %v", len(results), len(want), results)
174 + }
175 + for i := range want {
176 + if results[i].Path != want[i] {
177 + t.Errorf("result %d path = %q, want %q", i, results[i].Path, want[i])
178 + }
179 + }
180 +}
181 +
182 +func TestDiscoverRepeatedImportArgs(t *testing.T) {
183 + for _, tc := range []struct {
184 + name string
185 + root string
186 + }{
187 + {"file", "import selector.caddy a.caddy\nimport selector.caddy b.caddy\n"},
188 + {"snippet", "(select) {\nimport selector.caddy {args[0]}\n}\nimport select a.caddy\nimport select b.caddy\n"},
189 + } {
190 + t.Run(tc.name, func(t *testing.T) {
191 + dir := t.TempDir()
192 + root := filepath.Join(dir, "Caddyfile")
193 + selector := filepath.Join(dir, "selector.caddy")
194 + a := filepath.Join(dir, "a.caddy")
195 + b := filepath.Join(dir, "b.caddy")
196 + writeFile(t, root, tc.root)
197 + writeFile(t, selector, "import {args[0]}\n")
198 + writeFile(t, a, "a.local {\nrespond a\n}\n")
199 + writeFile(t, b, "b.local {\nrespond b\n}\n")
200 +
201 + blocks, err := Parse(root, []byte(tc.root))
202 + if err != nil || len(blocks) != 2 {
203 + t.Fatalf("expected two valid server blocks, got %d: %v", len(blocks), err)
204 + }
205 + results, err := FormatImports(root, FormatOptions{})
206 + if err != nil {
207 + t.Fatal(err)
208 + }
209 + want := []string{root, selector, a, b}
210 + if len(results) != len(want) {
211 + t.Fatalf("got paths %v, want %v", resultPaths(results), want)
212 + }
213 + for i, path := range want {
214 + if results[i].Path != mustCanonicalPath(t, path) {
215 + t.Errorf("result %d path = %q, want %q", i, results[i].Path, path)
216 + }
217 + }
218 + if want := "b.local {\n\trespond b\n}\n"; string(results[3].Content) != want {
219 + t.Errorf("second descendant content = %q, want %q", results[3].Content, want)
220 + }
221 + })
222 + }
223 +}
224 +
225 +func TestDiscoverRepeatedSymlinkImportsResolveRelativePaths(t *testing.T) {
226 + dir := t.TempDir()
227 + root := filepath.Join(dir, "Caddyfile")
228 + selector := filepath.Join(dir, "selector.caddy")
229 + writeFile(t, root, "import a/selector.caddy\nimport b/selector.caddy\n")
230 + writeFile(t, selector, "import child.caddy\n")
231 + for _, name := range []string{"a", "b"} {
232 + subdir := filepath.Join(dir, name)
233 + if err := os.Mkdir(subdir, 0o755); err != nil {
234 + t.Fatal(err)
235 + }
236 + if err := os.Symlink(selector, filepath.Join(subdir, "selector.caddy")); err != nil {
237 + t.Skipf("symlinks unavailable: %v", err)
238 + }
239 + writeFile(t, filepath.Join(subdir, "child.caddy"), name+".local {\nrespond 200\n}\n")
240 + }
241 + results, err := FormatImports(root, FormatOptions{})
242 + if err != nil {
243 + t.Fatal(err)
244 + }
245 + want := []string{root, selector, filepath.Join(dir, "a", "child.caddy"), filepath.Join(dir, "b", "child.caddy")}
246 + if len(results) != len(want) {
247 + t.Fatalf("got paths %v, want %v", resultPaths(results), want)
248 + }
249 + for i, path := range want {
250 + if results[i].Path != mustCanonicalPath(t, path) {
251 + t.Errorf("result %d path = %q, want %q", i, results[i].Path, path)
252 + }
253 + }
254 +}
255 +
256 +func TestDiscoverRepeatedImportBlocks(t *testing.T) {
257 + dir := t.TempDir()
258 + root := filepath.Join(dir, "Caddyfile")
259 + selector := filepath.Join(dir, "selector.caddy")
260 + input := "import selector.caddy {\nbody {\nimport a.caddy\n}\n}\n" +
261 + "import selector.caddy {\nbody {\nimport b.caddy\n}\n}\n"
262 + writeFile(t, root, input)
263 + writeFile(t, selector, "localhost {\n{blocks.body}\n}\n")
264 + writeFile(t, filepath.Join(dir, "a.caddy"), "respond a\n")
265 + writeFile(t, filepath.Join(dir, "b.caddy"), "respond b\n")
266 + if blocks, err := Parse(root, []byte(input)); err != nil || len(blocks) != 2 {
267 + t.Fatalf("expected two valid server blocks, got %d: %v", len(blocks), err)
268 + }
269 + results, err := FormatImports(root, FormatOptions{})
270 + if err != nil {
271 + t.Fatal(err)
272 + }
273 + want := []string{root, selector, filepath.Join(dir, "a.caddy"), filepath.Join(dir, "b.caddy")}
274 + if len(results) != len(want) {
275 + t.Fatalf("got paths %v, want %v", resultPaths(results), want)
276 + }
277 + for i, path := range want {
278 + if results[i].Path != mustCanonicalPath(t, path) {
279 + t.Errorf("result %d path = %q, want %q", i, results[i].Path, path)
280 + }
281 + }
282 +}
283 +
284 +func TestDiscoverSkipsActiveSymlinkAncestor(t *testing.T) {
285 + dir := t.TempDir()
286 + root := filepath.Join(dir, "Caddyfile")
287 + child := filepath.Join(dir, "child.caddy")
288 + writeFile(t, root, "import child.caddy\n")
289 + writeFile(t, child, "import dir-link/child.caddy\nlocalhost {\nrespond ok\n}\n")
290 + if err := os.Symlink(dir, filepath.Join(dir, "dir-link")); err != nil {
291 + t.Skipf("symlinks unavailable: %v", err)
292 + }
293 + results, err := FormatImports(root, FormatOptions{})
294 + if err != nil {
295 + t.Fatal(err)
296 + }
297 + if len(results) != 2 || results[1].Path != mustCanonicalPath(t, child) {
298 + t.Fatalf("got paths %v, want root and one physical child", resultPaths(results))
299 + }
300 +}
301 +
302 +func TestDiscoverDeduplicatesSymlinksAndCycles(t *testing.T) {
303 + if runtime.GOOS == "windows" {
304 + t.Skip("symlink creation is not generally available on Windows")
305 + }
306 + dir := t.TempDir()
307 + root := filepath.Join(dir, "Caddyfile")
308 + child := filepath.Join(dir, "child.caddy")
309 + writeFile(t, root, "import child.caddy\nimport child-link.caddy\nimport dir-link/Caddyfile\n")
310 + writeFile(t, child, "localhost {\n\trespond ok\n}\n")
311 + if err := os.Symlink(child, filepath.Join(dir, "child-link.caddy")); err != nil {
312 + t.Fatal(err)
313 + }
314 + if err := os.Symlink(dir, filepath.Join(dir, "dir-link")); err != nil {
315 + t.Fatal(err)
316 + }
317 +
318 + results, err := FormatImports(root, FormatOptions{})
319 + if err != nil {
320 + t.Fatal(err)
321 + }
322 + if len(results) != 2 {
323 + t.Fatalf("got paths %v, want root and one physical child", resultPaths(results))
324 + }
325 + if results[1].Path != mustCanonicalPath(t, child) {
326 + t.Fatalf("imported path = %q, want %q", results[1].Path, child)
327 + }
328 +}
329 +
330 +func TestDiscoverExpandsEnvironmentBeforeLexing(t *testing.T) {
331 + dir := t.TempDir()
332 + root := filepath.Join(dir, "Caddyfile")
333 + child := filepath.Join(dir, "child # file.caddy")
334 + t.Setenv("CADDYFILE_IMPORT_LINE", `"import" "child # file.caddy" # generated comment`)
335 + writeFile(t, root, "{$CADDYFILE_IMPORT_LINE}\n")
336 + writeFile(t, child, "localhost {\n\trespond ok\n}\n")
337 +
338 + results, err := FormatImports(root, FormatOptions{})
339 + if err != nil {
340 + t.Fatal(err)
341 + }
342 + if len(results) != 2 || results[1].Path != mustCanonicalPath(t, child) {
343 + t.Fatalf("got paths %v, want generated quoted import %q", resultPaths(results), child)
344 + }
345 +}
346 +
347 +func TestDiscoverIgnoresImportsInUnusedSnippetAndImportBlock(t *testing.T) {
348 + dir := t.TempDir()
349 + root := filepath.Join(dir, "Caddyfile")
350 + template := filepath.Join(dir, "template.caddy")
351 + dead := filepath.Join(dir, "dead.caddy")
352 + writeFile(t, root, `(unused) {
353 + import dead.caddy
354 +}
355 +
356 +import template.caddy {
357 + unused {
358 + import dead.caddy
359 + }
360 +}
361 +`)
362 + writeFile(t, template, "localhost {\n\trespond ok\n}\n")
363 + writeFile(t, dead, "dead.example {\n\trespond dead\n}\n")
364 +
365 + results, err := FormatImports(root, FormatOptions{})
366 + if err != nil {
367 + t.Fatal(err)
368 + }
369 + if len(results) != 2 || results[1].Path != mustCanonicalPath(t, template) {
370 + t.Fatalf("got paths %v, want only active template import", resultPaths(results))
371 + }
372 +}
373 +
374 +func TestDiscoverQuotedImportDirective(t *testing.T) {
375 + dir := t.TempDir()
376 + root := filepath.Join(dir, "Caddyfile")
377 + child := filepath.Join(dir, "child.caddy")
378 + writeFile(t, root, `"import" child.caddy`+"\n")
379 + writeFile(t, child, "localhost {\n\trespond ok\n}\n")
380 +
381 + results, err := FormatImports(root, FormatOptions{})
382 + if err != nil {
383 + t.Fatal(err)
384 + }
385 + if len(results) != 2 || results[1].Path != mustCanonicalPath(t, child) {
386 + t.Fatalf("got paths %v, want quoted import", resultPaths(results))
387 + }
388 +}
389 +
390 +func TestDiscoverUnreadableImportReturnsError(t *testing.T) {
391 + if runtime.GOOS == "windows" {
392 + t.Skip("Unix permission mode test")
393 + }
394 + dir := t.TempDir()
395 + root := filepath.Join(dir, "Caddyfile")
396 + child := filepath.Join(dir, "child.caddy")
397 + writeFile(t, root, "import child.caddy\n")
398 + writeFile(t, child, "localhost\n")
399 + if err := os.Chmod(child, 0); err != nil {
400 + t.Fatal(err)
401 + }
402 + t.Cleanup(func() { _ = os.Chmod(child, 0o600) })
403 + if file, err := os.Open(child); err == nil {
404 + file.Close()
405 + t.Skip("test process can read mode-000 files")
406 + }
407 +
408 + if _, err := FormatImports(root, FormatOptions{}); err == nil {
409 + t.Fatal("expected unreadable matched import to return an error")
410 + }
411 +}
412 +
413 +func TestFormatImportsNormalizesAllPaths(t *testing.T) {
414 + dir := t.TempDir()
415 + realDir := filepath.Join(dir, "real")
416 + linkDir := filepath.Join(dir, "link")
417 + if err := os.MkdirAll(realDir, 0o755); err != nil {
418 + t.Fatal(err)
419 + }
420 + if err := os.MkdirAll(linkDir, 0o755); err != nil {
421 + t.Fatal(err)
422 + }
423 + realRoot := filepath.Join(realDir, "Caddyfile")
424 + rootLink := filepath.Join(linkDir, "Caddyfile")
425 + child := filepath.Join(linkDir, "child.caddy")
426 + writeFile(t, realRoot, "import child.caddy\n")
427 + writeFile(t, child, "localhost {\n\trespond ok\n}\n")
428 + if err := os.Symlink(realRoot, rootLink); err != nil {
429 + if runtime.GOOS == "windows" {
430 + t.Skipf("cannot create symlink: %v", err)
431 + }
432 + t.Fatal(err)
433 + }
434 +
435 + results, err := FormatImports(rootLink, FormatOptions{})
436 + if err != nil {
437 + t.Fatal(err)
438 + }
439 + want := []string{mustCanonicalPath(t, realRoot), mustCanonicalPath(t, child)}
440 + if got := resultPaths(results); len(got) != len(want) || got[0] != want[0] || got[1] != want[1] {
441 + t.Fatalf("got paths %v, want canonical paths %v", got, want)
442 + }
443 +}
444 +
445 +func TestDiscoveryRetainsImportedContent(t *testing.T) {
446 + dir := t.TempDir()
447 + root := filepath.Join(dir, "Caddyfile")
448 + child := filepath.Join(dir, "child.caddy")
449 + writeFile(t, root, "import child.caddy\n")
450 + writeFile(t, child, "localhost {\nrespond old\n}\n")
451 + rootInput, err := os.ReadFile(root)
452 + if err != nil {
453 + t.Fatal(err)
454 + }
455 + rootInfo, err := os.Stat(root)
456 + if err != nil {
457 + t.Fatal(err)
458 + }
459 +
460 + files, err := discoverImportedFileContents(root, rootInput, rootInfo)
461 + if err != nil {
462 + t.Fatal(err)
463 + }
464 + writeFile(t, child, "localhost {\n\trespond replacement\n}\n")
465 + if len(files) != 1 || string(files[0].content) != "localhost {\nrespond old\n}\n" {
466 + t.Fatalf("discovery did not retain the bytes read by the parser: %+v", files)
467 + }
468 +}
469 +
470 +func resultPaths(results []FormattedFile) []string {
471 + paths := make([]string, len(results))
472 + for i := range results {
473 + paths[i] = results[i].Path
474 + }
475 + return paths
476 +}
477 +
478 +func mustCanonicalPath(t *testing.T, path string) string {
479 + t.Helper()
480 + canonical, err := canonicalPath(path)
481 + if err != nil {
482 + t.Fatal(err)
483 + }
484 + return canonical
485 +}
caddyconfig/caddyfile/formatter.go
+611 -292
@@ -16,372 +16,691 @@ package caddyfile
16
17 import (
18 "bytes"
19 - "io"
20 - "slices"
19 "strings"
20 "unicode"
21 )
22
25 -// Format formats the input Caddyfile to a standard, nice-looking
26 -// appearance. It works by reading each rune of the input and taking
27 -// control over all the bracing and whitespace that is written; otherwise,
28 -// words, comments, placeholders, and escaped characters are all treated
29 -// literally and written as they appear in the input.
23 +const maxIndent = 10
24 +
25 +// Format formats the input Caddyfile to a standard, nice-looking appearance.
26 +// It tokenizes the input in format mode and renders the token stream, taking
27 +// control over bracing, indentation, and whitespace; token bodies (words,
28 +// comments, placeholders, quoted/backtick strings, heredocs, and escaped
29 +// characters) are emitted verbatim.
30 func Format(input []byte) []byte {
31 - input = bytes.TrimSpace(input)
31 + return FormatWithOptions(input, FormatOptions{})
32 +}
33
33 - out := new(bytes.Buffer)
34 - rdr := bytes.NewReader(input)
34 +// FormatOptions configures optional formatting behavior.
35 +type FormatOptions struct {
36 + // WrapUnbracedSite, when true, wraps a single unbraced site block in braces
37 + // (Phase 3). Default false.
38 + WrapUnbracedSite bool
39 +}
40
36 - type heredocState int
41 +// FormatWithOptions formats the input Caddyfile with the given options.
42 +func FormatWithOptions(input []byte, opts FormatOptions) []byte {
43 + tokens, err := lexFormat(input)
44 + if err != nil {
45 + // On a lex error, fall back to the trimmed input with a trailing newline;
46 + // Format never panics (Invariant 3).
47 + return trimmedWithNewline(input)
48 + }
49 + // Some token shapes cannot be rendered without changing what they lex back
50 + // to, which would break idempotency. When one is present, preserve the
51 + // trimmed input verbatim instead of re-rendering; this is trivially
52 + // idempotent since a second pass hits the same condition.
53 + //
54 + // The mandatory trailing newline is the subtle case: if appending it changes
55 + // how the input tokenizes (an unterminated or escaped quote, or a dangling
56 + // escape, that swallows the newline into a token), then no rendering is a
57 + // fixed point. Detect that here so Format falls back to the trimmed input;
58 + // trimming removes the trailing newline again, so the fallback is stable.
59 + if hasUnformattableToken(tokens) || trailingNewlineChangesTokens(input) {
60 + return trimmedWithNewline(input)
61 + }
62 + parseTokens, parseErr := Tokenize(input, "")
63 + wrapped := false
64 + if opts.WrapUnbracedSite {
65 + // Decide whether to wrap on the NORMALIZED token stream, not the raw one.
66 + // isSingleUnbracedSite uses source-line gaps (a blank-line gap => ambiguous
67 + // multi-site => don't wrap), but formatTokens removes blank lines and folds
68 + // dangling braces, so a shape that reads as "not a single site" on the raw
69 + // stream can read as a single unbraced site on a second pass over the
70 + // already-formatted output. Deciding on the normalized stream makes the
71 + // decision stable across passes (idempotent). Once wrapped, the output is
72 + // braced, so isSingleUnbracedSite is false on any re-run and there is no
73 + // double-wrap.
74 + normalized := formatTokens(tokens)
75 + normToks, nerr := lexFormat(normalized)
76 + if nerr == nil && isSingleUnbracedSite(normToks) {
77 + // Prefer the original token positions when they are already
78 + // unambiguous. Normalization may insert top-level block spacing that
79 + // would become an unwanted interior blank line after wrapping.
80 + if isSingleUnbracedSite(tokens) {
81 + tokens = wrapUnbracedSite(tokens)
82 + } else {
83 + tokens = wrapUnbracedSite(normToks)
84 + }
85 + wrapped = true
86 + }
87 + }
88 + out := formatTokens(tokens)
89 +
90 + // Backstop: formatting must be idempotent and must not change the token
91 + // stream (only structural whitespace). Re-lex the output and require both
92 + // that its token texts match the input's and that re-rendering it reproduces
93 + // it exactly (a fixed point). If some pathological escape/whitespace
94 + // combination the checks above did not anticipate slips through — changing a
95 + // token or leaving the vertical spacing unstable — fall back to preserving
96 + // the trimmed input verbatim. Well-formed input is a fixed point on the first
97 + // pass, so this only affects degenerate input. A lex error on the output is
98 + // likewise treated as a divergence.
99 + reToks, rerr := lexFormat(out)
100 + outParseTokens, outParseErr := Tokenize(out, "")
101 + parseChanged := !wrapped && parseErr == nil && (outParseErr != nil || !sameTokenTexts(parseTokens, outParseTokens))
102 + if rerr != nil || parseChanged || !sameTokenTexts(tokens, reToks) ||
103 + !bytes.Equal(out, formatTokens(reToks)) {
104 + return trimmedWithNewline(input)
105 + }
106 + return out
107 +}
108
38 - const (
39 - heredocClosed heredocState = 0
40 - heredocOpening heredocState = 1
41 - heredocOpened heredocState = 2
42 - )
109 +// lexFormat captures comments and raw source while only normalizing braces the
110 +// parser already recognized as standalone structural tokens. Public Lex options
111 +// intentionally do not change ordinary token boundaries.
112 +func lexFormat(input []byte) ([]Token, error) {
113 + tokens, err := Lex(input, "", LexOptions{Comments: true, Raw: true})
114 + if err != nil {
115 + return nil, err
116 + }
117 + for i := range tokens {
118 + if isOpenCurlyBrace(tokens[i]) || isCloseCurlyBrace(tokens[i]) {
119 + tokens[i].raw = tokens[i].Text
120 + tokens[i].continuation = false
121 + }
122 + }
123 + return tokens, nil
124 +}
125
126 +// isSingleUnbracedSite reports whether the format-mode token stream is exactly
127 +// one top-level server block written WITHOUT wrapping braces — i.e. a single
128 +// site whose address line is followed directly by directives, with no outer
129 +// "{"..."}" around them. This is the shape the (default-off, Phase 3)
130 +// braced-wrap feature may wrap in braces, so detection must be conservative:
131 +// anything ambiguous, already-braced, or multi-site returns false.
132 +//
133 +// The analysis is purely structural (no filesystem, no import expansion, no env
134 +// substitution). It mirrors the parser's address-boundary logic
135 +// (parser.addresses, isSnippet, isNamedRoute in parse.go): the ADDRESS LIST is
136 +// the run of leading non-comment tokens up to the first newline, honoring
137 +// trailing-comma continuation across lines (a token ending in "," means another
138 +// address follows, possibly on the next line).
139 +//
140 +// It returns true iff:
141 +// - the address list is non-empty, and
142 +// - the first address token is not a snippet "(...)" or named route "&(...)",
143 +// and
144 +// - no structural "{" appears on the address line (which would make it an
145 +// already-braced site block, or, with no address before it, a
146 +// global-options block), and
147 +// - top-level directive groups are not separated by an ambiguous blank line.
148 +//
149 +// Interior directive braces (nesting >= 1, e.g. "reverse_proxy {"..."}") are
150 +// allowed and ignored. Comments are ignored for the boundary decision.
151 +func isSingleUnbracedSite(tokens []Token) bool {
152 + // Collect the leading address list: non-comment tokens up to the first
153 + // newline, extending across newlines while the previous address token ends
154 + // in a trailing comma (comma-continuation).
155 var (
45 - last rune // the last character that was written to the result
46 -
47 - space = true // whether current/previous character was whitespace (beginning of input counts as space)
48 - beginningOfLine = true // whether we are at beginning of line
49 -
50 - openBrace bool // whether current word/token is or started with open curly brace
51 - openBraceWritten bool // if openBrace, whether that brace was written or not
52 - openBraceSpace bool // whether there was a non-newline space before open brace
53 -
54 - newLines int // count of newlines consumed
55 -
56 - comment bool // whether we're in a comment
57 - quotes string // encountered quotes ('', '`', '"', '"`', '`"')
58 - escaped bool // whether current char is escaped
59 -
60 - heredoc heredocState // whether we're in a heredoc
61 - heredocEscaped bool // whether heredoc is escaped
62 - heredocMarker []rune
63 - heredocClosingMarker []rune
64 -
65 - nesting int // indentation level
66 -
67 - currentToken strings.Builder
68 - currentLineFirstToken string
69 - previousLineWasTopLevelImport bool
70 - openBraceOwnLine bool
156 + firstAddrIdx = -1 // index of the first address token
157 + addrLineEndIdx = -1 // index of the last address token
158 + expectAnother bool // previous address token ended in ","
159 )
72 -
73 - finishToken := func() {
74 - if currentToken.Len() == 0 {
75 - return
160 + for i := range len(tokens) {
161 + tk := tokens[i]
162 + if tk.isComment {
163 + continue
164 }
77 - if currentLineFirstToken == "" {
78 - currentLineFirstToken = currentToken.String()
165 + // A structural "{" on the address line ends the address list AND means
166 + // the block is braced (a real site block) or, with no address seen yet,
167 + // a global-options block. Either way, not a single unbraced site.
168 + if isOpenCurlyBrace(tk) {
169 + return false
170 + }
171 + // If this token begins a new line and we're not waiting for another
172 + // comma-continued address, the address list ended on the previous token.
173 + if firstAddrIdx >= 0 && !expectAnother && fmtNextOnNewLine(tokens[i-1], tk) {
174 + break
175 }
80 - currentToken.Reset()
81 - }
176
83 - finishLine := func() {
84 - finishToken()
85 - if currentLineFirstToken != "" {
86 - previousLineWasTopLevelImport = nesting == 0 && currentLineFirstToken == "import"
87 - } else if !openBrace || !openBraceOwnLine || openBraceWritten {
88 - previousLineWasTopLevelImport = false
177 + // This token is part of the address list.
178 + if firstAddrIdx < 0 {
179 + firstAddrIdx = i
180 }
90 - currentLineFirstToken = ""
181 + addrLineEndIdx = i
182 + expectAnother = strings.HasSuffix(tk.Text, ",")
183 }
184
93 - write := func(ch rune) {
94 - out.WriteRune(ch)
95 - last = ch
185 + // Zero address tokens: not a site.
186 + if firstAddrIdx < 0 {
187 + return false
188 }
97 -
98 - indent := func() {
99 - for tabs := nesting; tabs > 0; tabs-- {
100 - write('\t')
101 - }
189 + if tokens[firstAddrIdx].Text == "import" {
190 + return false
191 }
192
104 - nextLine := func() {
105 - write('\n')
106 - beginningOfLine = true
193 + // Snippet "(...)" or named route "&(...)" first address: not a site.
194 + if firstText := tokens[firstAddrIdx].Text; strings.HasPrefix(firstText, "&(") || strings.HasPrefix(firstText, "(") {
195 + return false
196 }
197
109 - flushOpenBrace := func() {
110 - if nesting == 0 && last == '}' {
111 - nextLine()
112 - nextLine()
198 + // Walk the remaining tokens (the block body) tracking structural nesting.
199 + // A blank line (a gap of >= 2 source lines) between two top-level token
200 + // groups is the ambiguous multi-site shape and disqualifies wrapping. The
201 + // renderer itself inserts such a gap after an interior directive block, so a
202 + // gap immediately following its closing brace is allowed.
203 + nesting := 0
204 + prevIdx := -1
205 + for j := addrLineEndIdx + 1; j < len(tokens); j++ {
206 + tk := tokens[j]
207 + if tk.isComment {
208 + continue
209 }
114 -
115 - openBrace = false
116 - if openBraceOwnLine && previousLineWasTopLevelImport {
117 - if last != '\n' {
118 - nextLine()
210 + if nesting == 0 && prevIdx >= 0 {
211 + // Blank line at top level between directive groups: ambiguous
212 + // multi-site. A gap of two or more lines is a blank line.
213 + prev := tokens[prevIdx]
214 + if !isCloseCurlyBrace(prev) && tk.Line-(prev.Line+fmtNumLineBreaks(prev)) >= 2 {
215 + return false
216 }
120 - indent()
121 - } else if beginningOfLine {
122 - indent()
123 - } else if !openBraceSpace || !unicode.IsSpace(last) {
124 - write(' ')
125 - }
126 - write('{')
127 - openBraceWritten = true
128 - openBraceOwnLine = false
129 - nextLine()
130 - newLines = 0
131 - // prevent infinite nesting from ridiculous inputs (issue #4169)
132 - if nesting < 10 {
217 + }
218 + switch {
219 + case isOpenCurlyBrace(tk):
220 nesting++
221 + case isCloseCurlyBrace(tk) && nesting > 0:
222 + nesting--
223 }
224 + prevIdx = j
225 }
226
137 - for {
138 - ch, _, err := rdr.ReadRune()
139 - if err != nil {
140 - if err == io.EOF {
141 - break
142 - }
143 - panic(err)
144 - }
145 - // detect whether we have the start of a heredoc
146 - if quotes == "" && (heredoc == heredocClosed && !heredocEscaped) &&
147 - space && last == '<' && ch == '<' {
148 - write(ch)
149 - heredoc = heredocOpening
150 - space = false
227 + return true
228 +}
229 +
230 +// wrapUnbracedSite transforms a single unbraced site block into a braced one by
231 +// inserting a synthetic structural "{" after the address list and appending a
232 +// synthetic structural "}" at the end; the renderer (formatTokens) then indents
233 +// the body and emits the braces. If the token stream is not a single unbraced
234 +// site (per isSingleUnbracedSite), the tokens are returned unchanged.
235 +//
236 +// The synthetic braces are structural (Text/Raw "{"/"}", wasQuoted 0). The
237 +// opening brace's Line is the last address token's Line so it folds onto the
238 +// address line; the closing brace's Line follows the last body token so it lands
239 +// on its own line. Because the wrapped output is a genuinely braced site,
240 +// isSingleUnbracedSite returns false for it, so re-formatting is a fixed point
241 +// and does not re-wrap (idempotency), and the FormatWithOptions backstop — which
242 +// requires the output to re-lex to the same tokens and re-render identically —
243 +// does not fire.
244 +func wrapUnbracedSite(tokens []Token) []Token {
245 + if !isSingleUnbracedSite(tokens) {
246 + return tokens
247 + }
248 +
249 + // Recompute the address-list boundary the same way isSingleUnbracedSite does:
250 + // the leading run of non-comment tokens up to the first newline, extended
251 + // across newlines while the previous address token ends in a trailing comma.
252 + addrLineEndIdx := -1
253 + firstAddrIdx := -1
254 + expectAnother := false
255 + for i := range len(tokens) {
256 + tk := tokens[i]
257 + if tk.isComment {
258 continue
259 }
153 -
154 - if heredoc == heredocOpening {
155 - if ch == '\n' {
156 - if len(heredocMarker) > 0 && heredocMarkerRegexp.MatchString(string(heredocMarker)) {
157 - heredoc = heredocOpened
158 - } else {
159 - heredocMarker = nil
160 - heredoc = heredocClosed
161 - nextLine()
162 - continue
163 - }
164 - write(ch)
165 - continue
166 - }
167 - if unicode.IsSpace(ch) {
168 - // a space means it's just a regular token and not a heredoc
169 - heredocMarker = nil
170 - heredoc = heredocClosed
171 - } else {
172 - heredocMarker = append(heredocMarker, ch)
173 - write(ch)
174 - continue
175 - }
260 + if firstAddrIdx >= 0 && !expectAnother && fmtNextOnNewLine(tokens[i-1], tk) {
261 + break
262 }
177 - // if we're in a heredoc, all characters are read&write as-is
178 - if heredoc == heredocOpened {
179 - heredocClosingMarker = append(heredocClosingMarker, ch)
180 - if len(heredocClosingMarker) > len(heredocMarker)+1 { // We assert that the heredocClosingMarker is followed by a unicode.Space
181 - heredocClosingMarker = heredocClosingMarker[1:]
182 - }
183 - // check if we're done
184 - if unicode.IsSpace(ch) && slices.Equal(heredocClosingMarker[:len(heredocClosingMarker)-1], heredocMarker) {
185 - heredocMarker = nil
186 - heredocClosingMarker = nil
187 - heredoc = heredocClosed
188 - } else {
189 - write(ch)
190 - if ch == '\n' {
191 - heredocClosingMarker = heredocClosingMarker[:0]
192 - }
193 - continue
194 - }
263 + if firstAddrIdx < 0 {
264 + firstAddrIdx = i
265 }
266 + addrLineEndIdx = i
267 + expectAnother = strings.HasSuffix(tk.Text, ",")
268 + }
269
197 - if last == '<' && space {
198 - space = false
199 - }
270 + openBrace := Token{
271 + File: tokens[addrLineEndIdx].File,
272 + Line: tokens[addrLineEndIdx].Line,
273 + Text: "{",
274 + raw: "{",
275 + }
276 + closeBrace := Token{
277 + File: tokens[len(tokens)-1].File,
278 + Line: tokens[len(tokens)-1].Line + fmtNumLineBreaks(tokens[len(tokens)-1]) + 1,
279 + Text: "}",
280 + raw: "}",
281 + }
282
201 - if comment {
202 - if ch == '\n' {
203 - comment = false
204 - space = true
205 - nextLine()
206 - continue
207 - } else {
208 - write(ch)
209 - continue
210 - }
211 - }
283 + wrapped := make([]Token, 0, len(tokens)+2)
284 + wrapped = append(wrapped, tokens[:addrLineEndIdx+1]...)
285 + wrapped = append(wrapped, openBrace)
286 + wrapped = append(wrapped, tokens[addrLineEndIdx+1:]...)
287 + wrapped = append(wrapped, closeBrace)
288 + return wrapped
289 +}
290
213 - if !escaped && ch == '\\' {
214 - if space {
215 - write(' ')
216 - space = false
217 - }
218 - write(ch)
219 - escaped = true
220 - continue
291 +// trimmedWithNewline trims surrounding whitespace (except significant leading
292 +// whitespace before a backslash) and adds a single trailing newline, in a
293 +// freshly allocated buffer. The copy matters:
294 +// bytes.TrimSpace returns a subslice that shares input's backing array, so
295 +// appending to it can write into the caller's buffer (os.ReadFile hands back a
296 +// slice with spare capacity), corrupting the input a caller still holds — for
297 +// example the "caddy fmt --diff" comparison of input against output.
298 +func trimmedWithNewline(input []byte) []byte {
299 + trimmed := bytes.TrimSpace(input)
300 + // Leading whitespace can make a backslash a line continuation. Removing
301 + // it would change the token stream and make this fallback unstable.
302 + if len(trimmed) > 0 && trimmed[0] == '\\' {
303 + trimmed = bytes.TrimRightFunc(input, unicode.IsSpace)
304 + }
305 + out := make([]byte, 0, len(trimmed)+1)
306 + out = append(out, trimmed...)
307 + return append(out, '\n')
308 +}
309 +
310 +// trailingNewlineChangesTokens reports whether appending a newline to input
311 +// changes its format-mode token-text sequence. This is true for inputs whose
312 +// final token would swallow Format's mandatory trailing newline — an
313 +// unterminated or escaped quote/backtick, or a dangling escape at EOF — for
314 +// which no rendering can be a fixed point. Inputs already ending in a newline,
315 +// and well-formed inputs, are unaffected.
316 +func trailingNewlineChangesTokens(input []byte) bool {
317 + if n := len(input); n > 0 && input[n-1] == '\n' {
318 + return false
319 + }
320 + a, erra := lexFormat(input)
321 + withNL := make([]byte, len(input)+1)
322 + copy(withNL, input)
323 + withNL[len(input)] = '\n'
324 + b, errb := lexFormat(withNL)
325 + if erra != nil || errb != nil {
326 + return erra != errb
327 + }
328 + return !sameTokenTexts(a, b)
329 +}
330 +
331 +// sameTokenTexts reports whether two token streams have the same sequence of
332 +// token texts, ignoring comment tokens (comment whitespace is intentionally
333 +// normalized by the formatter).
334 +func sameTokenTexts(a, b []Token) bool {
335 + ai, bi := 0, 0
336 + for {
337 + for ai < len(a) && a[ai].isComment {
338 + ai++
339 }
340 + for bi < len(b) && b[bi].isComment {
341 + bi++
342 + }
343 + if ai >= len(a) || bi >= len(b) {
344 + return ai >= len(a) && bi >= len(b)
345 + }
346 + if a[ai].Text != b[bi].Text {
347 + return false
348 + }
349 + ai++
350 + bi++
351 + }
352 +}
353
223 - if escaped {
224 - if ch == '<' {
225 - heredocEscaped = true
226 - }
227 - write(ch)
228 - escaped = false
354 +// hasUnformattableToken reports whether the token stream contains a token whose
355 +// rendering cannot be made idempotent, so Format falls back to preserving the
356 +// input verbatim. Two shapes qualify:
357 +//
358 +// - An ambiguous heredoc opener: a regular (unquoted, non-comment) token whose
359 +// verbatim source is a valid heredoc opener shape ("<<MARKER") and which is
360 +// the last token on its source line. It stays literal in the source only
361 +// because trailing space separated it from the newline; the formatter drops
362 +// that space, so the rendered output would re-lex as a real heredoc opener
363 +// and glue in the following line. (An escaped opener "\<<MARKER" re-lexes as
364 +// a regular token and is unaffected, so only a bare "<<MARKER" qualifies.)
365 +// - An unterminated quote/backtick: a token whose verbatim source begins with
366 +// a quote or backtick but which is not marked Quoted (the closing delimiter
367 +// was never seen). It swallowed the rest of the input; rendering it and
368 +// appending the mandatory trailing newline changes what it lexes back to.
369 +// - A leading line continuation: the first token carries continuation framing
370 +// ("\"+newline) even though there is no preceding token to continue from.
371 +// Its anchoring Line sits a line before its rendered content, so the
372 +// vertical-spacing math for the following token is unstable across passes.
373 +func hasUnformattableToken(tokens []Token) bool {
374 + for i, tk := range tokens {
375 + if tk.isComment {
376 continue
377 }
231 -
232 - if ch == '`' {
233 - switch quotes {
234 - case "\"`":
235 - quotes = "\""
236 - case "`":
237 - quotes = ""
238 - case "\"":
239 - quotes = "\"`"
240 - default:
241 - quotes = "`"
242 - }
378 + // Leading dangling line continuation on the first non-comment token.
379 + if !anyNonCommentBefore(tokens, i) && tk.continuation {
380 + return true
381 }
244 -
245 - if quotes == "\"" {
246 - if ch == '"' {
247 - quotes = ""
248 - }
249 - write(ch)
382 + // Messy line continuation: continuation framing whose backslash is not
383 + // immediately followed by the newline (e.g. "\"+vertical-tab+newline). The
384 + // formatter's continuation normalization cannot cleanly strip the stray
385 + // whitespace, so the token text shifts on re-lex.
386 + if tk.continuation && hasMessyContinuationFraming(tk.Raw()) {
387 + return true
388 + }
389 + raw := tk.Raw()
390 + // A non-quoted token whose verbatim source ends in a newline swallowed it:
391 + // an unterminated quote/backtick, or an escaped quote ("\\"") that ran to
392 + // end-of-input. A newline otherwise terminates a token, so no well-formed
393 + // token ends in one. Format emits the source verbatim and then trims
394 + // trailing whitespace, so such a token changes on re-lex and no rendering
395 + // is a fixed point. This has to be decided from the token itself rather
396 + // than from whether appending a newline changes the stream: once the
397 + // mandatory trailing newline has been appended it sits inside the
398 + // swallowing token, and appending another one no longer changes anything,
399 + // so only the token shape still reveals the problem. A lone "\\r" is kept
400 + // in raw but does not terminate a token, so it is deliberately not
401 + // treated as swallowed; heredoc tokens carry wasQuoted == '<'.
402 + if tk.wasQuoted == 0 && strings.HasSuffix(raw, "\n") {
403 + return true
404 + }
405 + if tk.wasQuoted != 0 {
406 continue
407 }
252 -
253 - if ch == '"' {
254 - switch quotes {
255 - case "":
256 - if space {
257 - quotes = "\""
258 - }
259 - case "`\"":
260 - quotes = "`"
261 - case "\"`":
262 - quotes = ""
263 - }
408 + // Ambiguous heredoc opener.
409 + if !strings.HasPrefix(raw, "<<") || len(raw) <= 2 {
410 + continue
411 }
265 -
266 - if strings.Contains(quotes, "`") {
267 - if ch == '`' && space && !beginningOfLine {
268 - write(' ')
269 - }
270 - write(ch)
271 - space = false
412 + marker := raw[2:]
413 + if !heredocMarkerRegexp.MatchString(marker) {
414 continue
415 }
416 + if i+1 >= len(tokens) || fmtNextOnNewLine(tk, tokens[i+1]) {
417 + return true
418 + }
419 + }
420 + return false
421 +}
422
275 - if unicode.IsSpace(ch) {
276 - finishToken()
277 - space = true
278 - heredocEscaped = false
279 - if ch == '\n' {
280 - finishLine()
281 - newLines++
282 - }
283 - continue
423 +// anyNonCommentBefore reports whether any token before index i is not a comment.
424 +func anyNonCommentBefore(tokens []Token, i int) bool {
425 + for j := range i {
426 + if !tokens[j].isComment {
427 + return true
428 }
285 - spacePrior := space
286 - space = false
429 + }
430 + return false
431 +}
432
288 - //////////////////////////////////////////////////////////
289 - // I find it helpful to think of the formatting loop in two
290 - // main sections; by the time we reach this point, we
291 - // know we are in a "regular" part of the file: we know
292 - // the character is not a space, not in a literal segment
293 - // like a comment or quoted, it's not escaped, etc.
294 - //////////////////////////////////////////////////////////
433 +// hasMessyContinuationFraming reports whether raw begins (after any leading
434 +// whitespace) with a line-continuation backslash that is not immediately
435 +// followed by a newline — i.e. there is stray whitespace between the "\" and the
436 +// newline (e.g. "\"+vertical-tab+newline). A clean continuation has "\" directly
437 +// before the newline.
438 +func hasMessyContinuationFraming(raw string) bool {
439 + s := strings.TrimLeft(raw, " \t\r\n\v\f")
440 + if !strings.HasPrefix(s, `\`) {
441 + return false
442 + }
443 + rest := s[1:]
444 + return len(rest) > 0 && rest[0] != '\n' && rest[0] != '\r'
445 +}
446
296 - if ch == '#' {
297 - comment = true
447 +// fmtNumLineBreaks returns how many physical line breaks a token spans, for use
448 +// in format-mode vertical-spacing decisions. For quoted tokens (double-quote,
449 +// backtick, and heredoc) it counts the newlines in the verbatim source (Raw)
450 +// rather than deriving the count from the processed Text. Raw is the exact
451 +// physical span, so this correctly handles an empty-bodied heredoc ("<<E\nE",
452 +// which Token.NumLineBreaks over-counts by always adding two) and a quoted token
453 +// whose Raw carries a leading line-continuation newline (e.g. "\\\n`...`", which
454 +// Token.NumLineBreaks under-counts); both otherwise place a following token on
455 +// the wrong line and break idempotency. Unquoted tokens keep Token.NumLineBreaks
456 +// (their Raw may carry "\"+newline continuation framing whose newline belongs to
457 +// the token's anchoring Line, not to its span).
458 +func fmtNumLineBreaks(t Token) int {
459 + if t.wasQuoted != 0 {
460 + return strings.Count(t.Raw(), "\n")
461 + }
462 + return t.NumLineBreaks()
463 +}
464 +
465 +// fmtNextOnNewLine mirrors isNextOnNewLine but uses fmtNumLineBreaks so that
466 +// empty-bodied heredocs are measured by their true line span in format mode.
467 +func fmtNextOnNewLine(t1, t2 Token) bool {
468 + if t1.File != t2.File {
469 + return true
470 + }
471 + if len(t1.imports) != len(t2.imports) {
472 + return true
473 + }
474 + for i, im := range t1.imports {
475 + if im != t2.imports[i] {
476 + return true
477 }
478 + }
479 + return t1.Line+fmtNumLineBreaks(t1) < t2.Line
480 +}
481
300 - if openBrace && spacePrior && !openBraceWritten {
301 - flushOpenBrace()
482 +// formatTokens renders a format-mode token stream (as produced by
483 +// Lex(..., LexOptions{Comments: true, Raw: true})) into a normalized
484 +// Caddyfile. It controls all indentation and structural whitespace; token
485 +// bodies are emitted verbatim via Token.Raw().
486 +func formatTokens(tokens []Token) []byte {
487 + var out bytes.Buffer
488 +
489 + nesting := 0 // current indentation level
490 + atLineStart := true // whether the cursor is at the start of a fresh line
491 + wrote := false // whether any non-whitespace has been written yet
492 + prevTopClose := false
493 +
494 + writeIndent := func() {
495 + for range min(nesting, maxIndent) {
496 + out.WriteByte('\t')
497 }
498 + }
499
304 - switch {
305 - case ch == '{':
306 - finishToken()
307 - if openBrace && !openBraceWritten {
308 - flushOpenBrace()
309 - }
310 - openBrace = true
311 - openBraceSpace = spacePrior && !beginningOfLine
312 - openBraceOwnLine = newLines > 0
313 - if openBraceSpace && newLines == 0 {
314 - write(' ')
500 + newline := func() {
501 + out.WriteByte('\n')
502 + atLineStart = true
503 + }
504 +
505 + // prevWasComment tracks whether the last non-whitespace token emitted was a
506 + // comment. A structural "{" must never fold onto a line whose last token is
507 + // a comment, since "#" runs to end of line and would comment the brace out.
508 + prevWasComment := false
509 + // prevWasClose tracks whether the last non-whitespace token emitted was a
510 + // structural "}". A token that shared the source line with that "}" (other
511 + // than another "}" or a trailing comment) must break to its own indented
512 + // line rather than folding after the brace (I4).
513 + prevWasClose := false
514 + // foldedOpenAt, when >= 0, is the index of a structural "{" that was already
515 + // emitted early (before a trailing comment) as part of the address/comment/
516 + // brace fold; when the loop reaches that index it is skipped.
517 + foldedOpenAt := -1
518 + // lineStartsWithImport tracks whether the first token of the current
519 + // nesting-zero source line is "import". A standalone "{" that opens a
520 + // global-options block immediately after such a line must stay on its own
521 + // line instead of folding onto the import line (the "import" directive and a
522 + // global-options "{" are unrelated statements). See the import exception in
523 + // the isOpen case below.
524 + lineStartsWithImport := false
525 +
526 + for i := range tokens {
527 + if i == foldedOpenAt {
528 + // This "{" was already emitted ahead of a trailing comment.
529 + continue
530 + }
531 + tk := tokens[i]
532 + // Track whether the first token of the current nesting-zero source line
533 + // is "import". A token starts a new source line when it is the very
534 + // first token or is on a later line than its predecessor. This is
535 + // computed against the pre-token nesting so that a top-level "import"
536 + // line is recognized before any brace on a following line dedents.
537 + // Braces are not directives, so a line beginning with "{" or "}" leaves
538 + // the tracker untouched: the isOpen import exception below needs to see
539 + // the state established by the preceding import line.
540 + if !isCloseCurlyBrace(tk) && !isOpenCurlyBrace(tk) {
541 + startsNewLine := !wrote || fmtNextOnNewLine(tokens[i-1], tk)
542 + if startsNewLine {
543 + lineStartsWithImport = nesting == 0 && !tk.isComment && tk.Text == "import"
544 }
316 - openBraceWritten = false
317 - if quotes == "`" {
318 - write('{')
319 - openBraceWritten = true
320 - openBraceOwnLine = false
321 - continue
545 + }
546 + isOpen := isOpenCurlyBrace(tk)
547 + // A structural close brace only closes a block when one is open. When
548 + // nesting is zero there is nothing to close (e.g. a "}" glued after a
549 + // backtick/quoted argument), so it is treated as an inline literal and
550 + // emitted verbatim at its source position.
551 + isClose := isCloseCurlyBrace(tk) && nesting > 0
552 +
553 + // A trailing comment shares its source line with the previous token
554 + // (e.g. after "}", after "{", or after a directive) and renders inline
555 + // on that line; a standalone comment sits alone on its own line.
556 + trailingComment := tk.isComment && wrote && !fmtNextOnNewLine(tokens[i-1], tk)
557 +
558 + // breaks is the number of newlines to emit before this token:
559 + // 0 = stay on the current line, 1 = new line, 2 = one blank line.
560 + breaks := 0
561 + if wrote {
562 + prev := tokens[i-1]
563 + if fmtNextOnNewLine(prev, tk) {
564 + breaks = 1
565 + // A gap of two or more lines yields at most one blank line.
566 + if tk.Line-(prev.Line+fmtNumLineBreaks(prev)) >= 2 {
567 + breaks = 2
568 + }
569 }
323 - continue
570 + }
571
325 - case ch == '}' && (spacePrior || !openBrace):
326 - finishToken()
327 - if quotes == "`" {
328 - write('}')
329 - continue
572 + switch {
573 + case isOpen:
574 + // An opening brace attaches to the current line: never break to a
575 + // new line before it (it joins the preceding token with a space).
576 + // EXCEPTION 1: a standalone comment line must keep the following "{"
577 + // on its own line, otherwise the "#" would comment the brace out.
578 + // EXCEPTION 2: a standalone "{" (one that begins a new source line)
579 + // that immediately follows a top-level "import" line, with no
580 + // intervening blank line, opens an unrelated global-options block
581 + // and must stay on its own line rather than folding onto the import.
582 + standaloneBrace := wrote && fmtNextOnNewLine(tokens[i-1], tk)
583 + if prevWasComment {
584 + breaks = 1
585 + } else if lineStartsWithImport && nesting == 0 && standaloneBrace {
586 + breaks = max(breaks, 1)
587 + } else {
588 + breaks = 0
589 }
331 - if last != '\n' {
332 - nextLine()
590 + case isClose:
591 + // A closing brace always goes on its own line, and dedents.
592 + if wrote && !atLineStart {
593 + breaks = 1
594 }
595 if nesting > 0 {
596 nesting--
597 }
337 - indent()
338 - write('}')
339 - newLines = 0
340 - continue
598 + case trailingComment:
599 + // A trailing comment stays on the current line.
600 + breaks = 0
601 }
602
343 - if newLines > 2 {
344 - newLines = 2
603 + // A token that shared the source line with a preceding structural "}"
604 + // (other than another "}" or a trailing comment) breaks to its own
605 + // indented line instead of folding after the brace (I4).
606 + if prevWasClose && breaks == 0 && !isClose && !trailingComment {
607 + breaks = 1
608 }
346 - for i := 0; i < newLines; i++ {
347 - nextLine()
609 +
610 + // The content following an opening brace always begins on its own
611 + // indented line, regardless of how the source was laid out. This
612 + // applies to any token, including a nested opening brace. A trailing
613 + // comment after "{" is the exception: it stays inline on the "{" line.
614 + if wrote && isOpenCurlyBrace(tokens[i-1]) && breaks == 0 && !trailingComment {
615 + breaks = 1
616 }
349 - newLines = 0
350 - if beginningOfLine {
351 - indent()
617 +
618 + // A blank line always follows a top-level closing brace, unless this
619 + // token is a comment trailing that brace on the same source line.
620 + if prevTopClose && breaks < 2 && !trailingComment {
621 + breaks = 2
622 }
353 - if nesting == 0 && last == '}' && beginningOfLine {
354 - nextLine()
355 - nextLine()
623 + prevTopClose = false
624 +
625 + // Address/comment/brace fold: when this is a trailing comment whose very
626 + // next token is a structural "{" that opens this line's block, emit the
627 + // "{" on the head line before the comment, so "addr # c"⏎"{" renders as
628 + // "addr { # c". This must happen before any newline is emitted.
629 + // It only applies when the comment trails the block's head line (a
630 + // directive/address). If the comment trails an opening "{" (i.e. the
631 + // previous token is itself a "{"), the following "{" opens a separate
632 + // nested block on its own line and must NOT be folded up onto the comment
633 + // line; doing so is non-idempotent (a second pass sees the two "{" already
634 + // adjacent and breaks them apart again).
635 + if trailingComment && i+1 < len(tokens) && isOpenCurlyBrace(tokens[i+1]) && breaks == 0 && !atLineStart &&
636 + !isOpenCurlyBrace(tokens[i-1]) {
637 + out.WriteByte(' ')
638 + out.WriteString(tokens[i+1].Raw())
639 + nesting++
640 + foldedOpenAt = i + 1
641 }
642
358 - if !beginningOfLine && spacePrior {
359 - write(' ')
643 + // Emit the vertical spacing.
644 + for breaks > 0 {
645 + newline()
646 + breaks--
647 }
648
362 - if openBrace && !openBraceWritten {
363 - write('{')
364 - openBraceWritten = true
649 + // Emit indentation at the start of a line.
650 + if atLineStart {
651 + writeIndent()
652 }
653
367 - if spacePrior && ch == '<' {
368 - space = true
654 + // A line continuation renders as "\", a newline, then a hanging indent
655 + // of nesting+1 tabs before the token (instead of a single space). The
656 + // token's Raw() carries the source continuation framing ("\"+newline+
657 + // indentation) as a prefix, so it is stripped and re-emitted normalized.
658 + body := tk.Raw()
659 + if tk.continuation && !atLineStart {
660 + out.WriteString(" \\\n")
661 + for range min(nesting+1, maxIndent) {
662 + out.WriteByte('\t')
663 + }
664 + body = strings.TrimLeft(strings.TrimPrefix(body, "\\"), " \t\r\n")
665 + atLineStart = false
666 + } else if !atLineStart {
667 + // Emit horizontal separation for same-line tokens: exactly one space
668 + // separates two tokens that share a line (including an attaching "{").
669 + // A structural close brace at nesting zero (an inline literal) stays
670 + // glued to its predecessor when the source had no space. A comment
671 + // glues to a preceding quoted/backtick/heredoc token only when no
672 + // space separated them ("x"#c); otherwise it takes a leading space.
673 + // precededBySpace is only reliable after a quoted/backtick/heredoc
674 + // token (the delimiting space of an unquoted predecessor is consumed
675 + // by its own scan), so gluing an inline close brace is allowed only
676 + // when the previous token is quoted; otherwise the separator must be
677 + // preserved so two distinct tokens (e.g. "0" then "}") do not merge.
678 + inlineClose := isCloseCurlyBrace(tk) && !isClose && tokens[i-1].Quoted()
679 + gluedComment := tk.isComment && tokens[i-1].Quoted() && !tk.precededBySpace
680 + if (!inlineClose || tk.precededBySpace) && !gluedComment {
681 + out.WriteByte(' ')
682 + }
683 }
684
371 - currentToken.WriteRune(ch)
372 - write(ch)
373 -
374 - beginningOfLine = false
375 - }
376 -
377 - if openBrace && !openBraceWritten {
378 - flushOpenBrace()
685 + // Emit the token body verbatim.
686 + out.WriteString(body)
687 + wrote = true
688 + atLineStart = false
689 + prevWasComment = tk.isComment
690 + prevWasClose = isClose
691 +
692 + // Structural bookkeeping after emitting. Vertical spacing after a
693 + // brace is produced by the next token's break calculation, so we don't
694 + // emit trailing newlines here.
695 + if isOpen {
696 + nesting++
697 + } else if isClose && nesting == 0 {
698 + prevTopClose = true
699 + }
700 }
701
381 - // the Caddyfile does not need any leading or trailing spaces, but...
382 - trimmedResult := bytes.TrimSpace(out.Bytes())
383 -
384 - // ...Caddyfiles should, however, end with a newline because
385 - // newlines are significant to the syntax of the file
386 - return append(trimmedResult, '\n')
702 + // Trim leading/trailing whitespace, then ensure exactly one trailing
703 + // newline (newlines are significant to Caddyfile syntax).
704 + trimmed := bytes.TrimSpace(out.Bytes())
705 + return append(trimmed, '\n')
706 }
caddyconfig/caddyfile/formatter_braced_test.go new
+117
@@ -0,0 +1,117 @@
1 +// Copyright 2015 Matthew Holt and The Caddy Authors
2 +//
3 +// Licensed under the Apache License, Version 2.0 (the "License");
4 +// you may not use this file except in compliance with the License.
5 +// You may obtain a copy of the License at
6 +//
7 +// http://www.apache.org/licenses/LICENSE-2.0
8 +//
9 +// Unless required by applicable law or agreed to in writing, software
10 +// distributed under the License is distributed on an "AS IS" BASIS,
11 +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 +// See the License for the specific language governing permissions and
13 +// limitations under the License.
14 +
15 +package caddyfile
16 +
17 +import (
18 + "bytes"
19 + "testing"
20 +)
21 +
22 +func TestFormatWithWrapUnbracedSite(t *testing.T) {
23 + // eligible: wrapped
24 + in := "localhost\nroot * /srv\nfile_server\n"
25 + want := "localhost {\n\troot * /srv\n\tfile_server\n}\n"
26 + if got := string(FormatWithOptions([]byte(in), FormatOptions{WrapUnbracedSite: true})); got != want {
27 + t.Errorf("got %q, want %q", got, want)
28 + }
29 + // ineligible: unchanged from default formatting (no-op wrap)
30 + snip := "(s) {\n\trespond 200\n}\n"
31 + if got := string(FormatWithOptions([]byte(snip), FormatOptions{WrapUnbracedSite: true})); got != string(Format([]byte(snip))) {
32 + t.Errorf("snippet should be a no-op for WrapUnbracedSite; got %q", got)
33 + }
34 + // default (option off) never wraps
35 + if got := string(Format([]byte(in))); got != "localhost\nroot * /srv\nfile_server\n" {
36 + t.Errorf("default Format must not wrap; got %q", got)
37 + }
38 +}
39 +
40 +func TestWrapUnbracedSiteIdempotentAndSemantic(t *testing.T) {
41 + in := "localhost\nrespond 200\n"
42 + once := FormatWithOptions([]byte(in), FormatOptions{WrapUnbracedSite: true})
43 + twice := FormatWithOptions(once, FormatOptions{WrapUnbracedSite: true})
44 + if !bytes.Equal(once, twice) {
45 + t.Errorf("not idempotent:\n once=%q\ntwice=%q", once, twice)
46 + }
47 +}
48 +
49 +func TestWrapUnbracedSiteIdempotentReproducer(t *testing.T) {
50 + // A valid Caddyfile (Parse accepts it) that used to be non-idempotent under
51 + // WrapUnbracedSite: it reads as "not a single site" on the raw stream (a
52 + // blank-line gap), but after formatTokens removes the blank line and folds a
53 + // dangling brace, a second pass saw a single unbraced site and wrapped it.
54 + // Deciding on the normalized stream makes the wrap decision stable.
55 + in := []byte(") 0((.b ))\na\t\tab(\n}\n\n { }")
56 + if _, err := Parse("", in); err != nil {
57 + t.Fatalf("reproducer must be a valid Caddyfile, got parse error: %v", err)
58 + }
59 + once := FormatWithOptions(in, FormatOptions{WrapUnbracedSite: true})
60 + twice := FormatWithOptions(once, FormatOptions{WrapUnbracedSite: true})
61 + if !bytes.Equal(once, twice) {
62 + t.Errorf("not idempotent:\n once=%q\ntwice=%q", once, twice)
63 + }
64 +}
65 +
66 +func FuzzWrapUnbracedSiteIdempotent(f *testing.F) {
67 + seeds := []string{
68 + "localhost\nrespond 200\n",
69 + "localhost\nroot * /srv\nfile_server\n",
70 + "example.com\nreverse_proxy {\n\tto a:80\n}\n",
71 + "a.com\nrespond 200\n\nb.com\nrespond 404\n",
72 + "(snip) {\n\trespond 200\n}\n",
73 + ") 0((.b ))\na\t\tab(\n}\n\n { }",
74 + "localhost {\n\trespond 200\n}\n",
75 + }
76 + for _, s := range seeds {
77 + f.Add([]byte(s))
78 + }
79 + f.Fuzz(func(t *testing.T, in []byte) {
80 + // Valid-only, like FuzzFormatIdempotent: skip inputs Parse rejects.
81 + if _, err := Parse("", in); err != nil {
82 + t.Skip()
83 + }
84 + once := FormatWithOptions(in, FormatOptions{WrapUnbracedSite: true})
85 + twice := FormatWithOptions(once, FormatOptions{WrapUnbracedSite: true})
86 + if !bytes.Equal(once, twice) {
87 + t.Errorf("WrapUnbracedSite not idempotent for %q:\n once=%q\ntwice=%q", in, once, twice)
88 + }
89 + })
90 +}
91 +
92 +func TestIsSingleUnbracedSite(t *testing.T) {
93 + yes := []string{
94 + "localhost\nrespond 200\n",
95 + "localhost\nreverse_proxy {\n\tto a:80\n}\n", // interior braces OK
96 + }
97 + no := []string{
98 + "localhost {\n\trespond 200\n}\n", // already braced
99 + "(snip) {\n\trespond 200\n}\n", // snippet
100 + "&(route) {\n\trespond 200\n}\n", // named route
101 + "{\n\tdebug\n}\n", // global options only
102 + "a.com\nrespond 200\n\nb.com\nrespond 404\n", // multi-site (ambiguous)
103 + "", // empty
104 + }
105 + for _, in := range yes {
106 + toks, _ := Lex([]byte(in), "", LexOptions{Comments: true, Raw: true})
107 + if !isSingleUnbracedSite(toks) {
108 + t.Errorf("want single-unbraced-site for %q", in)
109 + }
110 + }
111 + for _, in := range no {
112 + toks, _ := Lex([]byte(in), "", LexOptions{Comments: true, Raw: true})
113 + if isSingleUnbracedSite(toks) {
114 + t.Errorf("did NOT expect single-unbraced-site for %q", in)
115 + }
116 + }
117 +}
caddyconfig/caddyfile/formatter_test.go
+653 -21
@@ -15,10 +15,40 @@
15 package caddyfile
16
17 import (
18 + "bytes"
19 + "os"
20 + "reflect"
21 "strings"
22 "testing"
23 )
24
25 +func TestMain(m *testing.M) {
26 + // Pin an empty environment so replaceEnvVars is deterministic for fuzzing.
27 + os.Clearenv()
28 + os.Exit(m.Run())
29 +}
30 +
31 +func TestFormatWindowsTrailingBackslash(t *testing.T) {
32 + input := "localhost {\nroot * C:\\site\\\nfile_server\n}\n"
33 + want := "localhost {\n\troot * C:\\site\\\n\tfile_server\n}\n"
34 + for _, in := range []string{input, strings.ReplaceAll(input, "\n", "\r\n")} {
35 + out := Format([]byte(in))
36 + if strings.ReplaceAll(string(out), "\r\n", "\n") != want {
37 + t.Fatalf("got %q, want %q", out, want)
38 + }
39 + if !bytes.Equal(Format(out), out) {
40 + t.Fatal("formatting is not idempotent")
41 + }
42 + blocks, err := Parse("", out)
43 + if err != nil {
44 + t.Fatal(err)
45 + }
46 + if len(blocks) != 1 || len(blocks[0].Segments) != 2 {
47 + t.Fatalf("expected separate root and file_server directives, got %+v", blocks)
48 + }
49 + }
50 +}
51 +
52 func TestFormatter(t *testing.T) {
53 for i, tc := range []struct {
54 description string
@@ -127,15 +157,15 @@ m {
157
158 c{ d
159 }`,
130 - expect: `a {
131 - b
160 + expect: `a{
161 +b
162 }
163
134 -c {
135 - d
164 +c{ d
165 }`,
166 },
167 {
168 + // sanctioned divergence: invalid glued braces left literal (design decision)
169 description: "advanced spacing",
170 input: `abc {
171 def
@@ -144,12 +174,9 @@ c {
174 pqr}`,
175 expect: `abc {
176 def
147 -}
148 -
149 -ghi {
177 + }ghi{
178 jkl mno
151 - pqr
152 -}`,
179 + pqr}`,
180 },
181 {
182 description: "env var placeholders",
@@ -210,8 +237,7 @@ d {
237 # g
238 }
239
213 -h {
214 - # i
240 +h { # i
241 }`,
242 },
243 {
@@ -266,6 +292,7 @@ j {
292 `,
293 },
294 {
295 + // sanctioned divergence: invalid glued braces left literal (design decision)
296 description: "bad nesting (too many close)",
297 input: `a
298 {
@@ -273,9 +300,7 @@ j {
300 }}}`,
301 expect: `a {
302 {
276 - }
277 -}
278 -}
303 + }}}
304 `,
305 },
306 {
@@ -302,13 +327,10 @@ bar "{\"key\":34}"`,
327 expect: `foo{bar} foo{bar}baz`,
328 },
329 {
330 + // sanctioned divergence: invalid glued braces left literal (design decision)
331 description: "placeholders and malformed braces",
332 input: `foo{bar} foo{ bar}baz`,
307 - expect: `foo{bar} foo {
308 - bar
309 -}
310 -
311 -baz`,
333 + expect: `foo{bar} foo{ bar}baz`,
334 },
335 {
336 description: "hash within string is not a comment",
@@ -508,8 +530,8 @@ import ./conf.d/matcher_not_my_subnet.caddy
530 {
531 description: "keep adjacent opening braces that end the input",
532 input: `{{`,
511 - expect: `{
512 - {`,
533 + // Glued braces are a literal token, not structural blocks.
534 + expect: `{{`,
535 },
536 {
537 description: "keep spaced opening braces that end the input",
@@ -532,3 +554,613 @@ import ./conf.d/matcher_not_my_subnet.caddy
554 }
555 }
556 }
557 +
558 +func TestFormatImportStandaloneBrace(t *testing.T) {
559 + // immediately after a top-level import line: keep the brace standalone
560 + in1 := "import a.caddy\nimport b.caddy\n{\n\torder x first\n}\n"
561 + if got := string(Format([]byte(in1))); got != in1 {
562 + t.Errorf("standalone brace after import not preserved:\n got %q\nwant %q", got, in1)
563 + }
564 + // An intervening blank line must not turn the global options block into an
565 + // import block mapping.
566 + in2 := "import a.caddy\n\n{\n\torder x first\n}\n"
567 + want2 := in2
568 + if got := string(Format([]byte(in2))); got != want2 {
569 + t.Errorf("blank-line case:\n got %q\nwant %q", got, want2)
570 + }
571 +}
572 +
573 +func TestFormatDeepNestingCapsIndent(t *testing.T) {
574 + // Brace matching remains unbounded, but rendered indentation is capped to
575 + // keep linear-size malicious input from producing quadratic output.
576 + var b strings.Builder
577 + const depth = 1000
578 + for range depth {
579 + b.WriteString("l {\n")
580 + }
581 + b.WriteString("x\n")
582 + for range depth {
583 + b.WriteString("}\n")
584 + }
585 + out := string(Format([]byte(b.String())))
586 + if !strings.Contains(out, "\n"+strings.Repeat("\t", maxIndent)+"x\n") {
587 + t.Errorf("expected %d-tab indent for deepest token", maxIndent)
588 + }
589 + if len(out) > b.Len()+(2*depth+1)*maxIndent {
590 + t.Fatalf("formatted output grew unexpectedly: input=%d output=%d", b.Len(), len(out))
591 + }
592 +}
593 +
594 +func TestFormatPreservesLiteralBraceArguments(t *testing.T) {
595 + for _, tc := range []struct {
596 + in string
597 + want string
598 + }{
599 + {"localhost\nrespond foo{\n", "localhost\nrespond foo{\n"},
600 + {"localhost\nrespond {} 200\n", "localhost\nrespond {} 200\n"},
601 + } {
602 + before, err := Tokenize([]byte(tc.in), "")
603 + if err != nil {
604 + t.Fatalf("input %q: %v", tc.in, err)
605 + }
606 + out := Format([]byte(tc.in))
607 + if string(out) != tc.want {
608 + t.Errorf("Format(%q) = %q, want %q", tc.in, out, tc.want)
609 + }
610 + after, err := Tokenize(out, "")
611 + if err != nil {
612 + t.Fatalf("formatted input %q no longer parses: %v\nout=%q", tc.in, err, out)
613 + }
614 + if !sameTokenTexts(before, after) {
615 + t.Errorf("formatting changed parser tokens for %q:\nbefore=%v\nafter=%v\nout=%q", tc.in, before, after, out)
616 + }
617 + }
618 +}
619 +
620 +func TestFormatEscapedDelimitersDoesNotDisableFormatting(t *testing.T) {
621 + in := "localhost {\nrespond foo\\\"bar\nrespond foo\\`bar\n}\n"
622 + want := "localhost {\n\trespond foo\\\"bar\n\trespond foo\\`bar\n}\n"
623 + if got := string(Format([]byte(in))); got != want {
624 + t.Errorf("got %q, want %q", got, want)
625 + }
626 +}
627 +
628 +func TestFinalizeHeredocLargeBody(t *testing.T) {
629 + line := strings.Repeat("x", 1024)
630 + var val strings.Builder
631 + for range 1024 {
632 + val.WriteByte('\t')
633 + val.WriteString(line)
634 + val.WriteByte('\n')
635 + }
636 + val.WriteString("\tEND")
637 +
638 + got, err := new(lexer).finalizeHeredoc([]rune(val.String()), "END")
639 + if err != nil {
640 + t.Fatal(err)
641 + }
642 + wantLen := 1024*(len(line)+1) - 1
643 + if len(string(got)) != wantLen {
644 + t.Fatalf("large heredoc length = %d, want %d", len(string(got)), wantLen)
645 + }
646 +}
647 +
648 +func TestWrapUnbracedSiteRejectsLeadingImport(t *testing.T) {
649 + in := "import common.caddy\nlocalhost\nrespond 200\n"
650 + want := string(Format([]byte(in)))
651 + if got := string(FormatWithOptions([]byte(in), FormatOptions{WrapUnbracedSite: true})); got != want {
652 + t.Errorf("leading import was wrapped:\n got %q\nwant %q", got, want)
653 + }
654 +}
655 +
656 +func TestWrapUnbracedSiteAfterNestedDirective(t *testing.T) {
657 + in := "localhost\nroute {\nrespond ok\n}\nfile_server\n"
658 + want := "localhost {\n\troute {\n\t\trespond ok\n\t}\n\tfile_server\n}\n"
659 + if got := string(FormatWithOptions([]byte(in), FormatOptions{WrapUnbracedSite: true})); got != want {
660 + t.Errorf("got %q, want %q", got, want)
661 + }
662 +}
663 +
664 +func TestFormatAngleNotQuirked(t *testing.T) {
665 + if got := string(Format([]byte("foo < bar"))); got != "foo < bar\n" {
666 + t.Errorf("got %q, want %q", got, "foo < bar\n")
667 + }
668 +}
669 +
670 +func TestFormatEmptyBlocksExpand(t *testing.T) {
671 + cases := []struct{ in, want string }{
672 + {"route {}", "route {}\n"},
673 + {"route { }", "route {\n}\n"},
674 + {"a { b {} }", "a {\n\tb {}\n}\n"},
675 + }
676 + for _, c := range cases {
677 + if got := string(Format([]byte(c.in))); got != c.want {
678 + t.Errorf("in %q: got %q, want %q", c.in, got, c.want)
679 + }
680 + }
681 +}
682 +
683 +func TestFormatTokenAfterCloseBraceBreaks(t *testing.T) {
684 + in := "a {\n\tb {\n\t\tc\n\t} d\n}"
685 + want := "a {\n\tb {\n\t\tc\n\t}\n\td\n}\n"
686 + if got := string(Format([]byte(in))); got != want {
687 + t.Errorf("got %q, want %q", got, want)
688 + }
689 +}
690 +
691 +func TestFormatCommentsOnBraceLines(t *testing.T) {
692 + cases := []struct{ in, want string }{
693 + {"site {\n\tfoo\n} # after close\n", "site {\n\tfoo\n} # after close\n"},
694 + {"site { # note\n\tfoo\n}\n", "site { # note\n\tfoo\n}\n"},
695 + {"site # note\n{\n\tfoo\n}\n", "site { # note\n\tfoo\n}\n"},
696 + }
697 + for _, c := range cases {
698 + if got := string(Format([]byte(c.in))); got != c.want {
699 + t.Errorf("in %q:\n got %q\nwant %q", c.in, got, c.want)
700 + }
701 + }
702 +}
703 +
704 +func TestFormatBlankLineCapAfterComment(t *testing.T) {
705 + got := string(Format([]byte("foo # inline\n\n\nbar\n")))
706 + want := "foo # inline\n\nbar\n"
707 + if got != want {
708 + t.Errorf("got %q, want %q", got, want)
709 + }
710 +}
711 +
712 +func TestFormatContinuationHangingIndent(t *testing.T) {
713 + in := "route {\n\treverse_proxy \\\n\ta \\\n\tb\n}\n"
714 + want := "route {\n\treverse_proxy \\\n\t\ta \\\n\t\tb\n}\n"
715 + if got := string(Format([]byte(in))); got != want {
716 + t.Errorf("got %q, want %q", got, want)
717 + }
718 +}
719 +
720 +func TestFormatStrayCloseBraceKeepsTokens(t *testing.T) {
721 + for _, in := range []string{"0 }", "a }"} {
722 + once := Format([]byte(in))
723 + twice := Format(once)
724 + if !bytes.Equal(once, twice) {
725 + t.Errorf("not idempotent for %q:\n once=%q\ntwice=%q", in, once, twice)
726 + }
727 + a, err1 := Parse("Caddyfile", []byte(in))
728 + if err1 != nil {
729 + t.Fatalf("input %q did not parse: %v", in, err1)
730 + }
731 + b, err2 := Parse("Caddyfile", once)
732 + if err2 != nil {
733 + t.Fatalf("formatted %q did not parse: %v", in, err2)
734 + }
735 + if !sameStructure(a, b) {
736 + t.Errorf("stray close brace merged tokens for %q: got %q", in, once)
737 + }
738 + }
739 +}
740 +
741 +func TestFormatIncompleteHeredocIdempotent(t *testing.T) {
742 + for _, in := range []string{"0 <<0", "x <<E"} {
743 + once := Format([]byte(in))
744 + twice := Format(once)
745 + if !bytes.Equal(once, twice) {
746 + t.Errorf("not idempotent for %q:\n once=%q\ntwice=%q", in, once, twice)
747 + }
748 + }
749 +}
750 +
751 +// formatTokenText returns the format-mode non-comment token-text sequence for
752 +// input, or nil and false on a lex error. It is the basis of the semantic-
753 +// preservation invariant: Format must reproduce this exact sequence. Comment
754 +// tokens are omitted because the formatter intentionally normalizes comment
755 +// whitespace (e.g. trimming a trailing space, "# " -> "#"), which is not a
756 +// semantic change.
757 +func formatTokenText(in []byte) ([]string, bool) {
758 + toks, err := Lex(in, "", LexOptions{Comments: true, Raw: true})
759 + if err != nil {
760 + return nil, false
761 + }
762 + var texts []string
763 + for _, t := range toks {
764 + if !t.isComment {
765 + texts = append(texts, t.Text)
766 + }
767 + }
768 + return texts, true
769 +}
770 +
771 +// formatAndParseLexersDisagree reports whether the format-mode lexer and the
772 +// parse-path lexer produce different token-text sequences for the input (the
773 +// parse baseline is newline-terminated to match Format's mandatory trailing
774 +// newline, and comments — which the parse path discards — are ignored on the
775 +// format side). When the two lexers already disagree before any formatting, the
776 +// input is degenerate or structurally ambiguous — an unterminated or lone
777 +// quote/backtick, an escaped quote swallowing to EOF, a "{}" or a "{"/"}" glued
778 +// to a literal word that only one path splits — and has no well-defined
779 +// formatting. Such inputs are a sanctioned divergence excluded from the
780 +// invariant. Inputs on which both lexers agree, including the stray-close-brace
781 +// bug's "0 }" and standalone braces in argument position, stay in scope.
782 +func formatAndParseLexersDisagree(in []byte) bool {
783 + fmtToks, err := Lex(in, "", LexOptions{Comments: true, Raw: true})
784 + if err != nil {
785 + return true
786 + }
787 + var fmtText []string
788 + for _, t := range fmtToks {
789 + if t.isComment {
790 + continue
791 + }
792 + fmtText = append(fmtText, t.Text)
793 + // A non-quoted token whose verbatim source ends in whitespace, contains an
794 + // escaped quote, or whose text carries a newline results from an escaped
795 + // quote ("\"") swallowing trailing whitespace to EOF. The format-mode and
796 + // parse-path lexers can agree on such a token, but Format emits the source
797 + // verbatim and then trims trailing whitespace, changing the token on
798 + // re-lex. Treat these degenerate escaped-quote tokens as a disagreement.
799 + if t.wasQuoted == 0 {
800 + r := t.Raw()
801 + if n := len(r); n > 0 && (r[n-1] == ' ' || r[n-1] == '\t' || r[n-1] == '\v' || r[n-1] == '\f' || r[n-1] == '\r' || r[n-1] == '\n') {
802 + return true
803 + }
804 + if strings.Contains(r, `\"`) || strings.Contains(r, "\\`") {
805 + return true
806 + }
807 + if strings.ContainsRune(t.Text, '\n') {
808 + return true
809 + }
810 + }
811 + }
812 + base := in
813 + if n := len(base); n == 0 || base[n-1] != '\n' {
814 + base = append(append([]byte{}, base...), '\n')
815 + }
816 + parseToks, err := Tokenize(base, "")
817 + if err != nil {
818 + return true
819 + }
820 + parseText := make([]string, len(parseToks))
821 + for i, t := range parseToks {
822 + parseText[i] = t.Text
823 + }
824 + return !reflect.DeepEqual(fmtText, parseText)
825 +}
826 +
827 +// hasHeredocOpenerShapedToken reports whether the format-mode lexer produces a
828 +// token whose text begins with "<<" (a heredoc opener shape). Such a token is a
829 +// literal word only because no newline follows it (or a separating space breaks
830 +// the heredoc), but Format's mandatory trailing newline can turn it into a
831 +// heredoc opener, so the formatted output re-lexes to a different token stream.
832 +// This degenerate opener is a sanctioned divergence excluded from the invariant;
833 +// valid heredocs (whose token text does not begin with "<<") stay in scope. It
834 +// is detected by token text, so a "<<" split by a stripped carriage return
835 +// ("<\r<") is caught too.
836 +func hasHeredocOpenerShapedToken(in []byte) bool {
837 + toks, err := Lex(in, "", LexOptions{Comments: true, Raw: true})
838 + if err != nil {
839 + return true
840 + }
841 + for _, t := range toks {
842 + if t.wasQuoted == 0 && !t.isComment && strings.HasPrefix(t.Text, "<<") {
843 + return true
844 + }
845 + }
846 + return false
847 +}
848 +
849 +// TestFormatDoesNotMutateInput guards the fallback paths, which trim the input
850 +// and append the mandatory newline. bytes.TrimSpace returns a subslice sharing
851 +// the caller's backing array, so appending to it without copying writes into
852 +// the caller's buffer. os.ReadFile returns a slice with spare capacity, so this
853 +// corrupted the input that "caddy fmt --diff" compares its output against.
854 +func TestFormatDoesNotMutateInput(t *testing.T) {
855 + cases := []string{
856 + "foo \"unterminated \n\n", // unterminated quote + trailing whitespace
857 + "a b\\ \n\n", // literal trailing backslash + whitespace
858 + ` \ \`, // fallback with significant leading whitespace
859 + " site {\n\tfoo\n} \n\n", // ordinary input with surrounding whitespace
860 + }
861 + for _, in := range cases {
862 + // Spare capacity, as os.ReadFile and pooled buffers provide.
863 + buf := append(make([]byte, 0, len(in)+8), in...)
864 + out := Format(buf)
865 + if string(buf) != in {
866 + t.Errorf("Format mutated its input:\n in %q\n after %q", in, string(buf))
867 + }
868 + if len(out) > 0 && len(buf) > 0 && &out[0] == &buf[0] {
869 + t.Errorf("Format output aliases its input for %q", in)
870 + }
871 + }
872 +}
873 +
874 +func FuzzFormatIdempotent(f *testing.F) {
875 + for _, s := range []string{
876 + "", " ", "a{\nb\n}", "site {\n\tfoo # c\n}\n", "x <<E\nhi\nE\n",
877 + // Tokens that swallow end-of-input, and a lone CR that does not.
878 + "\"\"\"", "\"\"`", "``\\\"", "0 0\r",
879 + `0 \`, ` \ \`, // dangling continuation and significant leading space
880 + } {
881 + f.Add([]byte(s))
882 + }
883 + f.Fuzz(func(t *testing.T, in []byte) {
884 + once := Format(in)
885 + twice := Format(once)
886 + if !bytes.Equal(once, twice) {
887 + t.Errorf("not idempotent:\n once=%q\ntwice=%q", once, twice)
888 + }
889 + })
890 +}
891 +
892 +func FuzzFormatNoPanic(f *testing.F) {
893 + f.Add([]byte("\x00\x00"))
894 + f.Add([]byte("`unterminated"))
895 + f.Add([]byte("x <<E\nno end marker"))
896 + f.Fuzz(func(t *testing.T, in []byte) {
897 + _ = Format(in) // must not panic
898 + })
899 +}
900 +
901 +func FuzzFormatSemanticPreserve(f *testing.F) {
902 + for _, s := range []string{"site {\n\troot * /srv\n\tfile_server\n}\n"} {
903 + f.Add([]byte(s))
904 + }
905 + f.Fuzz(func(t *testing.T, in []byte) {
906 + // The semantic-preservation invariant: formatting only changes structural
907 + // whitespace, so the format-mode token-text sequence of Format(in) must
908 + // equal that of the input. This is stronger and more robust than comparing
909 + // parse structures, which the parse path's lenient/quirky grouping of
910 + // ambiguous brace layouts (e.g. standalone braces in argument position)
911 + // would spuriously flag even though every token is preserved.
912 + //
913 + // One class is genuinely out of scope: a heredoc-opener-shaped token
914 + // ("<<MARKER") that stays a literal word only because no newline follows
915 + // it. Format's mandatory trailing newline turns it into a real heredoc
916 + // opener, so the output legitimately re-lexes to a different token stream.
917 + // Valid heredocs (whose token text does not begin with "<<") stay in scope.
918 + // Also out of scope: any input on which the format-mode and parse-path
919 + // lexers already disagree (unterminated/lone quotes, escaped quotes at EOF,
920 + // "{}" or a brace glued to a literal word) — these are degenerate or
921 + // structurally ambiguous and have no well-defined formatting.
922 + if hasHeredocOpenerShapedToken(in) || formatAndParseLexersDisagree(in) {
923 + return
924 + }
925 + // Compare the format-mode token text of a newline-terminated baseline
926 + // against that of Format's output. Format always ends its output in a
927 + // single newline (Invariant); normalizing the baseline the same way keeps
928 + // the comparison about token content, not the mandatory trailing newline.
929 + base := in
930 + if n := len(base); n == 0 || base[n-1] != '\n' {
931 + base = append(append([]byte{}, base...), '\n')
932 + }
933 + before, ok := formatTokenText(base)
934 + if !ok {
935 + return // input does not lex; Format falls back and there is nothing to preserve
936 + }
937 + out := Format(in)
938 + after, ok := formatTokenText(out)
939 + if !ok {
940 + t.Fatalf("formatted output no longer lexes\ninput=%q\nout=%q", in, out)
941 + }
942 + if !reflect.DeepEqual(before, after) {
943 + t.Errorf("Format changed the token stream\ninput=%q\nout=%q\nbefore=%q\nafter=%q", in, out, before, after)
944 + }
945 + })
946 +}
947 +
948 +// TestFormatFuzzerAngles covers tricky / pathological inputs that a fuzzer would
949 +// exercise. Every case asserts idempotency (Format(Format(x)) == Format(x)) and
950 +// that Format does not panic. The subset of clearly-idiomatic inputs also asserts
951 +// an exact expected output.
952 +func TestFormatFuzzerAngles(t *testing.T) {
953 + type tc struct {
954 + name string
955 + input string
956 + exactExpect string // non-empty → assert this exact output in addition to idempotency
957 + }
958 +
959 + cases := []tc{
960 + // ---- clearly-idiomatic: assert exact output ----
961 +
962 + {
963 + // Empty input: Format always emits a single trailing newline.
964 + name: "empty input",
965 + input: "",
966 + exactExpect: "\n",
967 + },
968 + {
969 + // Whitespace-only input collapses to the same single newline.
970 + name: "whitespace-only input",
971 + input: " \n\t\n",
972 + exactExpect: "\n",
973 + },
974 + {
975 + // A bare env-var placeholder is a literal token and must be preserved.
976 + name: "env var {$X}",
977 + input: "{$X}",
978 + exactExpect: "{$X}\n",
979 + },
980 + {
981 + // Default-value env var.
982 + name: "env var {$X:def}",
983 + input: "{$X:def}",
984 + exactExpect: "{$X:def}\n",
985 + },
986 + {
987 + // Minimal env var with only the sigil.
988 + name: "env var {$}",
989 + input: "{$}",
990 + exactExpect: "{$}\n",
991 + },
992 + {
993 + // Backtick token as the first token inside a block must indent correctly.
994 + name: "backtick as first token after {",
995 + input: "a {\n\t`foo`\n}\n",
996 + exactExpect: "a {\n\t`foo`\n}\n",
997 + },
998 + {
999 + // { } expands to a proper block.
1000 + name: "{ } empty block",
1001 + input: "route { }",
1002 + exactExpect: "route {\n}\n",
1003 + },
1004 + {
1005 + // Hash inside a double-quoted string is not a comment.
1006 + name: "hash inside double-quoted string",
1007 + input: `foo "bar#baz" quux`,
1008 + exactExpect: "foo \"bar#baz\" quux\n",
1009 + },
1010 + {
1011 + // Hash inside a heredoc body is not a comment.
1012 + name: "hash inside heredoc body",
1013 + input: "x <<END\nfoo # not a comment\nEND\n",
1014 + exactExpect: "x <<END\nfoo # not a comment\nEND\n",
1015 + },
1016 + {
1017 + // Escaped heredoc opener (\<<) is a regular token, not a real heredoc.
1018 + name: "escaped heredoc \\<<",
1019 + input: "block {\n\theredoc \\<<HEREDOC\n\trespond hello 200\n}\n",
1020 + exactExpect: "block {\n\theredoc \\<<HEREDOC\n\trespond hello 200\n}\n",
1021 + },
1022 + {
1023 + // Heredoc whose marker appears as a substring of a body line (fooEND ≠ END).
1024 + name: "heredoc marker as substring of body line",
1025 + input: "x <<END\nfooEND\nEND\n",
1026 + exactExpect: "x <<END\nfooEND\nEND\n",
1027 + },
1028 +
1029 + // ---- pathological: idempotency + no-panic only ----
1030 +
1031 + {
1032 + // Unbalanced: one more opening brace than closing braces.
1033 + name: "unbalanced braces (too many open)",
1034 + input: "a {\n\tb {\n\t\tc\n\t}\n",
1035 + },
1036 + {
1037 + // Unbalanced: stray closing brace.
1038 + name: "unbalanced braces (too many close)",
1039 + input: "a }\n",
1040 + },
1041 + {
1042 + // CRLF line endings must not break idempotency.
1043 + name: "CRLF line endings",
1044 + input: "site {\r\n\tfoo\r\n}\r\n",
1045 + },
1046 + {
1047 + // Unterminated double-quoted string.
1048 + name: "unterminated double-quote",
1049 + input: "foo \"unterminated",
1050 + },
1051 + {
1052 + // Unterminated backtick string.
1053 + name: "unterminated backtick",
1054 + input: "foo `unterminated",
1055 + },
1056 + {
1057 + // An empty quoted token followed by a lone opening quote. The first
1058 + // pass falls back and appends the mandatory newline; on the second
1059 + // pass that newline is swallowed by the still-unterminated quote, so
1060 + // the unterminated token must be detected from the token itself, not
1061 + // from whether appending a newline changes the stream.
1062 + name: "empty quoted token followed by lone quote",
1063 + input: "\"\"\"",
1064 + },
1065 + {
1066 + // Same shape with a backtick opening the unterminated token.
1067 + name: "empty quoted token followed by lone backtick",
1068 + input: "\"\"`",
1069 + },
1070 + {
1071 + // An escaped quote running to end-of-input after a closed backtick
1072 + // token. Like the cases above, the second pass must still recognize
1073 + // the swallowing token once the mandatory newline sits inside it.
1074 + name: "empty backtick token followed by escaped quote",
1075 + input: "``\\\"",
1076 + },
1077 + {
1078 + // A lone CR does not terminate a token, so it stays in the token's
1079 + // verbatim source without meaning the token swallowed end-of-input.
1080 + // Treating it as swallowed would suppress normal rendering here and
1081 + // leave the double space uncollapsed on the first pass only.
1082 + name: "double space with trailing lone CR",
1083 + input: "0 0\r",
1084 + exactExpect: "0 0\n",
1085 + },
1086 + {
1087 + // Trailing backslash (dangling escape — not a line continuation).
1088 + name: "trailing backslash",
1089 + input: "a b\\",
1090 + },
1091 + {
1092 + // UTF-8 BOM at the start of the file; the BOM should not survive into
1093 + // the formatted output (it is part of the first token's raw source but
1094 + // the formatter strips leading/trailing whitespace).
1095 + name: "UTF-8 BOM prefix",
1096 + input: "\xef\xbb\xbfsite {\n\tfoo\n}\n",
1097 + },
1098 + {
1099 + // NUL byte embedded in a token.
1100 + name: "NUL byte in token",
1101 + input: "site {\nfoo\x00bar\n}\n",
1102 + },
1103 + {
1104 + // Arbitrary control bytes in a token.
1105 + name: "control bytes in token",
1106 + input: "a\x01b\x02c",
1107 + },
1108 + {
1109 + // Lone carriage-return (CR without LF). The lexer treats \r as part of
1110 + // the token raw text but strips it from the token's text field; the raw
1111 + // bytes are emitted verbatim by the formatter.
1112 + name: "lone CR (respond hello\\rworld)",
1113 + input: "respond hello\rworld",
1114 + },
1115 + }
1116 +
1117 + for _, c := range cases {
1118 + t.Run(c.name, func(t *testing.T) {
1119 + // No-panic: the call itself must complete.
1120 + once := Format([]byte(c.input))
1121 +
1122 + // Idempotency: a second pass must reproduce the first pass exactly.
1123 + twice := Format(once)
1124 + if !bytes.Equal(once, twice) {
1125 + t.Errorf("not idempotent:\n once=%q\ntwice=%q", once, twice)
1126 + }
1127 +
1128 + // Exact-output assertion for clearly-idiomatic cases.
1129 + if c.exactExpect != "" && string(once) != c.exactExpect {
1130 + t.Errorf("unexpected output:\n got %q\n want %q", string(once), c.exactExpect)
1131 + }
1132 + })
1133 + }
1134 +}
1135 +
1136 +// TestFormattingDifferenceStableOnFormatted verifies that an already-formatted
1137 +// config produces no diff from FormattingDifference (i.e. the formatter and the
1138 +// diff checker agree that a well-formed file needs no changes).
1139 +func TestFormattingDifferenceStableOnFormatted(t *testing.T) {
1140 + in := []byte("site {\n\troot * /srv\n\tfile_server\n}\n")
1141 + formatted := Format(in)
1142 + if _, diff := FormattingDifference("Caddyfile", formatted); diff {
1143 + t.Error("FormattingDifference reported a diff on already-formatted input")
1144 + }
1145 +}
1146 +
1147 +// sameStructure compares two parses by their per-segment token Text sequences.
1148 +func sameStructure(a, b []ServerBlock) bool {
1149 + seq := func(blocks []ServerBlock) []string {
1150 + var s []string
1151 + for _, blk := range blocks {
1152 + s = append(s, "K")
1153 + for _, k := range blk.Keys {
1154 + s = append(s, k.Text)
1155 + }
1156 + for _, seg := range blk.Segments {
1157 + s = append(s, "S")
1158 + for _, tk := range seg {
1159 + s = append(s, tk.Text)
1160 + }
1161 + }
1162 + }
1163 + return s
1164 + }
1165 + return reflect.DeepEqual(seq(a), seq(b))
1166 +}
caddyconfig/caddyfile/lexer.go
+248 -42
@@ -34,6 +34,14 @@ type (
34 token Token
35 line int
36 skippedLines int
37 + lastRune rune // preceding source character, for line continuations
38 +
39 + // format-mode configuration and state (Tasks 4-6)
40 + opts LexOptions
41 + src []byte // original input, for raw slicing
42 + pos int // bytes consumed from src
43 + lastSize int // byte size of the most recently read rune
44 + tokenStart int // byte offset in src where the current token began
45 }
46
47 // Token represents a single parsable unit.
@@ -45,16 +53,42 @@ type (
53 wasQuoted rune // enclosing quote character, if any
54 heredocMarker string
55 snippetName string
56 + // Immutable physical-file ancestry, populated only during import discovery.
57 + importAncestry *importAncestor
58 +
59 + // format mode only (populated by Lex with LexOptions; zero on the parse path)
60 + raw string // verbatim source bytes of the token
61 + isComment bool // this token is a # comment spanning to end-of-line
62 + precededBySpace bool // whitespace separated this token from the previous one on the same line
63 + continuation bool // this token followed a '\'+newline line continuation
64 }
65 )
66
51 -// Tokenize takes bytes as input and lexes it into
52 -// a list of tokens that can be parsed as a Caddyfile.
53 -// Also takes a filename to fill the token's File as
54 -// the source of the tokens, which is important to
55 -// determine relative paths for `import` directives.
67 +// LexOptions configures optional, non-default lexer behavior used for
68 +// formatting. The zero value reproduces Tokenize exactly.
69 +type LexOptions struct {
70 + // Comments, when true, emits comment tokens (spanning '#' to end of line)
71 + // instead of discarding comments.
72 + Comments bool
73 + // Raw, when true, records each token's verbatim source bytes in Token.raw.
74 + Raw bool
75 +}
76 +
77 +// Tokenize takes bytes as input and lexes it into a list of tokens that can be
78 +// parsed as a Caddyfile. filename fills each token's File field.
79 func Tokenize(input []byte, filename string) ([]Token, error) {
57 - l := lexer{}
80 + return Lex(input, filename, LexOptions{})
81 +}
82 +
83 +// Lex tokenizes input like Tokenize, but with the given options. With the zero
84 +// LexOptions it is identical to Tokenize. Format-mode options (Comments, Raw)
85 +// populate additional Token state used by the formatter and are not needed for
86 +// parsing.
87 +func Lex(input []byte, filename string, opts LexOptions) ([]Token, error) {
88 + l := lexer{opts: opts}
89 + if opts.Raw {
90 + l.src = input
91 + }
92 if err := l.load(bytes.NewReader(input)); err != nil {
93 return nil, err
94 }
@@ -73,6 +107,37 @@ func Tokenize(input []byte, filename string) ([]Token, error) {
107 return tokens, nil
108 }
109
110 +// readRune reads the next rune and advances the byte position tracker.
111 +func (l *lexer) readRune() (rune, error) {
112 + ch, size, err := l.reader.ReadRune()
113 + if err == nil {
114 + l.lastSize = size
115 + l.pos += size
116 + }
117 + return ch, err
118 +}
119 +
120 +// unreadRune pushes back the last rune read by readRune, keeping pos in sync.
121 +func (l *lexer) unreadRune() error {
122 + if err := l.reader.UnreadRune(); err != nil {
123 + return err
124 + }
125 + l.pos -= l.lastSize
126 + l.lastSize = 0
127 + return nil
128 +}
129 +
130 +// rawSlice returns the verbatim source of the current token. trimLast
131 +// excludes the most recently read rune (used when a delimiter terminated
132 +// the token and should not be part of its raw span).
133 +func (l *lexer) rawSlice(trimLast bool) string {
134 + end := l.pos
135 + if trimLast {
136 + end -= l.lastSize
137 + }
138 + return string(l.src[l.tokenStart:end])
139 +}
140 +
141 // load prepares the lexer to scan an input for tokens.
142 // It discards any leading byte order mark.
143 func (l *lexer) load(input io.Reader) error {
@@ -80,12 +145,12 @@ func (l *lexer) load(input io.Reader) error {
145 l.line = 1
146
147 // discard byte order mark, if present
83 - firstCh, _, err := l.reader.ReadRune()
148 + firstCh, err := l.readRune()
149 if err != nil {
150 return err
151 }
152 if firstCh != 0xFEFF {
88 - err := l.reader.UnreadRune()
153 + err := l.unreadRune()
154 if err != nil {
155 return err
156 }
@@ -104,15 +169,27 @@ func (l *lexer) load(input io.Reader) error {
169 // may be escaped. The rest of the line is skipped
170 // if a "#" character is read in. Returns true if
171 // a token was loaded; false otherwise.
172 +// Outside quotes, a backslash preceded by a space or tab can escape a newline
173 +// to continue the logical line. A backslash attached to a token is literal.
174 func (l *lexer) next() (bool, error) {
175 var val []rune
109 - var comment, quoted, btQuoted, inHeredoc, heredocEscaped, escaped bool
176 + var comment, quoted, btQuoted, inHeredoc, heredocEscaped, escaped, continueLine bool
177 var heredocMarker string
178 + var tokenStartSet bool // whether l.tokenStart has been set for the current token
179
112 - makeToken := func(quoted rune) bool {
180 + // format-mode separator tracking: record how this token was separated from
181 + // the previous one. Only maintained when opts.Comments || opts.Raw.
182 + formatMode := l.opts.Comments || l.opts.Raw
183 + var sawSpace, sawContinuation bool
184 +
185 + makeToken := func(quoted rune, trimLast bool) bool {
186 l.token.Text = string(val)
187 l.token.wasQuoted = quoted
188 l.token.heredocMarker = heredocMarker
189 + if l.opts.Raw {
190 + l.token.raw = l.rawSlice(trimLast)
191 + }
192 + tokenStartSet = false
193 return true
194 }
195
@@ -121,27 +198,39 @@ func (l *lexer) next() (bool, error) {
198 // read some characters, make a token. If we
199 // reached EOF, then no more tokens to read.
200 // If no EOF, then we had a problem.
124 - ch, _, err := l.reader.ReadRune()
201 + ch, err := l.readRune()
202 if err != nil {
203 + if escaped && !quoted && !comment && (len(val) > 0 || !continueLine) {
204 + val = append(val, '\\')
205 + }
206 if len(val) > 0 {
207 if inHeredoc {
208 return false, fmt.Errorf("incomplete heredoc <<%s on line #%d, expected ending marker %s", heredocMarker, l.line+l.skippedLines, heredocMarker)
209 }
210
131 - return makeToken(0), nil
211 + return makeToken(0, false), nil
212 + }
213 + // An empty-bodied heredoc that hits EOF with no closing marker is
214 + // silently dropped on the parse path (existing behavior). In format
215 + // mode this loses the token and breaks idempotency, so surface it as
216 + // an error there; Format then falls back to preserving the input.
217 + if inHeredoc && formatMode {
218 + return false, fmt.Errorf("incomplete heredoc <<%s on line #%d, expected ending marker %s", heredocMarker, l.line+l.skippedLines, heredocMarker)
219 }
220 if err == io.EOF {
221 return false, nil
222 }
223 return false, err
224 }
225 + previous := l.lastRune
226 + l.lastRune = ch
227
228 // detect whether we have the start of a heredoc
229 if (!quoted && !btQuoted) && (!inHeredoc && !heredocEscaped) &&
230 len(val) > 1 && string(val[:2]) == "<<" {
231 // a space means it's just a regular token and not a heredoc
232 if ch == ' ' {
144 - return makeToken(0), nil
233 + return makeToken(0, true), nil
234 }
235
236 // skip CR, we only care about LF
@@ -196,7 +285,7 @@ func (l *lexer) next() (bool, error) {
285 // set the line counter, and make the token
286 l.line += l.skippedLines
287 l.skippedLines = 0
199 - return makeToken('<'), nil
288 + return makeToken('<', false), nil
289 }
290
291 // stay in the heredoc until we find the ending marker
@@ -206,7 +295,20 @@ func (l *lexer) next() (bool, error) {
295 // track whether we found an escape '\' for the next
296 // iteration to be contextually aware
297 if !escaped && !btQuoted && ch == '\\' {
298 + // a leading backslash starts a new token (e.g. \<<marker)
299 + if len(val) == 0 {
300 + l.token = Token{Line: l.line}
301 + if formatMode {
302 + l.token.precededBySpace = sawSpace
303 + l.token.continuation = sawContinuation
304 + }
305 + }
306 + if l.opts.Raw && len(val) == 0 && !tokenStartSet {
307 + l.tokenStart = l.pos - l.lastSize
308 + tokenStartSet = true
309 + }
310 escaped = true
311 + continueLine = previous == ' ' || previous == '\t'
312 continue
313 }
314
@@ -220,7 +322,7 @@ func (l *lexer) next() (bool, error) {
322 escaped = false
323 } else {
324 if (quoted && ch == '"') || (btQuoted && ch == '`') {
223 - return makeToken(ch), nil
325 + return makeToken(ch, false), nil
326 }
327 }
328 // allow quoted text to wrap continue on multiple lines
@@ -238,43 +340,109 @@ func (l *lexer) next() (bool, error) {
340 if ch == '\r' {
341 continue
342 }
343 + // Only a backslash preceded by a space or tab continues a line.
344 + // A backslash attached to an unquoted token is literal, including
345 + // the trailing separator in a Windows path. Preserve it at any
346 + // token boundary.
347 + if escaped && !continueLine && !comment {
348 + val = append(val, '\\')
349 + escaped = false
350 + }
351 // end of the line
352 if ch == '\n' {
243 - // newlines can be escaped to chain arguments
244 - // onto multiple lines; else, increment the line count
245 - if escaped {
246 - l.skippedLines++
247 - escaped = false
353 + // when emitting comment tokens, the newline is handled in the
354 + // comment block below (unread + return); don't process it here.
355 + if comment && l.opts.Comments {
356 + // fall through to the comment block below
357 } else {
249 - l.line += 1 + l.skippedLines
250 - l.skippedLines = 0
358 + // newlines can be escaped to chain arguments
359 + // onto multiple lines; else, increment the line count
360 + if escaped {
361 + l.skippedLines++
362 + escaped = false
363 + if formatMode {
364 + sawContinuation = true
365 + }
366 + } else {
367 + l.line += 1 + l.skippedLines
368 + l.skippedLines = 0
369 + }
370 + // comments (#) are single-line only
371 + comment = false
372 + // any kind of space means we're at the end of this token
373 + if len(val) > 0 {
374 + return makeToken(0, true), nil
375 + }
376 + continue
377 + }
378 + } else {
379 + // non-newline space: end token if we have one (unless in comment mode)
380 + if comment && l.opts.Comments {
381 + // spaces inside a comment are part of the comment text; fall through
382 + } else {
383 + if len(val) > 0 {
384 + return makeToken(0, true), nil
385 + }
386 + if formatMode {
387 + sawSpace = true
388 + }
389 + continue
390 }
252 - // comments (#) are single-line only
253 - comment = false
254 - }
255 - // any kind of space means we're at the end of this token
256 - if len(val) > 0 {
257 - return makeToken(0), nil
391 }
259 - continue
392 }
393
394 // comments must be at the start of a token,
395 // in other words, preceded by space or newline
396 if ch == '#' && len(val) == 0 {
397 comment = true
398 + if l.opts.Comments {
399 + // begin a comment token; record its start for raw capture
400 + l.tokenStart = l.pos - l.lastSize
401 + tokenStartSet = true
402 + l.token = Token{Line: l.line, isComment: true}
403 + if formatMode {
404 + l.token.precededBySpace = sawSpace
405 + l.token.continuation = sawContinuation
406 + sawSpace, sawContinuation = false, false
407 + }
408 + }
409 }
410 if comment {
411 + if l.opts.Comments {
412 + // a newline ends the comment; don't consume it (leave line accounting
413 + // to the normal whitespace path on the next iteration)
414 + if ch == '\n' {
415 + if err := l.unreadRune(); err != nil {
416 + return false, err
417 + }
418 + l.token.isComment = true
419 + return makeToken(0, false), nil
420 + }
421 + val = append(val, ch)
422 + }
423 continue
424 }
425
426 if len(val) == 0 {
427 l.token = Token{Line: l.line}
428 + if formatMode {
429 + l.token.precededBySpace = sawSpace
430 + l.token.continuation = sawContinuation
431 + sawSpace, sawContinuation = false, false
432 + }
433 if ch == '"' {
434 + if l.opts.Raw && !tokenStartSet {
435 + l.tokenStart = l.pos - l.lastSize
436 + tokenStartSet = true
437 + }
438 quoted = true
439 continue
440 }
441 if ch == '`' {
442 + if l.opts.Raw && !tokenStartSet {
443 + l.tokenStart = l.pos - l.lastSize
444 + tokenStartSet = true
445 + }
446 btQuoted = true
447 continue
448 }
@@ -290,6 +458,13 @@ func (l *lexer) next() (bool, error) {
458 escaped = false
459 }
460
461 + // first character of a new unquoted token (tokenStart may already be
462 + // set by a leading backslash; only set it if not yet recorded)
463 + if l.opts.Raw && len(val) == 0 && !tokenStartSet {
464 + l.tokenStart = l.pos - l.lastSize
465 + tokenStartSet = true
466 + }
467 +
468 val = append(val, ch)
469 }
470 }
@@ -311,10 +486,13 @@ func (l *lexer) finalizeHeredoc(val []rune, marker string) ([]rune, error) {
486 paddingToStrip := stringVal[lastNewline+1 : len(stringVal)-len(marker)]
487
488 // iterate over each line and strip the whitespace from the front
314 - var out string
489 + var out strings.Builder
490 + if lastNewline > 0 {
491 + out.Grow(lastNewline)
492 + }
493 for lineNum, lineText := range lines[:len(lines)-1] {
494 if lineText == "" || lineText == "\r" {
317 - out += "\n"
495 + out.WriteByte('\n')
496 continue
497 }
498
@@ -329,16 +507,22 @@ func (l *lexer) finalizeHeredoc(val []rune, marker string) ([]rune, error) {
507
508 // strip, then append the line, with the newline, to the output.
509 // also removes all "\r" because Windows.
332 - out += strings.ReplaceAll(lineText[len(paddingToStrip):]+"\n", "\r", "")
510 + for _, ch := range lineText[len(paddingToStrip):] {
511 + if ch != '\r' {
512 + out.WriteRune(ch)
513 + }
514 + }
515 + out.WriteByte('\n')
516 }
517
518 // Remove the trailing newline from the loop
336 - if len(out) > 0 && out[len(out)-1] == '\n' {
337 - out = out[:len(out)-1]
519 + result := out.String()
520 + if len(result) > 0 && result[len(result)-1] == '\n' {
521 + result = result[:len(result)-1]
522 }
523
524 // return the final value
341 - return []rune(out), nil
525 + return []rune(result), nil
526 }
527
528 // Quoted returns true if the token was enclosed in quotes
@@ -347,6 +531,23 @@ func (t Token) Quoted() bool {
531 return t.wasQuoted > 0
532 }
533
534 +// Raw returns the verbatim source text of the token as it appeared in the
535 +// input, including quotes, backticks, escapes, and heredoc framing. It is only
536 +// populated when the token was produced by Lex in format mode; otherwise it
537 +// falls back to the processed Text.
538 +func (t Token) Raw() string {
539 + if t.raw != "" {
540 + return t.raw
541 + }
542 + return t.Text
543 +}
544 +
545 +// IsComment returns true if the token is a comment. Comment tokens are only
546 +// produced by Lex in format mode.
547 +func (t Token) IsComment() bool {
548 + return t.isComment
549 +}
550 +
551 // isOpenCurlyBrace returns true if the token is a structural (unquoted) open curly brace.
552 func isOpenCurlyBrace(t Token) bool {
553 return t.Text == "{" && t.wasQuoted == 0
@@ -372,13 +573,18 @@ func (t Token) NumLineBreaks() int {
573 // Clone returns a deep copy of the token.
574 func (t Token) Clone() Token {
575 return Token{
375 - File: t.File,
376 - imports: append([]string{}, t.imports...),
377 - Line: t.Line,
378 - Text: t.Text,
379 - wasQuoted: t.wasQuoted,
380 - heredocMarker: t.heredocMarker,
381 - snippetName: t.snippetName,
576 + File: t.File,
577 + imports: append([]string{}, t.imports...),
578 + Line: t.Line,
579 + Text: t.Text,
580 + wasQuoted: t.wasQuoted,
581 + heredocMarker: t.heredocMarker,
582 + snippetName: t.snippetName,
583 + importAncestry: t.importAncestry,
584 + raw: t.raw,
585 + isComment: t.isComment,
586 + precededBySpace: t.precededBySpace,
587 + continuation: t.continuation,
588 }
589 }
590
caddyconfig/caddyfile/lexer_test.go
+226 -3
@@ -15,6 +15,7 @@
15 package caddyfile
16
17 import (
18 + "reflect"
19 "testing"
20 )
21
@@ -123,7 +124,7 @@ func TestLexer(t *testing.T) {
124 },
125 },
126 {
126 - input: []byte("An escaped newline\\\noutside quotes"),
127 + input: []byte("An escaped newline \\\noutside quotes"),
128 expected: []Token{
129 {Line: 1, Text: "An"},
130 {Line: 1, Text: "escaped"},
@@ -133,7 +134,7 @@ func TestLexer(t *testing.T) {
134 },
135 },
136 {
136 - input: []byte("line1\\\nescaped\nline2\nline3"),
137 + input: []byte("line1 \\\nescaped\nline2\nline3"),
138 expected: []Token{
139 {Line: 1, Text: "line1"},
140 {Line: 1, Text: "escaped"},
@@ -142,7 +143,7 @@ func TestLexer(t *testing.T) {
143 },
144 },
145 {
145 - input: []byte("line1\\\nescaped1\\\nescaped2\nline4\nline5"),
146 + input: []byte("line1 \\\nescaped1 \\\nescaped2\nline4\nline5"),
147 expected: []Token{
148 {Line: 1, Text: "line1"},
149 {Line: 1, Text: "escaped1"},
@@ -521,6 +522,70 @@ EOF`),
522 }
523 }
524
525 +func TestTokenFormatModeAccessors(t *testing.T) {
526 + tok := Token{Text: "hello", raw: `"hello"`, isComment: false}
527 + if tok.Raw() != `"hello"` {
528 + t.Errorf("Raw() = %q, want %q", tok.Raw(), `"hello"`)
529 + }
530 + // falls back to Text when raw not captured
531 + if (Token{Text: "hi"}).Raw() != "hi" {
532 + t.Errorf("Raw() fallback = %q, want %q", (Token{Text: "hi"}).Raw(), "hi")
533 + }
534 + c := Token{Text: "# note", raw: "# note", isComment: true}
535 + if !c.IsComment() {
536 + t.Error("IsComment() = false, want true")
537 + }
538 + // Clone copies the new fields
539 + clone := c.Clone()
540 + if clone.raw != c.raw || clone.isComment != c.isComment {
541 + t.Errorf("Clone did not copy format-mode fields: %+v", clone)
542 + }
543 +}
544 +
545 +func TestLexRawCapture(t *testing.T) {
546 + cases := []struct {
547 + in string
548 + text string // processed Text of the single token
549 + wantRaw string // verbatim source of that token
550 + }{
551 + {`hello`, `hello`, `hello`},
552 + {`"a \"b\" "`, `a "b" `, `"a \"b\" "`},
553 + {"`raw \"x\"`", `raw "x"`, "`raw \"x\"`"},
554 + {`\<<NOTHEREDOC`, `<<NOTHEREDOC`, `\<<NOTHEREDOC`},
555 + }
556 + for _, c := range cases {
557 + toks, err := Lex([]byte(c.in), "T", LexOptions{Raw: true})
558 + if err != nil {
559 + t.Fatalf("%q: %v", c.in, err)
560 + }
561 + if len(toks) != 1 {
562 + t.Fatalf("%q: got %d tokens, want 1", c.in, len(toks))
563 + }
564 + if toks[0].Text != c.text {
565 + t.Errorf("%q: Text = %q, want %q", c.in, toks[0].Text, c.text)
566 + }
567 + if toks[0].Raw() != c.wantRaw {
568 + t.Errorf("%q: Raw() = %q, want %q", c.in, toks[0].Raw(), c.wantRaw)
569 + }
570 + }
571 +}
572 +
573 +func TestLexRawCaptureHeredoc(t *testing.T) {
574 + in := "x <<END\n\thello\n\tEND"
575 + toks, err := Lex([]byte(in), "T", LexOptions{Raw: true})
576 + if err != nil {
577 + t.Fatal(err)
578 + }
579 + // token 0 is "x"; token 1 is the heredoc
580 + if len(toks) != 2 {
581 + t.Fatalf("got %d tokens, want 2", len(toks))
582 + }
583 + // heredoc Text is the de-indented body; Raw is the full framing verbatim.
584 + if toks[1].Raw() != "<<END\n\thello\n\tEND" {
585 + t.Errorf("heredoc Raw() = %q, want %q", toks[1].Raw(), "<<END\n\thello\n\tEND")
586 + }
587 +}
588 +
589 func lexerCompare(t *testing.T, n int, expected, actual []Token) {
590 if len(expected) != len(actual) {
591 t.Fatalf("Test case %d: expected %d token(s) but got %d", n, len(expected), len(actual))
@@ -539,3 +604,161 @@ func lexerCompare(t *testing.T, n int, expected, actual []Token) {
604 }
605 }
606 }
607 +
608 +func TestLexLineContinuationRequiresSeparateBackslash(t *testing.T) {
609 + for _, tc := range []struct {
610 + name string
611 + input string
612 + want []Token
613 + }{
614 + {"space", "foo \\\nbar", []Token{{Line: 1, Text: "foo"}, {Line: 1, Text: "bar"}}},
615 + {"tab and CRLF", "foo\t\\\r\nbar", []Token{{Line: 1, Text: "foo"}, {Line: 1, Text: "bar"}}},
616 + {"attached", "foo\\\nbar", []Token{{Line: 1, Text: `foo\`}, {Line: 2, Text: "bar"}}},
617 + {"after quoted token", "\"foo\"\\\nbar", []Token{{Line: 1, Text: "foo"}, {Line: 1, Text: `\`}, {Line: 2, Text: "bar"}}},
618 + {"after backtick token", "`foo`\\\nbar", []Token{{Line: 1, Text: "foo"}, {Line: 1, Text: `\`}, {Line: 2, Text: "bar"}}},
619 + {"after quoted token with space", "\"foo\" \\\nbar", []Token{{Line: 1, Text: "foo"}, {Line: 1, Text: "bar"}}},
620 + {"start of file", "\\\nbar", []Token{{Line: 1, Text: `\`}, {Line: 2, Text: "bar"}}},
621 + {"indented continuation", "foo\n\t\\\nbar", []Token{{Line: 1, Text: "foo"}, {Line: 2, Text: "bar"}}},
622 + {"Windows CRLF", "root * C:\\site\\\r\nfile_server", []Token{{Line: 1, Text: "root"}, {Line: 1, Text: "*"}, {Line: 1, Text: `C:\site\`}, {Line: 2, Text: "file_server"}}},
623 + {"path before space", "C:\\site\\ next", []Token{{Line: 1, Text: `C:\site\`}, {Line: 1, Text: "next"}}},
624 + {"path at EOF", `C:\site\`, []Token{{Line: 1, Text: `C:\site\`}}},
625 + {"standalone backslash at EOF", `foo \`, []Token{{Line: 1, Text: "foo"}}},
626 + {"paired backslashes", "foo\\\\\nbar", []Token{{Line: 1, Text: `foo\\`}, {Line: 2, Text: "bar"}}},
627 + {"backtick path", "`C:\\site\\`\nbar", []Token{{Line: 1, Text: `C:\site\`}, {Line: 2, Text: "bar"}}},
628 + } {
629 + t.Run(tc.name, func(t *testing.T) {
630 + for _, opts := range []LexOptions{{}, {Raw: true}, {Comments: true}, {Raw: true, Comments: true}} {
631 + tokens, err := Lex([]byte(tc.input), "", opts)
632 + if err != nil {
633 + t.Fatal(err)
634 + }
635 + if len(tokens) != len(tc.want) {
636 + t.Fatalf("options %+v: got %d tokens, want %d", opts, len(tokens), len(tc.want))
637 + }
638 + for i, want := range tc.want {
639 + if tokens[i].Text != want.Text || tokens[i].Line != want.Line {
640 + t.Errorf("options %+v, token %d: got %q on line %d, want %q on line %d", opts, i, tokens[i].Text, tokens[i].Line, want.Text, want.Line)
641 + }
642 + }
643 + }
644 + })
645 + }
646 +}
647 +
648 +func TestLexComments(t *testing.T) {
649 + toks, err := Lex([]byte("foo # hi there\nbar"), "T", LexOptions{Comments: true, Raw: true})
650 + if err != nil {
651 + t.Fatal(err)
652 + }
653 + // foo | # hi there | bar
654 + if len(toks) != 3 {
655 + t.Fatalf("got %d tokens, want 3: %+v", len(toks), toks)
656 + }
657 + if !toks[1].IsComment() || toks[1].Text != "# hi there" {
658 + t.Errorf("token 1 = %q (comment=%v), want comment %q", toks[1].Text, toks[1].IsComment(), "# hi there")
659 + }
660 + if toks[2].Text != "bar" {
661 + t.Errorf("token 2 = %q, want bar", toks[2].Text)
662 + }
663 +}
664 +
665 +func TestLexHashMidTokenIsLiteral(t *testing.T) {
666 + toks, err := Lex([]byte("e#f redir /a/#/b"), "T", LexOptions{Comments: true})
667 + if err != nil {
668 + t.Fatal(err)
669 + }
670 + texts := []string{}
671 + for _, tk := range toks {
672 + texts = append(texts, tk.Text)
673 + }
674 + want := []string{"e#f", "redir", "/a/#/b"}
675 + if !reflect.DeepEqual(texts, want) {
676 + t.Errorf("texts = %v, want %v", texts, want)
677 + }
678 + for _, tk := range toks {
679 + if tk.IsComment() {
680 + t.Errorf("unexpected comment token %q", tk.Text)
681 + }
682 + }
683 +}
684 +
685 +func TestLexCommentsDisabledDiscards(t *testing.T) {
686 + toks, _ := Lex([]byte("foo # hi\nbar"), "T", LexOptions{})
687 + if len(toks) != 2 {
688 + t.Fatalf("got %d tokens, want 2 (comment discarded)", len(toks))
689 + }
690 +}
691 +
692 +func TestLexSeparatorKind(t *testing.T) {
693 + // glued comment after a closing quote: "x"#c -> [ "x", "#c" ] glued
694 + toks, _ := Lex([]byte(`"x"#c`), "T", LexOptions{Comments: true, Raw: true})
695 + if len(toks) != 2 || !toks[1].IsComment() {
696 + t.Fatalf("got %+v, want string + comment", toks)
697 + }
698 + if toks[1].precededBySpace {
699 + t.Error(`"x"#c: comment should be glued (precededBySpace=false)`)
700 + }
701 +
702 + toks2, _ := Lex([]byte(`"x" #c`), "T", LexOptions{Comments: true, Raw: true})
703 + if !toks2[1].precededBySpace {
704 + t.Error(`"x" #c: comment should be space-separated (precededBySpace=true)`)
705 + }
706 +
707 + // line continuation: `foo bar \<nl>baz` -> baz has continuation=true
708 + toks3, _ := Lex([]byte("foo bar \\\nbaz"), "T", LexOptions{Raw: true})
709 + last := toks3[len(toks3)-1]
710 + if last.Text != "baz" || !last.continuation {
711 + t.Errorf("continuation token = %q continuation=%v, want baz/true", last.Text, last.continuation)
712 + }
713 +}
714 +
715 +func lexTexts(t *testing.T, in string, opts LexOptions) []string {
716 + t.Helper()
717 + toks, err := Lex([]byte(in), "T", opts)
718 + if err != nil {
719 + t.Fatal(err)
720 + }
721 + out := []string{}
722 + for _, tk := range toks {
723 + out = append(out, tk.Text)
724 + }
725 + return out
726 +}
727 +
728 +func TestLexOptionsPreserveTokenBoundaries(t *testing.T) {
729 + cases := []struct {
730 + in string
731 + want []string
732 + }{
733 + {"example.com{", []string{"example.com{"}},
734 + {"{$A}{", []string{"{$A}{"}}, // placeholder tail: not split
735 + {"foo{bar}", []string{"foo{bar}"}}, // placeholder: not split
736 + {"route {}", []string{"route", "{}"}},
737 + {"route { }", []string{"route", "{", "}"}},
738 + {"a { b {} }", []string{"a", "{", "b", "{}", "}"}},
739 + {"site {\n\troot\n}", []string{"site", "{", "root", "}"}},
740 + }
741 + for _, c := range cases {
742 + for _, opts := range []LexOptions{{Raw: true}, {Comments: true}, {Raw: true, Comments: true}} {
743 + got := lexTexts(t, c.in, opts)
744 + if !reflect.DeepEqual(got, c.want) {
745 + t.Errorf("%q with %+v: got %v, want %v", c.in, opts, got, c.want)
746 + }
747 + }
748 + }
749 +}
750 +
751 +func TestLexEqualsTokenizeWithZeroOptions(t *testing.T) {
752 + in := []byte("site {\n\troot * /srv\n\tfile_server\n}\n")
753 + a, err := Tokenize(in, "Caddyfile")
754 + if err != nil {
755 + t.Fatal(err)
756 + }
757 + b, err := Lex(in, "Caddyfile", LexOptions{})
758 + if err != nil {
759 + t.Fatal(err)
760 + }
761 + if !reflect.DeepEqual(a, b) {
762 + t.Errorf("Lex(zero opts) != Tokenize\n a=%+v\n b=%+v", a, b)
763 + }
764 +}
caddyconfig/caddyfile/parse.go
+97 -35
@@ -110,6 +110,13 @@ func replaceEnvVars(input []byte) []byte {
110 return input
111 }
112
113 +// importAncestor tracks active physical files during formatting import discovery.
114 +// Nodes are immutable and shared by all tokens from one import expansion.
115 +type importAncestor struct {
116 + info os.FileInfo
117 + parent *importAncestor
118 +}
119 +
120 type parser struct {
121 *Dispenser
122 block ServerBlock // current server block being parsed
@@ -117,6 +124,11 @@ type parser struct {
124 definedSnippets map[string][]Token
125 nesting int
126 importGraph importGraph
127 + // importObserver, when set, runs after an imported file is opened and read but
128 + // before its tokens are inserted. Returning true skips token insertion. This
129 + // internal hook lets FormatImports observe the parser's actual import execution
130 + // and suppress physical-file cycles without duplicating parser semantics.
131 + importObserver func(string, os.FileInfo, []byte, *importAncestor) (bool, error)
132 }
133
134 func (p *parser) parseAll() ([]ServerBlock, error) {
@@ -425,26 +437,9 @@ func (p *parser) doImport(nesting int) error {
437 // make path relative to the file of the _token_ being processed rather
438 // than current working directory (issue #867) and then use glob to get
439 // list of matching filenames
428 - absFile, err := caddy.FastAbs(p.Dispenser.File())
429 - if err != nil {
430 - return p.Errf("Failed to get absolute path of file: %s: %v", p.Dispenser.File(), err)
431 - }
432 -
433 - var matches []string
434 - var globPattern string
435 - if !filepath.IsAbs(importPattern) {
436 - globPattern = filepath.Join(filepath.Dir(absFile), importPattern)
437 - } else {
438 - globPattern = importPattern
439 - }
440 - if strings.Count(globPattern, "*") > 1 || strings.Count(globPattern, "?") > 1 ||
441 - (strings.Contains(globPattern, "[") && strings.Contains(globPattern, "]")) {
442 - // See issue #2096 - a pattern with many glob expansions can hang for too long
443 - return p.Errf("Glob pattern may only contain one wildcard (*), but has others: %s", globPattern)
444 - }
445 - matches, err = filepath.Glob(globPattern)
440 + matches, globPattern, err := resolveImportGlob(p.Dispenser.File(), importPattern)
441 if err != nil {
447 - return p.Errf("Failed to use import pattern %s: %v", importPattern, err)
442 + return p.WrapErr(err)
443 }
444 if len(matches) == 0 {
445 if strings.ContainsAny(globPattern, "*?[]") {
@@ -460,20 +455,6 @@ func (p *parser) doImport(nesting int) error {
455 }
456 return p.Errf("File to import not found: %s", importPattern)
457 }
463 - } else {
464 - // See issue #5295 - should skip any files that start with a . when iterating over them.
465 - sep := string(filepath.Separator)
466 - segGlobPattern := strings.Split(globPattern, sep)
467 - if strings.HasPrefix(segGlobPattern[len(segGlobPattern)-1], "*") {
468 - var tmpMatches []string
469 - for _, m := range matches {
470 - seg := strings.Split(m, sep)
471 - if !strings.HasPrefix(seg[len(seg)-1], ".") {
472 - tmpMatches = append(tmpMatches, m)
473 - }
474 - }
475 - matches = tmpMatches
476 - }
458 }
459
460 // collect all the imported tokens
@@ -511,6 +492,11 @@ func (p *parser) doImport(nesting int) error {
492 // golang for range slice return a copy of value
493 // similarly, append also copy value
494 for i, token := range importedTokens {
495 + if p.importObserver != nil && p.definedSnippets[importPattern] != nil {
496 + // A snippet executes in its caller's ancestry, not the ancestry
497 + // of the file expansion where it was originally declared.
498 + token.importAncestry = p.Token().importAncestry
499 + }
500 // update the token's imports to refer to import directive filename, line number and snippet name if there is one
501 if token.snippetName != "" {
502 token.imports = append(token.imports, fmt.Sprintf("%s:%d (import %s)", p.File(), p.Line(), token.snippetName))
@@ -568,7 +554,13 @@ func (p *parser) doImport(nesting int) error {
554 if maybeSnippet {
555 tokensCopy = append(tokensCopy, token)
556 } else {
557 + start := len(tokensCopy)
558 tokensCopy = append(tokensCopy, tokensToAdd...)
559 + if p.importObserver != nil {
560 + for i := range tokensToAdd {
561 + tokensCopy[start+i].importAncestry = token.importAncestry
562 + }
563 + }
564 }
565 continue
566 }
@@ -613,6 +605,9 @@ func (p *parser) expandImportsInBlock(tokens []Token) ([]Token, error) {
605 Dispenser: NewDispenser(tokens),
606 definedSnippets: p.definedSnippets,
607 importGraph: p.importGraph,
608 + // Carry the observer so callers watching import execution (such as
609 + // FormatImports) also see files imported from inside a named route.
610 + importObserver: p.importObserver,
611 }
612
613 // Loop through the tokens. We only care about import directives that
@@ -642,9 +637,11 @@ func (p *parser) doSingleImport(importFile string) ([]Token, error) {
637 }
638 defer file.Close()
639
645 - if info, err := file.Stat(); err != nil {
640 + info, err := file.Stat()
641 + if err != nil {
642 return nil, p.Errf("Could not import %s: %v", importFile, err)
647 - } else if info.IsDir() {
643 + }
644 + if info.IsDir() {
645 return nil, p.Errf("Could not import %s: is a directory", importFile)
646 }
647
@@ -652,6 +649,18 @@ func (p *parser) doSingleImport(importFile string) ([]Token, error) {
649 if err != nil {
650 return nil, p.Errf("Could not read imported file %s: %v", importFile, err)
651 }
652 + // Notify only after a successful read so observers receive bytes and metadata
653 + // from the same open descriptor. A skipped cyclic import contributes no
654 + // tokens at this occurrence.
655 + if p.importObserver != nil {
656 + skip, err := p.importObserver(importFile, info, input, p.Token().importAncestry)
657 + if err != nil {
658 + return nil, p.Errf("Could not import %s: %v", importFile, err)
659 + }
660 + if skip {
661 + return nil, nil
662 + }
663 + }
664
665 // only warning in case of empty files
666 if len(input) == 0 || len(strings.TrimSpace(string(input))) == 0 {
@@ -670,8 +679,13 @@ func (p *parser) doSingleImport(importFile string) ([]Token, error) {
679 if err != nil {
680 return nil, p.Errf("Failed to get absolute path of file: %s: %v", importFile, err)
681 }
682 + var ancestry *importAncestor
683 + if p.importObserver != nil {
684 + ancestry = &importAncestor{info: info, parent: p.Token().importAncestry}
685 + }
686 for i := range importedTokens {
687 importedTokens[i].File = filename
688 + importedTokens[i].importAncestry = ancestry
689 }
690
691 return importedTokens, nil
@@ -865,6 +879,54 @@ func (s Segment) Directive() string {
879 return ""
880 }
881
882 +// resolveImportGlob resolves importPattern relative to importerFile into a
883 +// list of matching file paths. It handles making the pattern absolute (issue
884 +// #867), enforces the single-wildcard limit (issue #2096), runs filepath.Glob,
885 +// and skips dot-files for glob matches (issue #5295). It returns the matched
886 +// paths, the resolved glob pattern (so callers can distinguish glob vs literal
887 +// for warning/error purposes), and any hard error. Empty-match handling (the
888 +// Warn vs "not found" decision) is left to the caller.
889 +func resolveImportGlob(importerFile, importPattern string) (matches []string, globPattern string, err error) {
890 + absFile, err := caddy.FastAbs(importerFile)
891 + if err != nil {
892 + return nil, "", fmt.Errorf("failed to get absolute path of file: %s: %v", importerFile, err)
893 + }
894 +
895 + if !filepath.IsAbs(importPattern) {
896 + globPattern = filepath.Join(filepath.Dir(absFile), importPattern)
897 + } else {
898 + globPattern = importPattern
899 + }
900 +
901 + if strings.Count(globPattern, "*") > 1 || strings.Count(globPattern, "?") > 1 ||
902 + (strings.Contains(globPattern, "[") && strings.Contains(globPattern, "]")) {
903 + // See issue #2096 - a pattern with many glob expansions can hang for too long
904 + return nil, globPattern, fmt.Errorf("glob pattern may only contain one wildcard (*), but has others: %s", globPattern)
905 + }
906 +
907 + matches, err = filepath.Glob(globPattern)
908 + if err != nil {
909 + return nil, globPattern, fmt.Errorf("failed to use import pattern %s: %v", importPattern, err)
910 + }
911 +
912 + // See issue #5295 - skip files that start with a . when the last path
913 + // segment of the glob pattern starts with *.
914 + sep := string(filepath.Separator)
915 + segGlobPattern := strings.Split(globPattern, sep)
916 + if strings.HasPrefix(segGlobPattern[len(segGlobPattern)-1], "*") {
917 + var filtered []string
918 + for _, m := range matches {
919 + seg := strings.Split(m, sep)
920 + if !strings.HasPrefix(seg[len(seg)-1], ".") {
921 + filtered = append(filtered, m)
922 + }
923 + }
924 + matches = filtered
925 + }
926 +
927 + return matches, globPattern, nil
928 +}
929 +
930 // spanOpen and spanClose are used to bound spans that
931 // contain the name of an environment variable.
932 var (
caddyconfig/caddyfile/parse_test.go
+15
@@ -1110,6 +1110,21 @@ func TestImportedSnippetDefinitionRetainsBlockPlaceholder(t *testing.T) {
1110 }
1111 }
1112
1113 +func TestResolveImportGlob(t *testing.T) {
1114 + dir := t.TempDir()
1115 + os.WriteFile(filepath.Join(dir, "a.caddy"), []byte("a\n"), 0o600)
1116 + os.WriteFile(filepath.Join(dir, "b.caddy"), []byte("b\n"), 0o600)
1117 + os.WriteFile(filepath.Join(dir, ".hidden.caddy"), []byte("h\n"), 0o600)
1118 + importer := filepath.Join(dir, "Caddyfile")
1119 + matches, _, err := resolveImportGlob(importer, "*.caddy")
1120 + if err != nil {
1121 + t.Fatal(err)
1122 + }
1123 + if len(matches) != 2 { // dotfile skipped for glob
1124 + t.Errorf("got %d matches, want 2: %v", len(matches), matches)
1125 + }
1126 +}
1127 +
1128 func testParser(input string) parser {
1129 return parser{Dispenser: NewTestDispenser(input)}
1130 }
caddyconfig/httpcaddyfile/httptype_test.go
+25
@@ -9,6 +9,31 @@ import (
9 "github.com/caddyserver/caddy/v2/modules/caddyhttp"
10 )
11
12 +func TestWindowsRootTrailingBackslash(t *testing.T) {
13 + adapter := caddyfile.Adapter{ServerType: ServerType{}}
14 + input := "http://localhost {\nroot * C:\\site\\\nrespond ok\n}\n"
15 + rootJSON, err := json.Marshal(`C:\site\`)
16 + if err != nil {
17 + t.Fatal(err)
18 + }
19 + for _, in := range []string{input, strings.ReplaceAll(input, "\n", "\r\n")} {
20 + before, _, err := adapter.Adapt([]byte(in), nil)
21 + if err != nil {
22 + t.Fatal(err)
23 + }
24 + if !strings.Contains(string(before), `"root":`+string(rootJSON)) {
25 + t.Fatalf("Windows root not preserved in adapted config: %s", before)
26 + }
27 + after, _, err := adapter.Adapt(caddyfile.Format([]byte(in)), nil)
28 + if err != nil {
29 + t.Fatal(err)
30 + }
31 + if string(before) != string(after) {
32 + t.Fatalf("formatting changed adapted JSON:\nbefore: %s\nafter: %s", before, after)
33 + }
34 + }
35 +}
36 +
37 func TestMatcherSyntax(t *testing.T) {
38 for i, tc := range []struct {
39 input string
cmd/commandfuncs.go
+181
@@ -29,6 +29,7 @@ import (
29 "net/http"
30 "os"
31 "os/exec"
32 + "path/filepath"
33 "runtime"
34 "runtime/debug"
35 "strings"
@@ -693,6 +694,16 @@ func cmdFmt(fl Flags) (int, error) {
694 } else if configFile == "" {
695 configFile = configFlag
696 }
697 +
698 + useImports := fl.Bool("imports")
699 +
700 + // --imports cannot be used with stdin: there is no source directory to
701 + // resolve import paths against.
702 + if useImports && configFile == "-" {
703 + return caddy.ExitCodeFailedStartup,
704 + fmt.Errorf("cannot use --imports when reading from stdin (no source directory to resolve imports against)")
705 + }
706 +
707 // as a special case, read from stdin if the file name is "-"
708 if configFile == "-" {
709 input, err := io.ReadAll(os.Stdin)
@@ -704,6 +715,63 @@ func cmdFmt(fl Flags) (int, error) {
715 return caddy.ExitCodeSuccess, nil
716 }
717
718 + // --imports: format the root file and all files it references via import
719 + // directives, then write or print each one.
720 + if useImports {
721 + results, err := caddyfile.FormatImports(configFile, caddyfile.FormatOptions{})
722 + if err != nil {
723 + return caddy.ExitCodeFailedStartup, fmt.Errorf("formatting imports: %v", err)
724 + }
725 +
726 + if fl.Bool("overwrite") {
727 + if err := overwriteFormattedFiles(results); err != nil {
728 + return caddy.ExitCodeFailedStartup, err
729 + }
730 + return caddy.ExitCodeSuccess, nil
731 + }
732 +
733 + originals := make([][]byte, len(results))
734 + var formattingWarning caddyconfig.Warning
735 + for i, f := range results {
736 + original, readErr := os.ReadFile(f.Path)
737 + if readErr != nil {
738 + return caddy.ExitCodeFailedStartup, fmt.Errorf("reading file for formatting comparison %s: %v", f.Path, readErr)
739 + }
740 + originals[i] = original
741 + if warning, different := caddyfile.FormattingDifference(f.Path, original); different && formattingWarning.File == "" {
742 + formattingWarning = warning
743 + }
744 + }
745 +
746 + for i, f := range results {
747 + fmt.Printf("# %s\n", f.Path)
748 + if fl.Bool("diff") {
749 + diff := difflib.Diff(
750 + strings.Split(string(originals[i]), "\n"),
751 + strings.Split(string(f.Content), "\n"))
752 + for _, d := range diff {
753 + switch d.Delta {
754 + case difflib.Common:
755 + fmt.Printf(" %s\n", d.Payload)
756 + case difflib.LeftOnly:
757 + fmt.Printf("- %s\n", d.Payload)
758 + case difflib.RightOnly:
759 + fmt.Printf("+ %s\n", d.Payload)
760 + }
761 + }
762 + } else {
763 + fmt.Print(string(f.Content))
764 + }
765 + }
766 + if formattingWarning.File != "" {
767 + return caddy.ExitCodeFailedStartup, fmt.Errorf(`%s:%d: Caddyfile input is not formatted; Tip: use '--overwrite' to update your Caddyfile in-place instead of previewing it. Consult '--help' for more options`,
768 + formattingWarning.File,
769 + formattingWarning.Line,
770 + )
771 + }
772 + return caddy.ExitCodeSuccess, nil
773 + }
774 +
775 input, err := os.ReadFile(configFile)
776 if err != nil {
777 return caddy.ExitCodeFailedStartup,
@@ -747,6 +815,119 @@ func cmdFmt(fl Flags) (int, error) {
815 return caddy.ExitCodeSuccess, nil
816 }
817
818 +type formattedFileOverwrite struct {
819 + path string
820 + content []byte
821 + original []byte
822 + info fs.FileInfo
823 +}
824 +
825 +func overwriteFormattedFiles(results []caddyfile.FormattedFile) error {
826 + files := make([]formattedFileOverwrite, 0, len(results))
827 + for _, result := range results {
828 + path, err := filepath.EvalSymlinks(filepath.Dir(result.Path))
829 + if err != nil {
830 + return fmt.Errorf("preparing formatted file %s: resolving parent directory: %v", result.Path, err)
831 + }
832 + path = filepath.Join(path, filepath.Base(result.Path))
833 +
834 + file, err := os.Open(path)
835 + if err != nil {
836 + return fmt.Errorf("preparing formatted file %s: %v", result.Path, err)
837 + }
838 + info, statErr := file.Stat()
839 + pathInfo, lstatErr := os.Lstat(path)
840 + original, readErr := io.ReadAll(file)
841 + closeErr := file.Close()
842 + if statErr != nil {
843 + return fmt.Errorf("preparing formatted file %s: stating open file: %v", result.Path, statErr)
844 + }
845 + if lstatErr != nil {
846 + return fmt.Errorf("preparing formatted file %s: checking destination: %v", result.Path, lstatErr)
847 + }
848 + if readErr != nil {
849 + return fmt.Errorf("preparing formatted file %s: reading destination: %v", result.Path, readErr)
850 + }
851 + if closeErr != nil {
852 + return fmt.Errorf("preparing formatted file %s: %v", result.Path, closeErr)
853 + }
854 + if pathInfo.Mode()&os.ModeSymlink != 0 || !pathInfo.Mode().IsRegular() || !os.SameFile(info, pathInfo) {
855 + return fmt.Errorf("refusing to overwrite %s: destination is not the regular file that was opened", result.Path)
856 + }
857 + if !bytes.Equal(caddyfile.FormatWithOptions(original, caddyfile.FormatOptions{}), result.Content) {
858 + return fmt.Errorf("refusing to overwrite %s: file changed after it was formatted", result.Path)
859 + }
860 + files = append(files, formattedFileOverwrite{path: path, content: result.Content, original: original, info: info})
861 + }
862 +
863 + for i, file := range files {
864 + if err := atomicReplaceFormattedFile(file); err != nil {
865 + if i > 0 {
866 + return fmt.Errorf("overwriting formatted file %s after replacing %d of %d files: %v; configuration files may be partially updated", file.path, i, len(files), err)
867 + }
868 + return fmt.Errorf("overwriting formatted file %s before replacing any files: %v", file.path, err)
869 + }
870 + }
871 + return nil
872 +}
873 +
874 +func atomicReplaceFormattedFile(file formattedFileOverwrite) (err error) {
875 + temp, err := os.CreateTemp(filepath.Dir(file.path), ".caddy-fmt-*")
876 + if err != nil {
877 + return err
878 + }
879 + tempPath := temp.Name()
880 + defer func() {
881 + _ = temp.Close()
882 + if err != nil {
883 + _ = os.Remove(tempPath)
884 + }
885 + }()
886 +
887 + if _, err = temp.Write(file.content); err != nil {
888 + return err
889 + }
890 + if err = temp.Chmod(file.info.Mode().Perm()); err != nil {
891 + return err
892 + }
893 + if err = temp.Sync(); err != nil {
894 + return err
895 + }
896 + if err = temp.Close(); err != nil {
897 + return err
898 + }
899 +
900 + destination, err := os.Open(file.path)
901 + if err != nil {
902 + return fmt.Errorf("reopening destination: %v", err)
903 + }
904 + destinationInfo, statErr := destination.Stat()
905 + pathInfo, lstatErr := os.Lstat(file.path)
906 + current, readErr := io.ReadAll(destination)
907 + closeErr := destination.Close()
908 + if statErr != nil {
909 + return fmt.Errorf("restating destination: %v", statErr)
910 + }
911 + if lstatErr != nil {
912 + return fmt.Errorf("rechecking destination: %v", lstatErr)
913 + }
914 + if readErr != nil {
915 + return fmt.Errorf("rereading destination: %v", readErr)
916 + }
917 + if closeErr != nil {
918 + return fmt.Errorf("closing destination: %v", closeErr)
919 + }
920 + if pathInfo.Mode()&os.ModeSymlink != 0 || !pathInfo.Mode().IsRegular() ||
921 + !os.SameFile(file.info, destinationInfo) || !os.SameFile(destinationInfo, pathInfo) ||
922 + !bytes.Equal(current, file.original) {
923 + return fmt.Errorf("destination changed while formatting")
924 + }
925 + if err = os.Rename(tempPath, file.path); err != nil {
926 + return err
927 + }
928 + return nil
929 +}
930 +
931 // handleEnvFileFlag loads the environment variables from the given --envfile
932 // flag if specified. This should be called as early in the command function.
933 func handleEnvFileFlag(fl Flags) error {
cmd/commands.go
+7 -1
@@ -379,7 +379,7 @@ a tar archive.
379
380 RegisterCommand(Command{
381 Name: "fmt",
382 - Usage: "[--overwrite] [--diff] [<path>]",
382 + Usage: "[--overwrite] [--diff] [--imports] [<path>]",
383 Short: "Formats a Caddyfile",
384 Long: `
385 Formats the Caddyfile by adding proper indentation and spaces to improve
@@ -393,6 +393,11 @@ lines will be prefixed with '-' and '+' where they differ. Note that
393 unchanged lines are prefixed with two spaces for alignment, and that this
394 is not a valid patch format.
395
396 +If --imports is specified, all files referenced by import directives will
397 +also be formatted. Each file's output is preceded by a header line (# <path>)
398 +when printing to stdout, so multiple files are distinguishable. The --imports
399 +flag cannot be used when reading from stdin.
400 +
401 If you wish to use stdin instead of a regular file, use - as the path.
402 When reading from stdin, the --overwrite flag has no effect: the result
403 is always printed to stdout.
@@ -401,6 +406,7 @@ is always printed to stdout.
406 cmd.Flags().StringP("config", "c", "", "Configuration file")
407 cmd.Flags().BoolP("overwrite", "w", false, "Overwrite the input file with the results")
408 cmd.Flags().BoolP("diff", "d", false, "Print the differences between the input file and the formatted output")
409 + cmd.Flags().Bool("imports", false, "Also format files referenced by import directives")
410 cmd.RunE = WrapCommandFuncForCobra(cmdFmt)
411 },
412 })
cmd/commands_test.go
+272
@@ -1,12 +1,284 @@
1 package caddycmd
2
3 import (
4 + "bytes"
5 + "io"
6 "maps"
7 + "os"
8 + "path/filepath"
9 "reflect"
10 "slices"
11 + "strings"
12 "testing"
13 +
14 + "github.com/spf13/pflag"
15 +
16 + "github.com/caddyserver/caddy/v2"
17 + "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
18 )
19
20 +// newFmtFlags builds a Flags value wired up the same way the fmt cobra command does.
21 +func newFmtFlags(configFile string, overwrite, diff, imports bool) Flags {
22 + fs := pflag.NewFlagSet("fmt", pflag.ContinueOnError)
23 + fs.StringP("config", "c", "", "")
24 + fs.BoolP("overwrite", "w", false, "")
25 + fs.BoolP("diff", "d", false, "")
26 + fs.Bool("imports", false, "")
27 + if configFile != "" {
28 + _ = fs.Parse([]string{configFile})
29 + }
30 + if overwrite {
31 + _ = fs.Set("overwrite", "true")
32 + }
33 + if diff {
34 + _ = fs.Set("diff", "true")
35 + }
36 + if imports {
37 + _ = fs.Set("imports", "true")
38 + }
39 + return Flags{fs}
40 +}
41 +
42 +func writeTestFile(t *testing.T, path, content string) {
43 + t.Helper()
44 + if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
45 + t.Fatal(err)
46 + }
47 +}
48 +
49 +func canonicalTestPath(t *testing.T, path string) string {
50 + t.Helper()
51 + path, err := filepath.EvalSymlinks(path)
52 + if err != nil {
53 + t.Fatal(err)
54 + }
55 + return path
56 +}
57 +
58 +func captureStdout(t *testing.T, fn func() (int, error)) (string, int, error) {
59 + t.Helper()
60 + r, w, err := os.Pipe()
61 + if err != nil {
62 + t.Fatal(err)
63 + }
64 + oldStdout := os.Stdout
65 + os.Stdout = w
66 + t.Cleanup(func() { os.Stdout = oldStdout })
67 +
68 + code, callErr := fn()
69 + if err := w.Close(); err != nil {
70 + t.Fatal(err)
71 + }
72 + os.Stdout = oldStdout
73 + output, err := io.ReadAll(r)
74 + if err != nil {
75 + t.Fatal(err)
76 + }
77 + if err := r.Close(); err != nil {
78 + t.Fatal(err)
79 + }
80 + return string(output), code, callErr
81 +}
82 +
83 +// TestCmdFmtImportsStdinRejected verifies that --imports with stdin is rejected.
84 +func TestCmdFmtImportsStdinRejected(t *testing.T) {
85 + fl := newFmtFlags("-", false, false, true)
86 + code, err := cmdFmt(fl)
87 + if err == nil {
88 + t.Fatal("expected an error, got nil")
89 + }
90 + if !strings.Contains(err.Error(), "cannot use --imports") {
91 + t.Errorf("unexpected error message: %v", err)
92 + }
93 + if code != caddy.ExitCodeFailedStartup {
94 + t.Errorf("expected ExitCodeFailedStartup, got %d", code)
95 + }
96 +}
97 +
98 +// TestCmdFmtImportsOverwrite verifies that --imports --overwrite rewrites both
99 +// the root Caddyfile and its imported file.
100 +func TestCmdFmtImportsOverwrite(t *testing.T) {
101 + dir := t.TempDir()
102 +
103 + rootPath := filepath.Join(dir, "Caddyfile")
104 + sitesDir := filepath.Join(dir, "sites")
105 + if err := os.MkdirAll(sitesDir, 0o755); err != nil {
106 + t.Fatal(err)
107 + }
108 + importedPath := filepath.Join(sitesDir, "a.caddy")
109 +
110 + // Root imports the sites file; both are deliberately messy.
111 + writeTestFile(t, rootPath, "import sites/a.caddy\n")
112 + writeTestFile(t, importedPath, "localhost {\nrespond 200\n}\n")
113 + if err := os.Chmod(rootPath, 0o640); err != nil {
114 + t.Fatal(err)
115 + }
116 + rootInfoBefore, err := os.Stat(rootPath)
117 + if err != nil {
118 + t.Fatal(err)
119 + }
120 + importedInfoBefore, err := os.Stat(importedPath)
121 + if err != nil {
122 + t.Fatal(err)
123 + }
124 + // Windows loads FileInfo IDs lazily when SameFile is first called. Prime
125 + // them before replacement so these values continue to identify the old files.
126 + if !os.SameFile(rootInfoBefore, rootInfoBefore) || !os.SameFile(importedInfoBefore, importedInfoBefore) {
127 + t.Fatal("could not snapshot file identity")
128 + }
129 +
130 + fl := newFmtFlags(rootPath, true, false, true)
131 + code, err := cmdFmt(fl)
132 + if err != nil {
133 + t.Fatalf("cmdFmt returned error: %v", err)
134 + }
135 + if code != caddy.ExitCodeSuccess {
136 + t.Errorf("expected ExitCodeSuccess, got %d", code)
137 + }
138 +
139 + gotRoot, err := os.ReadFile(rootPath)
140 + if err != nil {
141 + t.Fatal(err)
142 + }
143 + if want := "import sites/a.caddy\n"; string(gotRoot) != want {
144 + t.Errorf("root file content = %q, want %q", string(gotRoot), want)
145 + }
146 + gotRootInfo, err := os.Stat(rootPath)
147 + if err != nil {
148 + t.Fatal(err)
149 + }
150 + if gotRootInfo.Mode().Perm() != rootInfoBefore.Mode().Perm() {
151 + t.Errorf("root file permissions = %o, want %o", gotRootInfo.Mode().Perm(), rootInfoBefore.Mode().Perm())
152 + }
153 + if os.SameFile(rootInfoBefore, gotRootInfo) {
154 + t.Error("root file was rewritten in place instead of atomically replaced")
155 + }
156 +
157 + got, err := os.ReadFile(importedPath)
158 + if err != nil {
159 + t.Fatal(err)
160 + }
161 + want := "localhost {\n\trespond 200\n}\n"
162 + if string(got) != want {
163 + t.Errorf("imported file content = %q, want %q", string(got), want)
164 + }
165 + gotImportedInfo, err := os.Stat(importedPath)
166 + if err != nil {
167 + t.Fatal(err)
168 + }
169 + if os.SameFile(importedInfoBefore, gotImportedInfo) {
170 + t.Error("imported file was rewritten in place instead of atomically replaced")
171 + }
172 +}
173 +
174 +func TestCmdFmtImportsPreviewReportsFormattingDifference(t *testing.T) {
175 + for _, diff := range []bool{false, true} {
176 + t.Run(map[bool]string{false: "preview", true: "diff"}[diff], func(t *testing.T) {
177 + dir := t.TempDir()
178 + rootPath := filepath.Join(dir, "Caddyfile")
179 + importedPath := filepath.Join(dir, "imported.caddy")
180 + writeTestFile(t, rootPath, "import imported.caddy\n")
181 + writeTestFile(t, importedPath, "localhost {\nrespond 200\n}\n")
182 +
183 + output, code, err := captureStdout(t, func() (int, error) {
184 + return cmdFmt(newFmtFlags(rootPath, false, diff, true))
185 + })
186 + if err == nil || !strings.Contains(err.Error(), "input is not formatted") {
187 + t.Fatalf("expected formatting error, got %v", err)
188 + }
189 + if code != caddy.ExitCodeFailedStartup {
190 + t.Errorf("expected ExitCodeFailedStartup, got %d", code)
191 + }
192 + if !strings.Contains(output, "# "+canonicalTestPath(t, importedPath)) {
193 + t.Errorf("output missing imported file header; got:\n%s", output)
194 + }
195 + if diff && !strings.Contains(output, "+ \trespond 200") {
196 + t.Errorf("output missing formatted diff; got:\n%s", output)
197 + }
198 + if !diff && !strings.Contains(output, "localhost {\n\trespond 200\n}") {
199 + t.Errorf("output missing formatted preview; got:\n%s", output)
200 + }
201 + })
202 + }
203 +}
204 +
205 +func TestOverwriteFormattedFilesRejectsSymlink(t *testing.T) {
206 + dir := t.TempDir()
207 + targetPath := filepath.Join(dir, "target")
208 + linkPath := filepath.Join(dir, "link")
209 + writeTestFile(t, targetPath, "original")
210 + if err := os.Symlink(targetPath, linkPath); err != nil {
211 + t.Skipf("symlinks unavailable: %v", err)
212 + }
213 +
214 + err := overwriteFormattedFiles([]caddyfile.FormattedFile{{Path: linkPath, Content: []byte("changed")}})
215 + if err == nil || !strings.Contains(err.Error(), "refusing to overwrite") {
216 + t.Fatalf("expected symlink rejection, got %v", err)
217 + }
218 + got, err := os.ReadFile(targetPath)
219 + if err != nil {
220 + t.Fatal(err)
221 + }
222 + if !bytes.Equal(got, []byte("original")) {
223 + t.Errorf("symlink target changed to %q", got)
224 + }
225 +}
226 +
227 +func TestOverwriteFormattedFilesReportsPartialUpdate(t *testing.T) {
228 + dir := t.TempDir()
229 + path := filepath.Join(dir, "Caddyfile")
230 + writeTestFile(t, path, "localhost {\nrespond 200\n}\n")
231 + formatted := []byte("localhost {\n\trespond 200\n}\n")
232 +
233 + err := overwriteFormattedFiles([]caddyfile.FormattedFile{
234 + {Path: path, Content: formatted},
235 + {Path: path, Content: formatted},
236 + })
237 + if err == nil || !strings.Contains(err.Error(), "after replacing 1 of 2 files") ||
238 + !strings.Contains(err.Error(), "may be partially updated") {
239 + t.Fatalf("expected partial-update error, got %v", err)
240 + }
241 + got, readErr := os.ReadFile(path)
242 + if readErr != nil {
243 + t.Fatal(readErr)
244 + }
245 + if !bytes.Equal(got, formatted) {
246 + t.Errorf("first replacement content = %q, want %q", got, formatted)
247 + }
248 +}
249 +
250 +// TestCmdFmtImportsPrintsHeaders verifies that without --overwrite each file's
251 +// output is preceded by a "# <path>" header line.
252 +func TestCmdFmtImportsPrintsHeaders(t *testing.T) {
253 + dir := t.TempDir()
254 +
255 + rootPath := filepath.Join(dir, "Caddyfile")
256 + sitesDir := filepath.Join(dir, "sites")
257 + if err := os.MkdirAll(sitesDir, 0o755); err != nil {
258 + t.Fatal(err)
259 + }
260 + importedPath := filepath.Join(sitesDir, "b.caddy")
261 +
262 + writeTestFile(t, rootPath, "import sites/b.caddy\n")
263 + writeTestFile(t, importedPath, "localhost {\n\trespond 200\n}\n")
264 +
265 + fl := newFmtFlags(rootPath, false, false, true)
266 + output, code, fmtErr := captureStdout(t, func() (int, error) { return cmdFmt(fl) })
267 +
268 + if fmtErr != nil {
269 + t.Fatalf("cmdFmt returned error: %v", fmtErr)
270 + }
271 + if code != caddy.ExitCodeSuccess {
272 + t.Errorf("expected ExitCodeSuccess, got %d", code)
273 + }
274 + if !strings.Contains(output, "# "+canonicalTestPath(t, rootPath)) {
275 + t.Errorf("output missing root header; got:\n%s", output)
276 + }
277 + if !strings.Contains(output, "# "+canonicalTestPath(t, importedPath)) {
278 + t.Errorf("output missing imported file header; got:\n%s", output)
279 + }
280 +}
281 +
282 func TestCommandsAreAvailable(t *testing.T) {
283 // trigger init, and build the default factory, so that
284 // all commands from this package are available