@tduncan / Nvidia-Optimization-Stack-VLLM
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
md 251 lines 11.3 KB
Rendered Raw
1 # Text watermarking
2
3 Text watermarking embeds a statistical signal in generated token choices. A
4 detector that knows the generation parameters can test for this signal without
5 access to the model weights.
6
7 The design and scope are discussed in
8 [RFC #53916](https://github.com/vllm-project/vllm/issues/53916).
9
10 ## Configuration
11
12 Enable watermarking by configuring an algorithm and secret key at engine
13 startup:
14
15 ```bash
16 vllm serve MODEL \
17 --watermark-config '{"algorithm":"gumbel","key":42}'
18 ```
19
20 Watermarking is disabled when `--watermark-config` is omitted. Gumbel-max (see
21 `gumbel`) is the default algorithm within an enabled `WatermarkConfig`. When
22 watermarking is configured, it is enabled for requests by default.
23
24 Requests can opt out without changing the engine-level algorithm or key:
25
26 ```python
27 from vllm import SamplingParams
28
29 sampling_params = SamplingParams(watermarking=False)
30 ```
31
32 The Python OpenAI-compatible APIs and the Rust chat/completions APIs accept the
33 same `watermarking: false` request field. The Rust token API accepts it in
34 `sampling_params`, and the Rust gRPC `GenerateRequest` accepts
35 the optional `watermarking` field. Omitting the field keeps watermarking enabled
36 when the engine has a watermark configuration.
37
38 Deployments that require watermarking must restrict this field to trusted
39 callers, or strip and validate it at the ingress boundary, so untrusted clients
40 cannot opt out.
41
42 `context_width` controls how many prior tokens seed each watermark decision
43 and defaults to 4. Larger values make the watermark less robust to
44 edits because an insertion, deletion, or substitution changes more subsequent
45 contexts. Values above 16 are allowed but emit a warning.
46
47 `allow_target_only_watermarking` defaults to false and only has an effect when
48 speculative decoding is enabled. It permits speculative decoding with a
49 watermarking algorithm that does not support it natively, at the cost of
50 weaker detectability. See
51 [Speculative decoding](#speculative-decoding).
52
53 ## Architecture
54
55 `WatermarkConfig` selects an algorithm and PRF. Model Runner V2 constructs the
56 corresponding `Watermarker`, and `GPUWatermarkSampler` invokes it for the final
57 stochastic token selection after temperature, min-p, top-k, and top-p are
58 applied. A watermarker can either select a token directly or transform logits
59 and delegate to vLLM's random sampler.
60
61 Detection is separate from generation. vLLM provides detector primitives for
62 the reference algorithms. `WatermarkDetector` consumes token IDs, so callers
63 remain responsible for using the tokenizer and watermark profile that match
64 generation.
65
66 ## Speculative decoding
67
68 Watermarking requires speculative decoding to use probabilistic draft sampling,
69 standard rejection sampling, and an autoregressive model-based method (`dspark`,
70 `eagle`, `eagle3`, or `mtp`). Parallel drafting is supported only by `dspark`.
71
72 A watermarking algorithm without native speculative-decoding support is
73 rejected before model loading. Set
74 `"allow_target_only_watermarking": true` to allow it: accepted draft tokens are
75 not watermarked, while target-side rejection recovery and bonus sampling remain
76 watermarked. The watermark signal is diluted in proportion to the share of
77 output tokens supplied by accepted drafts; rejected drafts do not dilute it
78 because their recovery tokens are watermarked.
79
80 For `dual_key_gumbel`, `alpha` has no effect under speculative decoding. The
81 speculative protocol selects the key for each token instead.
82
83 ## Context deduplication
84
85 When a token context is repeated, generation can use ordinary sampling for that
86 occurrence. Reusing the repeated context leads to a bias over the sequence, as
87 certain token choices would be correlated. Ordinary sampling at these positions
88 allows for single-sequence non-distortion (see section G.3 of the
89 [SynthID-Text supplementary materials](https://media.springernature.com/original/springer-static/esm/art%3A10.1038%2Fs41586-024-08025-4/MediaObjects/41586_2024_8025_MOESM1_ESM.pdf)).
90 This also applies to drafter watermarking during speculative decoding.
91 The detector independently deduplicates contexts so repeated keyed random
92 vectors are not treated as independent evidence, meaning that context
93 deduplication at generation time does not reduce the watermarking signal, unless
94 a user tampers with the output and removes the first occurrence while leaving an
95 unwatermarked repetition.
96
97 `deduplicate_contexts` controls which history is searched for a repeated
98 context:
99
100 - `"none"` disables context deduplication.
101 - `"single_turn"`, the default, searches the tokens generated for the current
102 request. This gives single-turn non-distortion.
103 - `"all"` also searches the prompt, which extends non-distortion across the
104 turns of a conversation. The first `context_width` generated tokens use
105 ordinary sampling, and subsequent tokens whose context already occurs in the
106 prompt are not watermarked. Previous turns of a session, or a prompt that
107 already contains the structure of the answer such as a tool result the model
108 extends, may therefore leave little of the answer watermarked. Given the
109 impact of `"all"` on both performance (sampling steps must scan over a
110 potentially large amount of previous context) and watermarking detectability
111 (we may significantly reduce the amount of watermarked tokens), you should
112 only use `"all"` when non-distortion across multiple turns is required.
113
114 `deduplicate_contexts_max_history` limits the search to the most recent
115 positions and defaults to 8,192. Each position is compared over the
116 `context_width` tokens before it, so the window of tokens read is that much
117 longer. A smaller value reduces scanning cost but only provides the guarantee
118 within that window. Set it to `null` to search back to the start of the
119 generation for `"single_turn"`, or of the request for `"all"`; an unbounded
120 search costs more as the sequence grows. The setting has no effect when
121 `deduplicate_contexts` is `"none"`. Values below 1,024 emit a warning because a
122 short window can miss repetition loops whose contexts recur farther apart.
123
124 For example, this checks prompt and completion history within the default
125 8,192-position window:
126
127 ```bash
128 vllm serve MODEL \
129 --watermark-config \
130 '{"algorithm":"gumbel","key":42,"deduplicate_contexts":"all"}'
131 ```
132
133 ## Algorithms
134
135 ### Gumbel-max
136
137 Gumbel-max derives a deterministic pseudorandom value from the key, prior token
138 context, and every candidate token, then uses the resulting Gumbel noise for
139 categorical sampling. See
140 [Aaronson's original presentation](https://simons.berkeley.edu/sites/default/files/2024-10/LLM24-2%20Slides%20-%20Scott%20Aaronson.pdf).
141
142 Gumbel-max requires stochastic sampling. Greedy requests (`temperature=0`)
143 bypass watermarking and emit a warning once per worker.
144
145 ### Dual-key Gumbel-max
146
147 Dual-key Gumbel-max derives independent keys A and B from one configured master
148 key. During ordinary generation, each token uses key A with probability
149 `1 - alpha` and key B with probability `alpha`. The algorithm-specific `alpha`
150 parameter defaults to 0.1. Detection scores every token against both keys.
151
152 The same two key streams support speculative decoding without changing its
153 acceptance rate. In this mode, the speculative protocol selects the key instead
154 of `alpha`: draft tokens use key A, while rejection recovery and bonus tokens
155 use key B. The ordinary target-to-draft probability-ratio test remains unchanged,
156 implementing [SynthID-Text Supplementary Algorithm
157 6](https://media.springernature.com/original/springer-static/esm/art%3A10.1038%2Fs41586-024-08025-4/MediaObjects/41586_2024_8025_MOESM1_ESM.pdf).
158
159 Select `dual_key_gumbel` together with probabilistic drafting:
160
161 ```bash
162 vllm serve MODEL \
163 --speculative-config \
164 '{"method":"mtp","num_speculative_tokens":3,"draft_sample_method":"probabilistic"}' \
165 --watermark-config '{"algorithm":"dual_key_gumbel","key":42,"alpha":0.1}'
166 ```
167
168 ### SynthID-Text
169
170 [SynthID-Text](https://www.nature.com/articles/s41586-024-08025-4) is planned but
171 not currently implemented.
172
173 ## Pseudorandom functions
174
175 A watermark PRF turns the secret key, token context, and candidate token into
176 reproducible random values. Generation and detection must produce identical
177 values across devices and releases. The values should also be uniform and
178 independent enough for the sampling algorithm and detector statistics. PRF
179 selection is an advanced compatibility and performance setting; most users
180 should keep the default.
181
182 Watermarked generation currently supports the `philox` PRF:
183
184 - `philox` is based on the counter-based Philox4x32-10
185 generator from the [Random123 paper](https://doi.org/10.1145/2063384.2063405).
186 It is parallel, vectorizes on accelerators, and avoids CPU transfers, but is
187 not a cryptographic PRF and does not provide key-recovery or forgery
188 resistance. vLLM versions its input mapping and provides compatibility
189 vectors so generation and detection remain interoperable.
190
191 ## Detection
192
193 The detector primitives operate on token IDs and do not require model weights:
194
195 ```python
196 from transformers import AutoTokenizer
197
198 from vllm.v1.watermarking import GumbelWatermarkDetector
199
200 tokenizer = AutoTokenizer.from_pretrained(MODEL)
201 token_ids = tokenizer.encode(text, add_special_tokens=False)
202 result = GumbelWatermarkDetector(key=42, prf="philox").detect(token_ids)
203 print(result.p_value, result.is_watermarked)
204 ```
205
206 The detection configuration must match the generation configuration, including
207 the tokenizer, PRF, watermarking algorithm, algorithm-specific watermarking
208 configuration, and key. In practice, this information is often unavailable
209 when checking a piece of text. Deployments should therefore retain the set of
210 candidate configurations they have served, test the text against each
211 candidate, and correct for multiple testing, for example with a Bonferroni
212 correction to the resulting p-values.
213
214 Gumbel-max detection scores repeated contexts once by default so identical PRF
215 random vectors are not treated as independent evidence. Keep
216 `deduplicate_contexts=True` unless the detector's calibration has been adjusted
217 for correlated scores.
218
219 The reported p-value is calibrated under the assumption that scored PRF inputs
220 are independent. A deployment uses one fixed key, so repeated structures across
221 documents reuse the same PRF values and can make the realized false-positive
222 rate key-dependent even when contexts are deduplicated within each document.
223 Measure the false-positive rate on representative unwatermarked traffic with the
224 deployed key before relying on `is_watermarked` for decisions.
225
226 A minimal HTTP detector is available in
227 `examples/basic/online_serving/watermark_detection_server.py`:
228
229 ```bash
230 python examples/basic/online_serving/watermark_detection_server.py \
231 --tokenizer MODEL --key 42 --prf philox
232 ```
233
234 ```bash
235 curl http://localhost:8000/detect \
236 -H 'Content-Type: application/json' \
237 -d '{"text":"Text to inspect"}'
238 ```
239
240 Scores and p-values expose information about the per-token watermark signal.
241 Repeated queries can use this information to construct text that imitates
242 watermarked output or to modify watermarked text so it is no longer detected.
243
244 ## Limitations
245
246 - Watermarking is currently available only with Model Runner V2.
247 - Not all watermarking algorithms have native speculative-decoding support.
248 - Beam search expands candidates from model log probabilities and does not apply
249 Gumbel-max watermarking.
250 - Models that replace the vLLM sampler with a custom sampler cannot use
251 configured watermarking.