@devries-cameron20 / composio Mirror · read-only / commits / 8878a9bcc
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Mirrored from https://github.com/Camo67/composio.git

Synced Nov 21, 12:15 UTC

fix(typesafe): fail closed on redaction structure changes and invalid gate options

Review findings from the TypeSafe (Jev) provider PR, fixed in both SDKs: - The confidence gate enforces a masking-only redaction contract: the redacted arguments must keep the original JSON structure (same keys, same array lengths) with every leaf replaced by a value of the same scalar type, or the call is blocked. Jev can no longer approve a call that differs from the one that runs. - `onUnavailable` is validated when the gate is built, in the companion both entry points share: a typo'd mode raises TypesafeInvalidOptionsError at construction instead of failing open at check time. - `minItems` is carried through classification and compilation into array arguments, and a selection with fewer members than `minItems` counts as not stated, so a required array stays missing instead of executing with `[]`. - The boolean class gains `nullable`: a ['boolean', 'null'] property compiles to a yes/no/null Choice so Jev can bind null; plain booleans still compile to yes/no only, and one-boolean enum sets stay open-ended. - `stable()` rejects `undefined` wherever it appears instead of rewriting it, and the gate builds its state without a `context` key when there is no context, so a context-less gate call still works. - `Probability` and the classify number type accept JSON integers, so an API score of 0 or 1 and integer `maxItems`/`minItems` parse instead of marking the property open-ended or the response malformed. Bools, strings, NaN, inf, and out-of-range values are still rejected.

jkomyno committed Sep 17, 2026 at 18:33 UTC 8878a9bcc5a43f875329f18a4c4a7ce06b94d8a8
17 files changed +666 -49
python/providers/typesafe/README.md
+1 -1
@@ -224,7 +224,7 @@ composio.tools.execute(
224
225 - A veto raises `TypesafeGateVetoError`. After `max_vetoes` vetoes (default 3) the gate blocks every later call in the run, so a hijacked LLM cannot vary arguments until one passes. Create a fresh gate for each new run. Never share a gate across users or runs, and never recreate it for each retry.
226 - One gate checks one call at a time, so parallel calls cannot get past `max_vetoes` together. Parallel calls through the same gate wait for each other. Separate gates do not.
227 -- `redact_arguments` gets a deep copy of the arguments and must return a `dict` with string keys. The call that runs keeps its original arguments. Any other return value blocks the call.
227 +- `redact_arguments` gets a deep copy of the arguments and must return a `dict` with string keys that masks the arguments it got: the same JSON structure (same keys, same array lengths) with every leaf replaced by a value of the same JSON type. The call that runs keeps its original arguments. Anything that deletes or adds a key, changes an array's length, or retypes a value blocks the call, because Jev must approve exactly the call that runs.
228 - When TypeSafe cannot be reached the gate blocks with `TypesafeGateUnavailableError`. `on_unavailable="allow"` lets those calls through and reports each one to `on_bypass`. A tool the gate was not given, an oversized call, a malformed response, and arguments or context that hold a value that is not JSON always block. The gate never truncates or drops a value to make a call fit.
229 - The gate checks that a call is consistent with the request. It is not an authorization check. File uploads run before `before_execute`, so a veto happens after the upload.
230 - The gate covers direct execution only. A session applies no modifiers on `session.execute`.
python/providers/typesafe/composio_typesafe/classify.py
+14 -4
@@ -7,8 +7,8 @@ import typing as t
7 import typing_extensions as te
8 from pydantic import (
9 BaseModel,
10 + BeforeValidator,
11 ConfigDict,
11 - Field,
12 StrictBool,
13 StrictStr,
14 model_validator,
@@ -16,13 +16,16 @@ from pydantic import (
16 from pydantic import ValidationError as PydanticValidationError
17
18 from .keys import MAX_CHOICE_OPTIONS
19 +from .types import _json_number
20
21 _MAX_SAFE_INTEGER = 2**53 - 1
22
23 _JsonSchemaType = t.Literal[
24 "string", "number", "integer", "boolean", "object", "array", "null"
25 ]
25 -_Number = te.Annotated[float, Field(strict=True)]
26 +# A JSON number: an integer on the wire is as valid as a float. `bool` is an `int`
27 +# subclass, so `_json_number` rejects it, and strings never reach the float.
28 +_Number = te.Annotated[float, BeforeValidator(_json_number)]
29
30
31 class JsonSchemaProperty(BaseModel):
@@ -45,6 +48,7 @@ class JsonSchemaProperty(BaseModel):
48 enum: t.Optional[t.List[t.Any]] = None
49 const: t.Any = None
50 maxItems: t.Optional[_Number] = None
51 + minItems: t.Optional[_Number] = None
52
53 @model_validator(mode="before")
54 @classmethod
@@ -83,12 +87,14 @@ class EnumClass(te.TypedDict):
87
88 class BooleanClass(te.TypedDict):
89 kind: t.Literal["boolean"]
90 + nullable: bool
91
92
93 class EnumArrayClass(te.TypedDict):
94 kind: t.Literal["enum_array"]
95 values: t.List[t.Union[str, int]]
96 maxItems: te.NotRequired[t.Union[int, float]]
97 + minItems: te.NotRequired[t.Union[int, float]]
98
99
100 class OpenClass(te.TypedDict):
@@ -163,7 +169,9 @@ def _classify_members(members: t.List[t.Any], nullable_hint: bool) -> ArgumentCl
169 return _OPEN
170 if all(isinstance(value, bool) for value in values):
171 # The boolean Choice offers both values, so a set that allows only one is open-ended.
166 - return {"kind": "boolean"} if len(values) == 2 else _OPEN
172 + if len(values) != 2:
173 + return _OPEN
174 + return {"kind": "boolean", "nullable": nullable}
175 strings = [value for value in values if isinstance(value, str)]
176 integers = [
177 value
@@ -209,11 +217,13 @@ def classify_property(parsed: t.Mapping[str, t.Any]) -> ArgumentClass:
217 }
218 if "maxItems" in parsed:
219 array_class["maxItems"] = _normalize_number(parsed["maxItems"])
220 + if "minItems" in parsed:
221 + array_class["minItems"] = _normalize_number(parsed["minItems"])
222 return array_class
223
224 members = _literal_members(parsed)
225 if members is not None:
226 return _classify_members(members, nullable_hint)
227 if value_types == ["boolean"]:
218 - return {"kind": "boolean"}
228 + return {"kind": "boolean", "nullable": nullable_hint}
229 return _OPEN
python/providers/typesafe/composio_typesafe/companion.py
+54
@@ -117,6 +117,50 @@ class GateModifier(t.Protocol):
117 ) -> ToolExecuteParams: ...
118
119
120 +def _stable_or_block(value: t.Any) -> TypesafeJsonValue:
121 + """A value that is not JSON blocks the call. Nothing is dropped or rewritten."""
122 + try:
123 + return stable(value)
124 + except TypesafeInvalidOptionsError:
125 + pass
126 + # Raised outside the `except` block, so the invalid-options error is not the context.
127 + raise TypesafeGateBlockedError("check_failed")
128 +
129 +
130 +def _leaf_kind(value: t.Any) -> str:
131 + # JSON scalar kinds. `bool` is checked first, because it is an `int` subclass.
132 + if isinstance(value, bool):
133 + return "boolean"
134 + if isinstance(value, (int, float)):
135 + return "number"
136 + if isinstance(value, str):
137 + return "string"
138 + return "null"
139 +
140 +
141 +def _assert_masking_only(original: t.Any, redacted: t.Any) -> None:
142 + """
143 + The redactor is a masker: the redacted arguments must have exactly the original's
144 + JSON structure, meaning the same key sets and the same array lengths, and every leaf
145 + replacement must keep the same scalar kind. Anything else blocks, because Jev would
146 + approve a call that differs from the one that runs.
147 + """
148 + if isinstance(original, list):
149 + if not isinstance(redacted, list) or len(redacted) != len(original):
150 + raise TypesafeGateBlockedError("check_failed")
151 + for entry, replacement in zip(original, redacted):
152 + _assert_masking_only(entry, replacement)
153 + return
154 + if isinstance(original, dict):
155 + if not isinstance(redacted, dict) or set(redacted) != set(original):
156 + raise TypesafeGateBlockedError("check_failed")
157 + for key, entry in original.items():
158 + _assert_masking_only(entry, redacted[key])
159 + return
160 + if _leaf_kind(original) != _leaf_kind(redacted):
161 + raise TypesafeGateBlockedError("check_failed")
162 +
163 +
164 def confidence_gate(
165 *,
166 ask: Ask,
@@ -148,6 +192,11 @@ def confidence_gate(
192 )
193 if not _is_integer(max_vetoes) or max_vetoes < 1:
194 raise TypesafeInvalidOptionsError("`max_vetoes` must be a positive integer.")
195 + if on_unavailable not in ("block", "allow"):
196 + # A typo'd mode must never fail open at check time.
197 + raise TypesafeInvalidOptionsError(
198 + "`on_unavailable` must be 'block' or 'allow'."
199 + )
200 by_slug = {tool.slug: tool for tool in tools}
201 lock = threading.Lock()
202 vetoes = 0
@@ -179,11 +228,16 @@ def confidence_gate(
228 pass
229 if copied is None:
230 raise TypesafeGateBlockedError("check_failed")
231 + # The originals are checked first: what the gate reads must be JSON in full.
232 + original_arguments = _stable_or_block(call_arguments)
233 call_arguments = redact_arguments(raw_tool.slug, copied)
234 if not isinstance(call_arguments, dict) or not all(
235 isinstance(key, str) for key in call_arguments
236 ):
237 raise TypesafeGateBlockedError("check_failed")
238 + # Jev evaluates what is sent, and the original arguments run. The redactor
239 + # may only mask, so an approved call is exactly the call that executes.
240 + _assert_masking_only(original_arguments, _stable_or_block(call_arguments))
241 description = getattr(raw_tool, "description", None)
242 # Only the slug and the arguments are copied out of the execution parameters.
243 # `user_id`, `connected_account_id`, and the custom auth fields never leave.
python/providers/typesafe/composio_typesafe/compile.py
+15 -5
@@ -106,6 +106,18 @@ def _compile_argument(
106 {"key": "yes", "value": True},
107 {"key": "no", "value": False},
108 ]
109 + if argument_class["nullable"]:
110 + flags.append({"key": NULL_KEY, "value": None})
111 +
112 + def boolean_text(argument: str, option: TypesafeOption) -> str:
113 + if option["value"] is None:
114 + return (
115 + f'The request explicitly asks for no value (null) for "{argument}".'
116 + )
117 + if option["value"] is True:
118 + return f'The request states that "{argument}" is true.'
119 + return f'The request states that "{argument}" is false.'
120 +
121 return {
122 "kind": "choice",
123 "name": argument,
@@ -116,11 +128,7 @@ def _compile_argument(
128 argument,
129 description,
130 flags,
119 - lambda option: (
120 - f'The request states that "{argument}" is true.'
121 - if option["value"] is True
122 - else f'The request states that "{argument}" is false.'
123 - ),
131 + lambda option: boolean_text(argument, option),
132 ),
133 "options": flags,
134 "notStatedKey": NOT_STATED_KEY,
@@ -176,6 +184,8 @@ def _compile_argument(
184 }
185 if "maxItems" in argument_class:
186 compiled["maxItems"] = t.cast(int, argument_class["maxItems"])
187 + if "minItems" in argument_class:
188 + compiled["minItems"] = t.cast(int, argument_class["minItems"])
189 return compiled
190
191
python/providers/typesafe/composio_typesafe/decide.py
+3
@@ -430,6 +430,9 @@ def _read_argument(
430 )
431 if "maxItems" in argument:
432 selected = selected[: int(argument["maxItems"])]
433 + # An undersized selection is not what the request stated, so nothing is bound.
434 + if len(selected) < argument.get("minItems", 0):
435 + return None
436 score = min([mentioned, *(max(p, 1 - p) for _, _, p in members)])
437 # The output keeps the declared order of the members.
438 return _Stated([value for _, value, _ in sorted(selected)], score)
python/providers/typesafe/composio_typesafe/provider.py
+5 -3
@@ -295,9 +295,11 @@ class TypesafeProvider(
295 :param on_unavailable: What to do when Jev cannot be reached.
296 :param on_bypass: Called for every call allowed through without an answer from Jev.
297 :param max_vetoes: After this many vetoes the gate blocks every later call.
298 - :param redact_arguments: Replaces secret-bearing arguments before they are sent
299 - to TypeSafe. It gets a deep copy and must return a `dict`; the call that runs
300 - keeps its original arguments.
298 + :param redact_arguments: Masks secret-bearing arguments before they are sent to
299 + TypeSafe. It gets a deep copy and must return a `dict` with the same
300 + structure as the arguments, with every leaf replaced by a value of the same
301 + JSON type; anything that deletes, adds, or retypes a value blocks the call.
302 + The call that runs keeps its original arguments.
303 """
304 return before_execute(
305 companion.confidence_gate(
python/providers/typesafe/composio_typesafe/types.py
+18 -2
@@ -5,7 +5,7 @@ from __future__ import annotations
5 import typing as t
6
7 import typing_extensions as te
8 -from pydantic import ConfigDict, Field, TypeAdapter, with_config
8 +from pydantic import BeforeValidator, ConfigDict, Field, TypeAdapter, with_config
9 from pydantic import ValidationError as PydanticValidationError
10
11 from composio.client.types import Tool
@@ -112,6 +112,8 @@ class TypesafeArrayArgument(te.TypedDict):
112 mentioned: TypesafeNoulQuestion
113 members: t.List[TypesafeArrayMember]
114 maxItems: te.NotRequired[int]
115 + minItems: te.NotRequired[int]
116 + """A selection with fewer members than this is not stated, so the argument stays missing."""
117
118
119 TypesafeArgumentQuestion: te.TypeAlias = t.Union[
@@ -172,7 +174,21 @@ TypesafeState: te.TypeAlias = t.Union[str, TypesafeRequestState]
174 # ---------------------------------------------------------------------------
175
176
175 -Probability = te.Annotated[float, Field(ge=0, le=1, strict=True, allow_inf_nan=False)]
177 +def _json_number(value: t.Any) -> t.Any:
178 + """
179 + Accepts a JSON number and returns it as a float, so a whole number on the wire (``0``
180 + or ``1``) is valid. ``bool`` is an ``int`` subclass, so it is rejected here.
181 + """
182 + if isinstance(value, bool) or not isinstance(value, (int, float)):
183 + raise ValueError("expected a JSON number")
184 + return float(value)
185 +
186 +
187 +Probability = te.Annotated[
188 + float,
189 + BeforeValidator(_json_number),
190 + Field(ge=0, le=1, allow_inf_nan=False),
191 +]
192 _Path = te.Annotated[t.List[str], Field(min_length=1)]
193 # Pydantic validates a stored decision against these same TypedDicts, strictly.
194 _STRICT = ConfigDict(strict=True)
python/tests/test_typesafe_provider.py
+298 -12
@@ -39,17 +39,23 @@ from composio_typesafe import ( # noqa: E402
39 TypesafeMissingApiKeyError,
40 TypesafeProvider,
41 TypesafeProviderError,
42 + companion,
43 + parse_decision,
44 )
43 -from composio_typesafe.classify import classify_property # noqa: E402
45 +from composio_typesafe.classify import classify_property, parse_property # noqa: E402
46 from composio_typesafe.compile import routing_question # noqa: E402
47 +from composio_typesafe.decide import normalize_state, stable # noqa: E402
48 from composio_typesafe.keys import ( # noqa: E402
49 build_options,
50 index_prefixed_key,
51 is_safe_label,
52 option_values,
53 )
54 +from composio_typesafe.types import Probability # noqa: E402
55 from hypothesis import given # noqa: E402
56 from hypothesis import strategies as st # noqa: E402
57 +from pydantic import TypeAdapter # noqa: E402
58 +from pydantic import ValidationError as PydanticValidationError # noqa: E402
59
60 from composio.client.types import Tool # noqa: E402
61 from composio.core.models.base import allow_tracking # noqa: E402
@@ -318,7 +324,10 @@ class TestQuestionCorpus:
324 class TestClassifyAndKeys:
325 def test_python_number_traps(self) -> None:
326 # bool is an int subclass, JSON 1.0 is the integer 1, and 1 and "1" are mixed types.
321 - assert classify_property({"enum": [True, False]}) == {"kind": "boolean"}
327 + assert classify_property({"enum": [True, False]}) == {
328 + "kind": "boolean",
329 + "nullable": False,
330 + }
331 assert classify_property({"enum": [1, True]}) == {"kind": "open"}
332 assert classify_property({"enum": [1, "1"]}) == {"kind": "open"}
333 assert classify_property({"enum": [1.0, 2, 2.0]}) == {
@@ -331,19 +340,45 @@ class TestClassifyAndKeys:
340 assert classify_property({"enum": [2**53 - 1]})["kind"] == "enum"
341
342 @pytest.mark.parametrize(
334 - ("schema", "kind"),
343 + ("schema", "expected"),
344 [
345 # A yes/no Choice would offer a value the schema forbids.
337 - ({"const": True}, "open"),
338 - ({"enum": [False, None]}, "open"),
339 - ({"oneOf": [{"enum": [False]}, {"type": "null"}]}, "open"),
340 - ({"type": ["boolean", "null"]}, "boolean"),
341 - ({"enum": [False, True, None]}, "boolean"),
342 - ({"anyOf": [{"const": True}, {"const": False}]}, "boolean"),
346 + ({"const": True}, {"kind": "open"}),
347 + ({"enum": [False, None]}, {"kind": "open"}),
348 + ({"oneOf": [{"enum": [False]}, {"type": "null"}]}, {"kind": "open"}),
349 + ({"type": ["boolean", "null"]}, {"kind": "boolean", "nullable": True}),
350 + ({"type": "boolean"}, {"kind": "boolean", "nullable": False}),
351 + ({"enum": [False, True, None]}, {"kind": "boolean", "nullable": True}),
352 + (
353 + {"anyOf": [{"const": True}, {"const": False}]},
354 + {"kind": "boolean", "nullable": False},
355 + ),
356 ],
357 )
345 - def test_a_boolean_needs_both_values(self, schema: t.Any, kind: str) -> None:
346 - assert classify_property(schema) == {"kind": kind}
358 + def test_a_boolean_needs_both_values(
359 + self, schema: t.Any, expected: t.Dict[str, t.Any]
360 + ) -> None:
361 + assert classify_property(schema) == expected
362 +
363 + def test_an_enum_array_keeps_integer_max_items_and_min_items(self) -> None:
364 + # JSON whole numbers arrive as ints; a strict float used to make the property
365 + # open-ended instead.
366 + prop = parse_property(
367 + {
368 + "type": "array",
369 + "items": {"type": "string", "enum": ["a", "b"]},
370 + "maxItems": 2,
371 + "minItems": 1,
372 + }
373 + )
374 + assert prop is not None
375 + assert classify_property(prop) == {
376 + "kind": "enum_array",
377 + "values": ["a", "b"],
378 + "maxItems": 2,
379 + "minItems": 1,
380 + }
381 + assert parse_property({"type": "array", "items": {}, "maxItems": "2"}) is None
382
383 def test_option_count_limit(self) -> None:
384 members = [f"m{index}" for index in range(254)]
@@ -532,6 +567,101 @@ class TestDecideOutcomes:
567 )
568 assert capped["arguments"] == {"pick": ["b"]}
569
570 + def test_a_selection_below_min_items_is_not_stated(self) -> None:
571 + tool = make_tool(
572 + "PICK_SOME",
573 + {
574 + "pick": {
575 + "type": "array",
576 + "items": {"enum": ["a", "b"]},
577 + "minItems": 1,
578 + },
579 + "pick_many": {
580 + "type": "array",
581 + "items": {"enum": ["x", "y"]},
582 + "minItems": 2,
583 + },
584 + },
585 + ["pick"],
586 + )
587 + compiled = json.loads(
588 + json.dumps(TypesafeProvider().wrap_tool(tool)["arguments"])
589 + )
590 + assert [a["name"] for a in compiled] == ["pick", "pick_many"]
591 + assert compiled[0]["minItems"] == 1
592 + assert compiled[1]["minItems"] == 2
593 +
594 + # "mentioned" with zero members selected leaves the required argument missing.
595 + none = decide_both(
596 + respond(("PICK_SOME", 0.9), answers={"t0_a0_mentioned": 0.9}),
597 + [tool],
598 + "pick some",
599 + )
600 + assert matches(
601 + none, {"kind": "partial", "missing": [["pick"]], "arguments": {}}
602 + )
603 +
604 + # Enough members selected binds the argument.
605 + enough = decide_both(
606 + respond(
607 + ("PICK_SOME", 0.9),
608 + answers={"t0_a0_mentioned": 0.9, "t0_a0_m0": 0.9},
609 + ),
610 + [tool],
611 + "pick a",
612 + )
613 + assert matches(enough, {"kind": "call", "arguments": {"pick": ["a"]}})
614 +
615 + # An optional argument that stays below minItems is not bound and not dropped.
616 + optional = decide_both(
617 + respond(
618 + ("PICK_SOME", 0.9),
619 + answers={
620 + "t0_a0_mentioned": 0.9,
621 + "t0_a0_m0": 0.9,
622 + "t0_a1_mentioned": 0.9,
623 + "t0_a1_m0": 0.9,
624 + },
625 + ),
626 + [tool],
627 + "pick some",
628 + )
629 + assert matches(
630 + optional,
631 + {"kind": "call", "arguments": {"pick": ["a"]}, "dropped": []},
632 + )
633 +
634 + def test_a_nullable_boolean_offers_and_binds_null(self) -> None:
635 + nullable = make_tool("CLEAR_FLAG", {"flag": {"type": ["boolean", "null"]}})
636 + compiled = json.loads(
637 + json.dumps(TypesafeProvider().wrap_tool(nullable)["arguments"][0])
638 + )
639 + assert compiled["options"] == [
640 + {"key": "yes", "value": True},
641 + {"key": "no", "value": False},
642 + {"key": "__null__", "value": None},
643 + ]
644 + assert (
645 + compiled["question"]["criteria"]["__null__"]
646 + == 'The request explicitly asks for no value (null) for "flag".'
647 + )
648 + bound = decide_both(
649 + respond(("CLEAR_FLAG", 0.9), answers={"t0_a0": ("__null__", 0.9)}),
650 + [nullable],
651 + "clear the flag",
652 + )
653 + assert matches(bound, {"kind": "call", "arguments": {"flag": None}})
654 + assert bound["arguments"]["flag"] is None
655 +
656 + plain = make_tool("SET_FLAG", {"flag": {"type": "boolean"}})
657 + plain_compiled = json.loads(
658 + json.dumps(TypesafeProvider().wrap_tool(plain)["arguments"][0])
659 + )
660 + assert plain_compiled["options"] == [
661 + {"key": "yes", "value": True},
662 + {"key": "no", "value": False},
663 + ]
664 +
665 def test_restores_typed_values(self) -> None:
666 tool = make_tool(
667 "TYPED",
@@ -620,6 +750,27 @@ class TestDecideWithoutARequest:
750 {"request": "open a ticket", "context": cyclic},
751 )
752
753 + def test_stable_keeps_json_null_and_rejects_the_rest(self) -> None:
754 + # JSON null is a value; nothing else is dropped or rewritten.
755 + assert stable(None) is None
756 + assert stable({"b": 1, "a": None}) == {"a": None, "b": 1}
757 + assert stable([1, None]) == [1, None]
758 + for not_json in (
759 + float("nan"),
760 + float("inf"),
761 + datetime.date(2026, 1, 2),
762 + object(),
763 + ):
764 + with pytest.raises(TypesafeInvalidOptionsError):
765 + stable(not_json)
766 +
767 + contextless = normalize_state({"request": "open a ticket"})
768 + assert contextless.has_context is False
769 + assert contextless.full == {"request": "open a ticket"}
770 + with_context = normalize_state({"request": "open", "context": {"a": None}})
771 + assert with_context.has_context is True
772 + assert with_context.full == {"context": {"a": None}, "request": "open"}
773 +
774
775 def many_tools(count: int) -> t.List[Tool]:
776 return [make_tool(f"TOOL_{index}", description="x") for index in range(count)]
@@ -823,6 +974,60 @@ class TestThresholdsAndRisk:
974 )
975
976
977 +class TestProbabilityAcceptsJsonNumbers:
978 + def test_probability_accepts_whole_numbers_and_rejects_the_rest(self) -> None:
979 + adapter = TypeAdapter(Probability)
980 + assert adapter.validate_python(0) == 0.0
981 + assert adapter.validate_python(1) == 1.0
982 + assert adapter.validate_python(0.5) == 0.5
983 + assert isinstance(adapter.validate_python(1), float)
984 + for bad in (True, "0.5", 1.5, float("nan"), float("inf"), -0.1, None):
985 + with pytest.raises(PydanticValidationError):
986 + adapter.validate_python(bad)
987 +
988 + def test_a_stored_decision_with_whole_number_confidence_parses(self) -> None:
989 + parsed = parse_decision({**CALL, "confidence": 1})
990 + assert parsed["confidence"] == 1.0
991 + for bad in (True, "0.9", 1.5, float("nan")):
992 + with pytest.raises(TypesafeMalformedDecisionError):
993 + parse_decision({**CALL, "confidence": bad})
994 +
995 + def test_an_api_answer_with_whole_number_probabilities_parses(self) -> None:
996 + # JSON 0 and 1 are valid probabilities, not malformed responses.
997 + response = {
998 + "model": "jev-1.13",
999 + "answers": {
1000 + "route": {
1001 + "type": "choice",
1002 + "choice": "TICKETS_CREATE",
1003 + "confidence": 1,
1004 + "probabilities": {"TICKETS_CREATE": 1, NONE: 0},
1005 + },
1006 + "gate_0": {"type": "noul", "noul": 1},
1007 + "gate_1": {"type": "noul", "noul": 0},
1008 + "gate_2": {"type": "noul", "noul": 1},
1009 + "t0_a0": {
1010 + "type": "choice",
1011 + "choice": "high",
1012 + "confidence": 1,
1013 + "probabilities": {
1014 + "low": 0,
1015 + "medium": 0,
1016 + "high": 1,
1017 + NOT_STATED: 0,
1018 + },
1019 + },
1020 + },
1021 + }
1022 + client = Mock()
1023 + client.system_one = MagicMock(return_value=response)
1024 + provider = TypesafeProvider(client=client)
1025 + decision = provider.decide(provider.wrap_tools([TICKETS]), "open a ticket")
1026 + # `title` is required and open-ended, so the decision stays partial.
1027 + assert matches(decision, {"kind": "partial", "arguments": {"priority": "high"}})
1028 + assert decision["confidence"] == (1 + 0 + 1) / 3
1029 +
1030 +
1031 VALID_ROUTE = {
1032 "type": "choice",
1033 "choice": "TICKETS_CREATE",
@@ -1303,10 +1508,53 @@ class TestConfidenceGate:
1508 ("options", "arguments"),
1509 [
1510 ({"redact_arguments": lambda _slug, _arguments: "secret"}, {}),
1511 + (
1512 + {
1513 + "redact_arguments": lambda _slug, arguments: {
1514 + k: v for k, v in arguments.items() if k != "password"
1515 + }
1516 + },
1517 + {"password": "hunter2"},
1518 + ),
1519 + (
1520 + {
1521 + "redact_arguments": lambda _slug, arguments: {
1522 + **arguments,
1523 + "password": 42,
1524 + }
1525 + },
1526 + {},
1527 + ),
1528 + (
1529 + {
1530 + "redact_arguments": lambda _slug, arguments: {
1531 + **arguments,
1532 + "to": arguments["to"][:-1],
1533 + }
1534 + },
1535 + {"to": ["a@example.com", "b@example.com"]},
1536 + ),
1537 + (
1538 + {
1539 + "redact_arguments": lambda _slug, arguments: {
1540 + **arguments,
1541 + "meta": {"visible": arguments["meta"]["visible"]},
1542 + }
1543 + },
1544 + {"meta": {"visible": "v", "secret": "s"}},
1545 + ),
1546 ({}, {"when": datetime.datetime(2026, 1, 2)}),
1547 ({"get_context": lambda _context: {"size": float("inf")}}, {}),
1548 ],
1309 - ids=["redactor returns no dict", "arguments not JSON", "context not JSON"],
1549 + ids=[
1550 + "redactor returns no dict",
1551 + "redactor deletes a key",
1552 + "redactor retypes a value",
1553 + "redactor shortens an array",
1554 + "redactor deletes a nested key",
1555 + "arguments not JSON",
1556 + "context not JSON",
1557 + ],
1558 )
1559 def test_blocks_and_sends_nothing(self, options: t.Any, arguments: t.Any) -> None:
1560 on_bypass = Mock()
@@ -1316,9 +1564,47 @@ class TestConfidenceGate:
1564 with pytest.raises(TypesafeGateBlockedError) as caught:
1565 run_gate(gate, {**PARAMS, "arguments": arguments})
1566 assert caught.value.reason == "check_failed"
1567 + assert caught.value.__cause__ is None and caught.value.__context__ is None
1568 client.system_one.assert_not_called()
1569 on_bypass.assert_not_called()
1570
1571 + def test_a_masking_redactor_keeps_the_executed_call_original(self) -> None:
1572 + before = json.loads(json.dumps(PARAMS["arguments"]))
1573 +
1574 + def redact(_slug: str, arguments: t.Dict[str, t.Any]) -> t.Dict[str, t.Any]:
1575 + return {
1576 + **arguments,
1577 + "password": "•" * len(t.cast(str, arguments["password"])),
1578 + }
1579 +
1580 + gate, client = gate_with(0.9, redact_arguments=redact)
1581 + assert run_gate(gate) is PARAMS
1582 + assert PARAMS["arguments"] == before
1583 + sent_state = client.sent()["state"]
1584 + assert sent_state["proposed_call"]["arguments"] == {
1585 + "to": "ada@example.com",
1586 + "password": "•" * len("hunter2-SENTINEL-4"),
1587 + }
1588 +
1589 + @pytest.mark.parametrize("value", ["alow", "ALLOW", 1, None])
1590 + def test_an_invalid_on_unavailable_raises_at_construction(
1591 + self, value: t.Any
1592 + ) -> None:
1593 + # Both entry points validate: the provider method and the module function.
1594 + with pytest.raises(TypesafeInvalidOptionsError):
1595 + TypesafeProvider(client=MockClient(respond())).confidence_gate(
1596 + tools=RAW_TOOLS,
1597 + get_request=lambda _context: "email ada",
1598 + on_unavailable=value,
1599 + )
1600 + with pytest.raises(TypesafeInvalidOptionsError):
1601 + companion.confidence_gate(
1602 + ask=t.cast(t.Any, lambda *_: None),
1603 + tools=RAW_TOOLS,
1604 + get_request=lambda _context: "email ada",
1605 + on_unavailable=value,
1606 + )
1607 +
1608 def test_blocks_on_unavailability_and_allow_lets_it_through(self) -> None:
1609 with pytest.raises(TypesafeGateUnavailableError) as caught:
1610 run_gate(failing_gate(UNAVAILABLE))
ts/packages/providers/typesafe/README.md
+1 -1
@@ -199,7 +199,7 @@ await composio.tools.execute('GITHUB_CREATE_AN_ISSUE', params, { beforeExecute:
199
200 - A veto throws `TypesafeGateVetoError`. After `maxVetoes` vetoes (default 3) the gate blocks every later call in the run, so a hijacked LLM cannot vary arguments until one passes. Create a fresh gate for each new run. Never share a gate across users or runs, and never recreate it for each retry.
201 - When TypeSafe cannot be reached the gate blocks with `TypesafeGateUnavailableError`. `onUnavailable: 'allow'` lets those calls through and reports each one to `onBypass`. A tool the gate was not given, an oversized call, arguments or context that are not JSON, and a malformed response always block.
202 -- `redactArguments` gets a copy of the arguments, so it can redact in place without changing the call that runs. It must return an object. Anything else blocks the call, because falling back to the original arguments would send the secrets.
202 +- `redactArguments` gets a copy of the arguments, so it can redact in place without changing the call that runs. It must mask the arguments it got: the return must have the same JSON structure (same keys, same array lengths) with every leaf replaced by a value of the same JSON type. A redactor that deletes or adds a key, changes an array's length, or retypes a value blocks the call, because Jev must approve exactly the call that runs.
203 - One gate checks one call at a time, so concurrent calls cannot slip past `maxVetoes`. Separate gates run in parallel.
204 - The gate checks that a call is consistent with the request. It is not an authorization check. File uploads run before `beforeExecute`, so a veto happens after the upload.
205 - The gate covers direct execution only. A session applies no modifiers on `session.execute`.
ts/packages/providers/typesafe/src/classify.ts
+7 -4
@@ -3,8 +3,8 @@ import { MAX_CHOICE_OPTIONS } from './keys';
3
4 export type ArgumentClass =
5 | { kind: 'enum'; values: Array<string | number>; nullable: boolean }
6 - | { kind: 'boolean' }
7 - | { kind: 'enum_array'; values: Array<string | number>; maxItems?: number }
6 + | { kind: 'boolean'; nullable: boolean }
7 + | { kind: 'enum_array'; values: Array<string | number>; maxItems?: number; minItems?: number }
8 | { kind: 'open' };
9
10 const OPEN: ArgumentClass = { kind: 'open' };
@@ -42,7 +42,7 @@ function classifyMembers(members: unknown[], nullableHint: boolean): ArgumentCla
42 if (values.length === 0) return OPEN;
43 if (values.every(value => typeof value === 'boolean')) {
44 // The boolean Choice offers both yes and no, so a set that allows one of them stays open.
45 - return values.length === 2 ? { kind: 'boolean' } : OPEN;
45 + return values.length === 2 ? { kind: 'boolean', nullable } : OPEN;
46 }
47 const strings = values.filter((value): value is string => typeof value === 'string');
48 const integers = values.filter(
@@ -74,11 +74,14 @@ export function classifyProperty(property: JSONSchemaProperty): ArgumentClass {
74 kind: 'enum_array',
75 values: items.values,
76 ...(property.maxItems === undefined ? {} : { maxItems: property.maxItems }),
77 + ...(property.minItems === undefined ? {} : { minItems: property.minItems }),
78 };
79 }
80
81 const members = literalMembers(property);
82 if (members !== undefined) return classifyMembers(members, nullableHint);
82 - if (valueTypes.length === 1 && valueTypes[0] === 'boolean') return { kind: 'boolean' };
83 + if (valueTypes.length === 1 && valueTypes[0] === 'boolean') {
84 + return { kind: 'boolean', nullable: nullableHint };
85 + }
86 return OPEN;
87 }
ts/packages/providers/typesafe/src/companion.ts
+54 -5
@@ -88,6 +88,42 @@ function jsonOrBlock(value: unknown): TypesafeJsonValue {
88 }
89 }
90
91 +const scalarTypeOf = (value: TypesafeJsonValue): string => (value === null ? 'null' : typeof value);
92 +
93 +/**
94 + * The redactor is a masker: the redacted arguments must have exactly the original's JSON
95 + * structure — the same key sets, the same array lengths — and every leaf replacement must
96 + * keep the same scalar type. Anything else blocks, because Jev would approve a call that
97 + * differs from the one that runs.
98 + */
99 +function assertMaskingOnly(original: TypesafeJsonValue, redacted: TypesafeJsonValue): void {
100 + if (Array.isArray(original)) {
101 + if (!Array.isArray(redacted) || redacted.length !== original.length) {
102 + throw new TypesafeGateBlockedError('check_failed');
103 + }
104 + original.forEach((entry, index) => assertMaskingOnly(entry, redacted[index]));
105 + return;
106 + }
107 + if (original !== null && typeof original === 'object') {
108 + if (redacted === null || typeof redacted !== 'object' || Array.isArray(redacted)) {
109 + throw new TypesafeGateBlockedError('check_failed');
110 + }
111 + const originalKeys = Object.keys(original).sort();
112 + const redactedKeys = Object.keys(redacted).sort();
113 + if (
114 + originalKeys.length !== redactedKeys.length ||
115 + originalKeys.some((key, index) => key !== redactedKeys[index])
116 + ) {
117 + throw new TypesafeGateBlockedError('check_failed');
118 + }
119 + for (const key of originalKeys) assertMaskingOnly(original[key], redacted[key]);
120 + return;
121 + }
122 + if (scalarTypeOf(original) !== scalarTypeOf(redacted)) {
123 + throw new TypesafeGateBlockedError('check_failed');
124 + }
125 +}
126 +
127 const AVAILABILITY_REASONS: ReadonlyArray<string> = [
128 'timeout',
129 'connection',
@@ -113,6 +149,14 @@ export function confidenceGate(options: TypesafeGateOptions, ask: Ask): beforeEx
149 if (!Number.isInteger(maxVetoes) || maxVetoes < 1) {
150 throw new TypesafeInvalidOptionsError('`maxVetoes` must be a positive integer.');
151 }
152 + if (
153 + options.onUnavailable !== undefined &&
154 + options.onUnavailable !== 'block' &&
155 + options.onUnavailable !== 'allow'
156 + ) {
157 + // A typo'd mode must never fail open at check time.
158 + throw new TypesafeInvalidOptionsError("`onUnavailable` must be 'block' or 'allow'.");
159 + }
160 const tools = new Map(options.tools.map(tool => [tool.slug, tool]));
161 let vetoes = 0;
162 // Checks run one at a time. Concurrent calls would otherwise all read the veto count
@@ -133,17 +177,22 @@ export function confidenceGate(options: TypesafeGateOptions, ask: Ask): beforeEx
177 // The redactor works on a copy, so redacting in place cannot alter the executed call.
178 // A return that is not an object blocks: falling back would send the secrets.
179 // The originals are checked first, since cloning flattens a class instance into JSON.
136 - jsonOrBlock(callArguments);
137 - sentArguments = options.redactArguments(tool.slug, structuredClone(callArguments));
138 - if (!RedactedArgumentsSchema.safeParse(sentArguments).success) {
180 + const original = jsonOrBlock(callArguments);
181 + const redacted = options.redactArguments(tool.slug, structuredClone(callArguments));
182 + if (!RedactedArgumentsSchema.safeParse(redacted).success) {
183 throw new TypesafeGateBlockedError('check_failed');
184 }
185 + // Jev evaluates what is sent, and the original arguments run. The redactor may only
186 + // mask, so an approved call is exactly the call that executes.
187 + assertMaskingOnly(original, jsonOrBlock(redacted));
188 + sentArguments = redacted;
189 }
190 // Only the slug and the arguments are copied out of the execution parameters.
143 - // `userId`, `connectedAccountId`, and the custom auth fields never leave.
191 + // `userId`, `connectedAccountId`, and the custom auth fields never leave. The `context`
192 + // key exists only when there is one, so a context-less state stays context-less.
193 const state = jsonOrBlock({
194 request,
146 - context: gateContext,
195 + ...(gateContext === undefined ? {} : { context: gateContext }),
196 proposed_call: {
197 tool: tool.slug,
198 description: tool.description ?? tool.name,
ts/packages/providers/typesafe/src/compile.ts
+7 -3
@@ -80,15 +80,18 @@ function compileArgument(
80 { key: 'yes', value: true },
81 { key: 'no', value: false },
82 ];
83 + if (argumentClass.nullable) options.push({ key: NULL_KEY, value: null });
84 return {
85 kind: 'choice',
86 name: argument,
87 required,
88 questionId,
89 question: choiceQuestion(toolName, argument, description, options, option =>
89 - option.value === true
90 - ? `The request states that "${argument}" is true.`
91 - : `The request states that "${argument}" is false.`
90 + option.value === null
91 + ? `The request explicitly asks for no value (null) for "${argument}".`
92 + : option.value === true
93 + ? `The request states that "${argument}" is true.`
94 + : `The request states that "${argument}" is false.`
95 ),
96 options,
97 notStatedKey: NOT_STATED_KEY,
@@ -133,6 +136,7 @@ function compileArgument(
136 },
137 })),
138 ...(argumentClass.maxItems === undefined ? {} : { maxItems: argumentClass.maxItems }),
139 + ...(argumentClass.minItems === undefined ? {} : { minItems: argumentClass.minItems }),
140 };
141 }
142
ts/packages/providers/typesafe/src/decide.ts
+6 -3
@@ -108,13 +108,15 @@ const notJson = (): TypesafeInvalidOptionsError =>
108
109 /**
110 * Rebuilds a JSON value with sorted object keys, so equal states serialize identically.
111 - * Throws on a value JSON cannot hold, which `JSON.stringify` would drop or rewrite silently.
111 + * Throws on a value JSON cannot hold, which `JSON.stringify` would drop or rewrite silently:
112 + * `undefined` is rejected wherever it appears, and `null` stays JSON null.
113 */
114 export function stable(
115 value: unknown,
116 ancestors: ReadonlySet<object> = new Set()
117 ): TypesafeJsonValue {
117 - if (value === null || value === undefined) return null;
118 + if (value === null) return null;
119 + if (value === undefined) throw notJson();
120 if (typeof value === 'string' || typeof value === 'boolean') return value;
121 if (typeof value === 'number') {
122 if (!Number.isFinite(value)) throw notJson();
@@ -130,7 +132,6 @@ export function stable(
132 // `fromEntries` defines own properties, so a `__proto__` key stays data.
133 return Object.fromEntries(
134 Object.entries(value)
133 - .filter(([, entry]) => entry !== undefined)
135 .sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0))
136 .map(([key, entry]) => [key, stable(entry, path)])
137 );
@@ -269,6 +270,8 @@ function readArgument(
270 .sort((left, right) => right.probability - left.probability || left.index - right.index)
271 .slice(0, argument.maxItems ?? members.length)
272 .sort((left, right) => left.index - right.index);
273 + // An undersized selection is not what the request stated, so nothing is bound.
274 + if (selected.length < (argument.minItems ?? 0)) return { kind: 'not_stated' };
275 const score = Math.min(
276 mentioned,
277 ...members.map(member => Math.max(member.probability, 1 - member.probability))
ts/packages/providers/typesafe/src/types.ts
+6 -1
@@ -85,6 +85,8 @@ export interface TypesafeArrayArgument {
85 mentioned: TypesafeNoulQuestion;
86 members: Array<{ questionId: string; value: string | number; question: TypesafeNoulQuestion }>;
87 maxItems?: number;
88 + /** A selection with fewer members than this is not stated, so the argument stays missing. */
89 + minItems?: number;
90 }
91
92 export type TypesafeArgumentQuestion = TypesafeChoiceArgument | TypesafeArrayArgument;
@@ -261,7 +263,10 @@ export interface TypesafeGateOptions {
263 maxVetoes?: number;
264 /**
265 * Replace secret-bearing arguments before they are sent to TypeSafe. It gets a copy, so the
264 - * executed call never changes. A return that is not an object blocks the call.
266 + * executed call never changes. The return must be a masking of the input: the same JSON
267 + * structure (same keys, same array lengths) with every leaf replaced by a value of the same
268 + * scalar type. Anything that deletes, adds, retypes, or is not an object blocks the call,
269 + * because Jev must approve the call that actually runs.
270 */
271 redactArguments?: (toolSlug: string, args: Record<string, unknown>) => Record<string, unknown>;
272 model?: string;
ts/packages/providers/typesafe/test/classify.test.ts
+23 -5
@@ -36,17 +36,27 @@ describe('classifyProperty', () => {
36 });
37
38 it('classifies booleans, including a boolean enum', () => {
39 - expect(classifyProperty({ type: 'boolean' })).toEqual({ kind: 'boolean' });
40 - expect(classifyProperty({ enum: [true, false] })).toEqual({ kind: 'boolean' });
39 + expect(classifyProperty({ type: 'boolean' })).toEqual({ kind: 'boolean', nullable: false });
40 + expect(classifyProperty({ enum: [true, false] })).toEqual({
41 + kind: 'boolean',
42 + nullable: false,
43 + });
44 });
45
46 it.each([
47 [{ type: 'boolean', enum: [false, true] }],
48 [{ anyOf: [{ const: true }, { const: false }] }],
49 + ])('classifies %j as a non-nullable boolean, since it allows both values', schema => {
50 + expect(classifyProperty(schema)).toEqual({ kind: 'boolean', nullable: false });
51 + });
52 +
53 + it.each([
54 + [{ type: ['boolean', 'null'] }],
55 + [{ type: 'boolean', nullable: true }],
56 [{ oneOf: [{ enum: [true] }, { enum: [false] }, { type: 'null' }] }],
57 [{ enum: [true, false, null] }],
48 - ])('classifies %j as boolean, since it allows both values', schema => {
49 - expect(classifyProperty(schema)).toEqual({ kind: 'boolean' });
58 + ])('classifies %j as a nullable boolean, so Jev can bind null', schema => {
59 + expect(classifyProperty(schema)).toEqual({ kind: 'boolean', nullable: true });
60 });
61
62 // A yes/no Choice could bind the value the schema forbids.
@@ -63,10 +73,18 @@ describe('classifyProperty', () => {
73 expect(classifyProperty(schema)).toEqual({ kind: 'open' });
74 });
75
66 - it('classifies an array of enum values and keeps maxItems', () => {
76 + it('classifies an array of enum values and keeps maxItems and minItems', () => {
77 expect(
78 classifyProperty({ type: 'array', items: { type: 'string', enum: ['a', 'b'] }, maxItems: 1 })
79 ).toEqual({ kind: 'enum_array', values: ['a', 'b'], maxItems: 1 });
80 + expect(
81 + classifyProperty({
82 + type: 'array',
83 + items: { type: 'string', enum: ['a', 'b'] },
84 + maxItems: 2,
85 + minItems: 1,
86 + })
87 + ).toEqual({ kind: 'enum_array', values: ['a', 'b'], maxItems: 2, minItems: 1 });
88 });
89
90 it('keeps a single-member enum and a scalar const closed', () => {
ts/packages/providers/typesafe/test/companion.test.ts
+75
@@ -6,6 +6,7 @@ import {
6 TypesafeGateBlockedError,
7 TypesafeGateUnavailableError,
8 TypesafeGateVetoError,
9 + TypesafeInvalidOptionsError,
10 TypesafeProvider,
11 type TypesafeGateOptions,
12 type TypesafeSystemOneRequest,
@@ -123,9 +124,17 @@ describe('confidenceGate', () => {
124 request: 'email ada the report',
125 proposed_call: { tool: 'GMAIL_SEND_EMAIL', arguments: { to: 'ada@example.com' } },
126 });
127 + // A context-less gate sends no context key at all, not an empty one.
128 + expect(request.state).not.toHaveProperty('context');
129 expect(JSON.stringify(request.questions)).not.toContain('ada@example.com');
130 });
131
132 + it('sends context exactly as getContext returned it when there is one', async () => {
133 + const { gate, systemOne } = gateWith(0.9, { getContext: () => ({ thread: 't1' }) });
134 + await gate(context);
135 + expect(sent(systemOne).state).toMatchObject({ context: { thread: 't1' } });
136 + });
137 +
138 it('hands redactArguments a copy, so redacting in place leaves the executed call alone', async () => {
139 const before = structuredClone(params.arguments);
140 const { gate, systemOne } = gateWith(0.9, {
@@ -141,8 +150,57 @@ describe('confidenceGate', () => {
150 expect(JSON.stringify(sent(systemOne))).toContain('[redacted]');
151 });
152
153 + it('lets a masking redactor rewrite nested values and keeps the executed call original', async () => {
154 + const before = structuredClone(params.arguments);
155 + const { gate, systemOne } = gateWith(0.9, {
156 + getContext: () => ({ thread: 't1' }),
157 + redactArguments: (_slug, args) => ({
158 + ...args,
159 + password: String(args.password).replaceAll(/./g, '•'),
160 + }),
161 + });
162 + const executed = await gate(context);
163 + expect(executed).toBe(params);
164 + expect(executed.arguments).toEqual(before);
165 + const call = sent(systemOne).state as {
166 + proposed_call: { arguments: Record<string, unknown> };
167 + };
168 + expect(call.proposed_call.arguments).toEqual({
169 + to: 'ada@example.com',
170 + password: '•'.repeat(String(params.arguments?.password).length),
171 + });
172 + });
173 +
174 it.each<[string, Partial<TypesafeGateOptions>, Record<string, unknown>]>([
175 ['a redactor that returns no object', { redactArguments: () => 'secret' as never }, {}],
176 + [
177 + 'a redactor that deletes a key',
178 + {
179 + redactArguments: (_slug, { password: _deleted, ...rest }) =>
180 + rest as Record<string, unknown>,
181 + },
182 + { password: 'hunter2' },
183 + ],
184 + [
185 + 'a redactor that changes a leaf type',
186 + { redactArguments: (_slug, args) => ({ ...args, password: 42 }) },
187 + {},
188 + ],
189 + [
190 + 'a redactor that shortens an array',
191 + { redactArguments: (_slug, args) => ({ ...args, to: (args.to as string[]).slice(1) }) },
192 + { to: ['a@example.com', 'b@example.com'] },
193 + ],
194 + [
195 + 'a redactor that deletes a nested key',
196 + {
197 + redactArguments: (_slug, args) => ({
198 + ...args,
199 + meta: { visible: (args.meta as { visible: string }).visible },
200 + }),
201 + },
202 + { meta: { visible: 'v', secret: 's' } },
203 + ],
204 ['arguments that are not JSON', {}, { when: new Date(0) }],
205 ['a context that is not JSON', { getContext: () => ({ size: 1n }) as never }, {}],
206 ])('blocks and sends nothing for %s', async (_label, options, callArguments) => {
@@ -155,6 +213,23 @@ describe('confidenceGate', () => {
213 expect(onBypass).not.toHaveBeenCalled();
214 });
215
216 + it.each<[string, unknown]>([
217 + ['a typo', 'alow'],
218 + ['uppercase', 'ALLOW'],
219 + ['a non-string', 1],
220 + ])(
221 + 'rejects an onUnavailable of %s when the gate is built, not at check time',
222 + (_label, value) => {
223 + expect(() =>
224 + new TypesafeProvider({ client: mockClient(() => undefined).client }).confidenceGate({
225 + tools,
226 + getRequest: () => 'email ada',
227 + onUnavailable: value as never,
228 + })
229 + ).toThrow(TypesafeInvalidOptionsError);
230 + }
231 + );
232 +
233 const unavailable = Object.assign(new Error('down'), {
234 name: 'InternalServerError',
235 status: 503,
ts/packages/providers/typesafe/test/decide.test.ts
+79
@@ -167,6 +167,70 @@ describe('decide outcomes', () => {
167 ).toMatchObject({ arguments: { pick: ['b'] } });
168 });
169
170 + it('treats a selection below minItems as not stated, so a required array stays missing', async () => {
171 + const tool = makeTool(
172 + 'PICK_SOME',
173 + {
174 + pick: { type: 'array', items: { enum: ['a', 'b'] }, minItems: 1 },
175 + pick_many: { type: 'array', items: { enum: ['x', 'y'] }, minItems: 2 },
176 + },
177 + ['pick']
178 + );
179 + const compileFor = new TypesafeProvider();
180 + expect(compileFor.wrapTool(tool).arguments).toEqual([
181 + expect.objectContaining({ name: 'pick', minItems: 1 }),
182 + expect.objectContaining({ name: 'pick_many', minItems: 2 }),
183 + ]);
184 +
185 + // "mentioned" with zero members selected leaves the required argument missing.
186 + const none = setup({ route: ['PICK_SOME', 0.9], answers: { t0_a0_mentioned: 0.9 } });
187 + expect(await none.provider.decide(none.provider.wrapTools([tool]), 'pick some')).toMatchObject({
188 + kind: 'partial',
189 + missing: [['pick']],
190 + arguments: {},
191 + });
192 +
193 + // Enough members selected binds the argument.
194 + const enough = setup({
195 + route: ['PICK_SOME', 0.9],
196 + answers: { t0_a0_mentioned: 0.9, t0_a0_m0: 0.9 },
197 + });
198 + expect(await enough.provider.decide(enough.provider.wrapTools([tool]), 'pick a')).toMatchObject(
199 + { kind: 'call', arguments: { pick: ['a'] } }
200 + );
201 +
202 + // An optional argument that stays below minItems is not bound and not dropped.
203 + const optional = setup({
204 + route: ['PICK_SOME', 0.9],
205 + answers: { t0_a0_mentioned: 0.9, t0_a0_m0: 0.9, t0_a1_mentioned: 0.9, t0_a1_m0: 0.9 },
206 + });
207 + expect(
208 + await optional.provider.decide(optional.provider.wrapTools([tool]), 'pick some')
209 + ).toMatchObject({ kind: 'call', arguments: { pick: ['a'] }, dropped: [] });
210 + });
211 +
212 + it('offers null for a nullable boolean, binds it, and keeps a plain boolean at yes/no', async () => {
213 + const wrapFor = new TypesafeProvider();
214 + const nullable = makeTool('CLEAR_FLAG', { flag: { type: ['boolean', 'null'] } });
215 + const compiled = wrapFor.wrapTool(nullable).arguments[0];
216 + expect(compiled.kind === 'choice' && compiled.options).toEqual([
217 + { key: 'yes', value: true },
218 + { key: 'no', value: false },
219 + { key: '__null__', value: null },
220 + ]);
221 + const bound = setup({ route: ['CLEAR_FLAG', 0.9], answers: { t0_a0: ['__null__', 0.9] } });
222 + expect(
223 + await bound.provider.decide(bound.provider.wrapTools([nullable]), 'clear the flag')
224 + ).toMatchObject({ kind: 'call', arguments: { flag: null } });
225 +
226 + const plain = makeTool('SET_FLAG', { flag: { type: 'boolean' } });
227 + const plainCompiled = wrapFor.wrapTool(plain).arguments[0];
228 + expect(plainCompiled.kind === 'choice' && plainCompiled.options).toEqual([
229 + { key: 'yes', value: true },
230 + { key: 'no', value: false },
231 + ]);
232 + });
233 +
234 it('restores typed values: integers, booleans, and null', async () => {
235 const tool = makeTool('TYPED', {
236 weight: { type: 'integer', enum: [1, 2, 3] },
@@ -339,6 +403,8 @@ describe('state', () => {
403 ['NaN in context', { request: 'open a ticket', context: { nested: [Number.NaN] } }],
404 ['a Date in context', { request: 'open a ticket', context: new Date(0) }],
405 ['a bigint in context', { request: 'open a ticket', context: 1n }],
406 + ['an undefined context property', { request: 'open a ticket', context: { a: undefined } }],
407 + ['an undefined array entry', { request: 'open a ticket', context: [1, undefined] }],
408 ])('rejects %s and sends nothing', async (_label, state) => {
409 const { provider, systemOne } = setup({ route: ['TICKETS_CREATE', 0.9] });
410 await expect(
@@ -346,6 +412,19 @@ describe('state', () => {
412 ).rejects.toBeInstanceOf(TypesafeInvalidOptionsError);
413 expect(systemOne).not.toHaveBeenCalled();
414 });
415 +
416 + it('keeps JSON null in context while rejecting undefined', async () => {
417 + const { provider, systemOne } = setup({ route: ['TICKETS_CREATE', 0.9] });
418 + await provider.decide(provider.wrapTools([tickets]), {
419 + request: 'open a ticket',
420 + context: { a: null, nested: [null] },
421 + });
422 + // Context splits routing from argument questions, so the argument request carries it.
423 + expect(sentRequest(systemOne, 1).state).toEqual({
424 + request: 'open a ticket',
425 + context: { a: null, nested: [null] },
426 + });
427 + });
428 });
429
430 describe('request and context separation', () => {