Support foldable simulators in xcrs
An iPhone Duo has two integrated displays and only the one in use shows content, so a plain screenshot of an unfolded device was a black frame. - Simctl::displays reads the integrated displays from the device type's capabilities.plist, cached per device type. screenshot takes an optional display. - screen_capture accepts display (outer, inner, all or a screen ID). Without it, a device with more than one display returns a labelled image for each. Single-display devices return one image as before. - device_capabilities adds displays and foldable for simulators. - posture_get reports the runner's hinge status and angle and which display is live. posture_set presses the matching Device Hub hinge button through xcrs-host-inspect and polls until the hinge changes, and does nothing when it is already in that state. - Both tools are added to the standalone server and to smb --mcp --scope automation. - docs/controlkit.md describes the Duo displays, the fold mechanism and its limits, and the runner methods. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Seto Elkahfi committed
Oct 26, 2026 at 00:14 UTC
570fe29fbe9f31ae37f4f919e3711b4ca49955cd
5 files changed
+847
-33
crates/cli/src/mcp.rs
+3
-1
@@ -2051,7 +2051,7 @@ mod tests {
2051
let tools = SmbMcpServer::cloud_tool_router().list_all();
2052
let names: Vec<&str> = tools.iter().map(|tool| tool.name.as_ref()).collect();
2053
2054
- const AUTOMATION_TOOL_NAMES: [&str; 19] = [
2054
+ const AUTOMATION_TOOL_NAMES: [&str; 21] = [
2055
"device_list",
2056
"device_select",
2057
"device_capabilities",
@@ -2071,6 +2071,8 @@ mod tests {
2071
"input_spatial_tap",
2072
"orientation_get",
2073
"orientation_set",
2074
+ "posture_get",
2075
+ "posture_set",
2076
];
2077
2078
for automation_only in AUTOMATION_TOOL_NAMES {
crates/cli/src/mcp_xcrs.rs
+7
-3
@@ -46,7 +46,9 @@ xcrs::xcrs_mcp_tools!(
46
"input_click",
47
"input_spatial_tap",
48
"orientation_get",
49
- "orientation_set"
49
+ "orientation_set",
50
+ "posture_get",
51
+ "posture_set"
52
);
53
54
#[rmcp::tool_handler(router = Self::xcrs_tool_router())]
@@ -89,7 +91,7 @@ pub async fn serve() -> Result<()> {
91
mod tests {
92
use super::*;
93
92
- const CANONICAL_TOOL_NAMES: [&str; 19] = [
94
+ const CANONICAL_TOOL_NAMES: [&str; 21] = [
95
"device_list",
96
"device_select",
97
"device_capabilities",
@@ -109,10 +111,12 @@ mod tests {
111
"input_spatial_tap",
112
"orientation_get",
113
"orientation_set",
114
+ "posture_get",
115
+ "posture_set",
116
];
117
118
#[test]
115
- fn automation_router_exposes_exactly_the_canonical_nineteen_tools() {
119
+ fn automation_router_exposes_exactly_the_canonical_twentyone_tools() {
120
let tools = AutomationMcpServer::xcrs_tool_router().list_all();
121
let mut names: Vec<&str> = tools.iter().map(|tool| tool.name.as_ref()).collect();
122
names.sort_unstable();
crates/xcrs/src/mcp.rs
+391
-25
@@ -5,10 +5,14 @@
5
//! `device_capabilities`, `app_install_launch`, `screen_capture`, `app_launch`,
6
//! `app_terminate`, `url_open`, `ui_describe`, `ui_element_list`, `input_tap`,
7
//! `input_text`, `input_swipe`, `input_button`, `input_click`, `input_spatial_tap`,
8
-//! `orientation_get`, and `orientation_set`. Each name is supplied by the embedder
8
+//! `orientation_get`, `orientation_set`, `posture_get`, and `posture_set`. Each name is supplied by the embedder
9
//! through [`xcrs_mcp_tools`] so the standalone crate and CLI profile share one
10
//! implementation and one public contract.
11
//!
12
+//! `screen_capture` takes a `display` for foldable simulators such as iPhone Duo,
13
+//! which have an outer and an inner display; `posture_get` and `posture_set` read
14
+//! and change the fold state.
15
+//!
16
//! Targets are Apple simulators, Apple ControlKit hosts (physical/remote devices or
17
//! a local macOS runner), or Android devices reachable through adb. A target can be
18
//! selected once with `device_select` and reused by later calls, or passed explicitly
@@ -336,6 +340,92 @@ impl Orientation {
340
}
341
}
342
343
+/// The fold state `posture_set` can request on a foldable simulator.
344
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, JsonSchema)]
345
+#[serde(rename_all = "lowercase")]
346
+pub enum Posture {
347
+ /// Hinge closed: the outer display is live.
348
+ Folded,
349
+ /// Hinge fully open: the inner display is live.
350
+ Unfolded,
351
+}
352
+
353
+impl Posture {
354
+ pub fn as_str(self) -> &'static str {
355
+ match self {
356
+ Self::Folded => "folded",
357
+ Self::Unfolded => "unfolded",
358
+ }
359
+ }
360
+
361
+ /// The Device Hub hinge button that sets this posture.
362
+ pub fn hinge_button(self) -> &'static str {
363
+ match self {
364
+ Self::Folded => "closed",
365
+ Self::Unfolded => "open",
366
+ }
367
+ }
368
+
369
+ /// Whether a runner's `device.hinge.get` status means this posture.
370
+ pub fn matches_hinge_status(self, status: &str) -> bool {
371
+ status
372
+ == match self {
373
+ Self::Folded => "closed",
374
+ Self::Unfolded => "fullyOpen",
375
+ }
376
+ }
377
+}
378
+
379
+/// The display that shows content for a runner hinge status: the outer display
380
+/// when the hinge is closed, the inner display otherwise. `None` when the status
381
+/// is unknown or the device does not have an outer and an inner display.
382
+pub fn live_display<'a>(
383
+ hinge_status: &str,
384
+ displays: &'a [crate::SimDisplay],
385
+) -> Option<&'a crate::SimDisplay> {
386
+ let role = match hinge_status {
387
+ "closed" => "outer",
388
+ "partiallyOpen" | "fullyOpen" => "inner",
389
+ _ => return None,
390
+ };
391
+ displays.iter().find(|display| display.role == role)
392
+}
393
+
394
+#[derive(Debug, Deserialize, JsonSchema)]
395
+pub struct PostureGetArgs {
396
+ /// Exact Apple simulator name. Omit to use the target selected with
397
+ /// `device_select`.
398
+ #[serde(default)]
399
+ pub simulator_name: Option<String>,
400
+ /// Apple simulator UDID. Omit to use the target selected with `device_select`.
401
+ #[serde(default)]
402
+ pub simulator_udid: Option<String>,
403
+ /// ControlKit host of a physical device or remote runner. Omit for a local
404
+ /// simulator.
405
+ #[serde(default)]
406
+ pub host: Option<String>,
407
+ /// Local ControlKit JSON-RPC port. Defaults to 12004.
408
+ #[serde(default)]
409
+ pub controlkit_port: Option<u16>,
410
+}
411
+
412
+#[derive(Debug, Deserialize, JsonSchema)]
413
+pub struct PostureSetArgs {
414
+ /// Exact Apple simulator name. Omit to use the target selected with
415
+ /// `device_select`.
416
+ #[serde(default)]
417
+ pub simulator_name: Option<String>,
418
+ /// Apple simulator UDID. Omit to use the target selected with `device_select`.
419
+ #[serde(default)]
420
+ pub simulator_udid: Option<String>,
421
+ /// Local ControlKit JSON-RPC port. Defaults to 12004.
422
+ #[serde(default)]
423
+ pub controlkit_port: Option<u16>,
424
+ /// `folded` closes the hinge so the outer display is live; `unfolded` opens it
425
+ /// fully so the inner display is live.
426
+ pub posture: Posture,
427
+}
428
+
429
/// Static, hand-maintained capability description for Android targets. Unlike
430
/// Apple's `device.capabilities` ControlKit call, adb has no capability-discovery
431
/// RPC, so this mirrors exactly the actions `AndroidDebugBridge` implements.
@@ -469,11 +559,56 @@ pub struct ScreenCaptureArgs {
559
/// field.
560
#[serde(default)]
561
pub device: Option<String>,
562
+ /// Which display of an Apple simulator to capture: `outer`, `inner`, `all`, or
563
+ /// a screen ID from `device_capabilities`. Omit it to capture every display of
564
+ /// a foldable device (such as iPhone Duo), or the only display of any other.
565
+ /// Only valid for Apple simulators.
566
+ #[serde(default)]
567
+ pub display: Option<String>,
568
/// Android device serial from `adb devices -l`.
569
#[serde(default)]
570
pub android_serial: Option<String>,
571
}
572
573
+/// Capture the requested displays of a simulator as MCP content. A selection of
574
+/// more than one display yields a text label followed by an image per display;
575
+/// a single display yields just the image, as it always has.
576
+pub fn capture_simulator(
577
+ udid: &str,
578
+ display: Option<&str>,
579
+) -> Result<Vec<rmcp::model::ContentBlock>> {
580
+ let tools = crate::XcodeCommandLineTools::new();
581
+ let simctl = tools.simctl();
582
+
583
+ let displays = match simctl.displays(udid) {
584
+ Ok(displays) => displays,
585
+ // Without display metadata only the primary display can be captured.
586
+ Err(_) if display.is_none() => {
587
+ let screenshot = simctl.screenshot(udid, None)?;
588
+ return Ok(vec![rmcp::model::ContentBlock::image(
589
+ crate::encode_base64(screenshot),
590
+ "image/png",
591
+ )]);
592
+ }
593
+ Err(error) => return Err(error),
594
+ };
595
+
596
+ let selected = crate::select_displays(&displays, display)?;
597
+ let labelled = selected.len() > 1;
598
+ let mut content = Vec::new();
599
+ for entry in selected {
600
+ let screenshot = simctl.screenshot(udid, Some(&entry.screen_id.to_string()))?;
601
+ if labelled {
602
+ content.push(rmcp::model::ContentBlock::text(entry.describe()));
603
+ }
604
+ content.push(rmcp::model::ContentBlock::image(
605
+ crate::encode_base64(screenshot),
606
+ "image/png",
607
+ ));
608
+ }
609
+ Ok(content)
610
+}
611
+
612
#[derive(Debug, Deserialize, JsonSchema)]
613
pub struct AppTargetArgs {
614
/// Exact Apple simulator name. Omit to use the target selected with
@@ -749,7 +884,9 @@ macro_rules! xcrs_mcp_tools {
884
$input_click_name:literal,
885
$input_spatial_tap_name:literal,
886
$orientation_get_name:literal,
752
- $orientation_set_name:literal
887
+ $orientation_set_name:literal,
888
+ $posture_get_name:literal,
889
+ $posture_set_name:literal
890
) => {
891
#[::rmcp::tool_router(router = xcrs_tool_router, vis = "pub(crate)")]
892
impl $server {
@@ -1085,12 +1222,23 @@ macro_rules! xcrs_mcp_tools {
1222
.map_err(|error| {
1223
::rmcp::model::ErrorData::internal_error(error.to_string(), None)
1224
})?;
1225
+ let displays = simulator.as_ref().and_then(|simulator| {
1226
+ $crate::XcodeCommandLineTools::new()
1227
+ .simctl()
1228
+ .displays(&simulator.udid)
1229
+ .ok()
1230
+ });
1231
+ let mut payload = ::serde_json::json!({
1232
+ "platform": "apple",
1233
+ "simulator": simulator,
1234
+ "capabilities": result,
1235
+ });
1236
+ if let (Some(displays), Some(object)) = (displays, payload.as_object_mut()) {
1237
+ object.insert("foldable".to_string(), ::serde_json::json!(displays.len() > 1));
1238
+ object.insert("displays".to_string(), ::serde_json::json!(displays));
1239
+ }
1240
Ok(::rmcp::model::CallToolResult::success(vec![
1089
- ::rmcp::model::ContentBlock::json(&::serde_json::json!({
1090
- "platform": "apple",
1091
- "simulator": simulator,
1092
- "capabilities": result,
1093
- }))?,
1241
+ ::rmcp::model::ContentBlock::json(&payload)?,
1242
]))
1243
}
1244
}
@@ -1140,7 +1288,7 @@ macro_rules! xcrs_mcp_tools {
1288
name = $screen_capture_name,
1289
title = "Capture screenshot",
1290
annotations(title = "Capture screenshot", read_only_hint = true, idempotent_hint = true),
1143
- description = "Purpose: capture a PNG screenshot of the resolved target's current screen. When to use vs siblings: use this for a pixel image; use ui_describe or ui_element_list instead when you need element coordinates or labels rather than pixels. Behavior: for an Apple simulator, captures through `simctl io screenshot`; for a paired physical Apple device, pass its CoreDevice identifier as `device` to capture through `devicectl` (this bypasses simulator/host/android_serial resolution entirely and is mutually exclusive with them); for an Android device, captures through `adb exec-out screencap`. Prerequisites: the target must be booted/connected and, for simulators, `simctl` must be able to reach it. Failure modes: returns an error if `device` is combined with any other target field, if the resolved target is an Apple ControlKit host (screenshot over ControlKit is not implemented; use `device` or select a simulator instead), or if the underlying capture command fails."
1291
+ description = "Purpose: capture a PNG screenshot of the resolved target's current screen. When to use vs siblings: use this for a pixel image; use ui_describe or ui_element_list instead when you need element coordinates or labels rather than pixels. Behavior: for an Apple simulator, captures through `simctl io screenshot`; a foldable simulator such as iPhone Duo has two displays (outer and inner), and only the one currently in use shows content, so by default every display is captured, each preceded by a text block naming it (for example `screen 1 outer 1398x2034`), and `display` (`outer`, `inner`, `all` or a screen ID) narrows that; for a paired physical Apple device, pass its CoreDevice identifier as `device` to capture through `devicectl` (this bypasses simulator/host/android_serial resolution entirely and is mutually exclusive with them); for an Android device, captures through `adb exec-out screencap`. Prerequisites: the target must be booted/connected and, for simulators, `simctl` must be able to reach it. Failure modes: returns an error if `device` is combined with any other target field, if `display` is given for a physical device or an Android device or names a display the simulator does not have, if the resolved target is an Apple ControlKit host (screenshot over ControlKit is not implemented; use `device` or select a simulator instead), or if the underlying capture command fails."
1292
)]
1293
async fn screen_capture(
1294
&self,
@@ -1154,6 +1302,12 @@ macro_rules! xcrs_mcp_tools {
1302
::rmcp::model::ErrorData,
1303
> {
1304
if let Some(device) = &args.device {
1305
+ if args.display.is_some() {
1306
+ return Err(::rmcp::model::ErrorData::invalid_request(
1307
+ "display only applies to Apple simulators, not to a physical device.",
1308
+ None,
1309
+ ));
1310
+ }
1311
if args.simulator_name.is_some()
1312
|| args.simulator_udid.is_some()
1313
|| args.host.is_some()
@@ -1185,17 +1339,27 @@ macro_rules! xcrs_mcp_tools {
1339
args.controlkit_port,
1340
args.android_serial,
1341
)?;
1188
- let screenshot = match &target {
1342
+ let content = match &target {
1343
$crate::mcp::SelectedTarget::Android { serial } => {
1344
+ if args.display.is_some() {
1345
+ return Err(::rmcp::model::ErrorData::invalid_request(
1346
+ "display only applies to Apple simulators, not to Android devices.",
1347
+ None,
1348
+ ));
1349
+ }
1350
let device = Self::android_device_for(serial.clone()).await?;
1351
let device_serial = device.serial.clone();
1192
- Self::run_android_blocking(move || {
1352
+ let screenshot = Self::run_android_blocking(move || {
1353
$crate::AndroidDebugBridge::new().screenshot(&device_serial)
1354
})
1355
.await
1356
.map_err(|error| {
1357
::rmcp::model::ErrorData::internal_error(error.to_string(), None)
1198
- })?
1358
+ })?;
1359
+ vec![::rmcp::model::ContentBlock::image(
1360
+ $crate::encode_base64(screenshot),
1361
+ "image/png",
1362
+ )]
1363
}
1364
$crate::mcp::SelectedTarget::AppleSimulator {
1365
simulator_name,
@@ -1203,9 +1367,7 @@ macro_rules! xcrs_mcp_tools {
1367
..
1368
} => {
1369
let simulator = Self::simulator_for(simulator_name, simulator_udid)?;
1206
- $crate::XcodeCommandLineTools::new()
1207
- .simctl()
1208
- .screenshot(&simulator.udid)
1370
+ $crate::mcp::capture_simulator(&simulator.udid, args.display.as_deref())
1371
.map_err(|error| {
1372
::rmcp::model::ErrorData::internal_error(error.to_string(), None)
1373
})?
@@ -1219,12 +1381,7 @@ macro_rules! xcrs_mcp_tools {
1381
));
1382
}
1383
};
1222
- Ok(::rmcp::model::CallToolResult::success(vec![
1223
- ::rmcp::model::ContentBlock::image(
1224
- $crate::encode_base64(screenshot),
1225
- "image/png",
1226
- ),
1227
- ]))
1384
+ Ok(::rmcp::model::CallToolResult::success(content))
1385
}
1386
1387
#[::rmcp::tool(
@@ -2035,6 +2192,142 @@ macro_rules! xcrs_mcp_tools {
2192
)),
2193
]))
2194
}
2195
+
2196
+ #[::rmcp::tool(
2197
+ name = $posture_get_name,
2198
+ title = "Get fold posture",
2199
+ annotations(title = "Get fold posture", read_only_hint = true, idempotent_hint = true),
2200
+ description = "Purpose: report whether a foldable simulator (such as iPhone Duo) is folded or unfolded, and which display is live. When to use vs siblings: use this before screen_capture or ui_element_list on a foldable to know which display shows content; use posture_set to change it. Behavior: calls the runner's ControlKit `device.hinge.get` and returns its status (`closed`, `partiallyOpen`, `fullyOpen`, or `unsupported` on a device without a hinge) and angle in radians (0 closed, pi fully open), plus `live_display`: the outer display when closed, the inner display otherwise. Prerequisites: a reachable ControlKit runner built with hinge support, running on iOS 27.1 or later. Failure modes: errors if no target can be resolved, if the target is Android, or if the runner is unreachable or outdated."
2201
+ )]
2202
+ async fn posture_get(
2203
+ &self,
2204
+ ::rmcp::handler::server::wrapper::Parameters(
2205
+ args,
2206
+ ): ::rmcp::handler::server::wrapper::Parameters<
2207
+ $crate::mcp::PostureGetArgs,
2208
+ >,
2209
+ ) -> ::std::result::Result<
2210
+ ::rmcp::model::CallToolResult,
2211
+ ::rmcp::model::ErrorData,
2212
+ > {
2213
+ let target = Self::dispatch_apple_target(
2214
+ $posture_get_name,
2215
+ args.simulator_name,
2216
+ args.simulator_udid,
2217
+ args.host,
2218
+ args.controlkit_port,
2219
+ )?;
2220
+ let (simulator, controlkit) = Self::controlkit_for_target(&target)?;
2221
+ let hinge = controlkit
2222
+ .call("device.hinge.get", ::serde_json::json!({}))
2223
+ .await
2224
+ .map_err(|error| {
2225
+ ::rmcp::model::ErrorData::internal_error(error.to_string(), None)
2226
+ })?;
2227
+ let status = hinge
2228
+ .get("status")
2229
+ .and_then(::serde_json::Value::as_str)
2230
+ .unwrap_or("unsupported");
2231
+ let live_display = simulator.as_ref().and_then(|simulator| {
2232
+ let displays = $crate::XcodeCommandLineTools::new()
2233
+ .simctl()
2234
+ .displays(&simulator.udid)
2235
+ .ok()?;
2236
+ $crate::mcp::live_display(status, &displays).cloned()
2237
+ });
2238
+ Ok(::rmcp::model::CallToolResult::success(vec![
2239
+ ::rmcp::model::ContentBlock::json(&::serde_json::json!({
2240
+ "simulator": simulator,
2241
+ "hinge": hinge,
2242
+ "live_display": live_display,
2243
+ }))?,
2244
+ ]))
2245
+ }
2246
+
2247
+ #[::rmcp::tool(
2248
+ name = $posture_set_name,
2249
+ title = "Set fold posture",
2250
+ annotations(title = "Set fold posture", read_only_hint = false, destructive_hint = false, idempotent_hint = true),
2251
+ description = "Purpose: fold or unfold a foldable simulator (such as iPhone Duo). When to use vs siblings: use this to switch which display is live; use posture_get to read the current state. Behavior: reads the hinge through ControlKit, and if it is not already in the requested posture presses the matching hinge button (Closed or Open) in the simulator's Device Hub window through macOS accessibility, then polls `device.hinge.get` for up to 5 seconds until the hinge reports the new state. Prerequisites: Device Hub running with the simulator's window open (simctl and the Simulator app have no fold control), the Accessibility permission granted to the app running xcrs, and a ControlKit runner with hinge support. The window is found by the simulator's name, so keep a single Device Hub window per name. Failure modes: errors if the target is not an Apple simulator, if the device reports no hinge, if Device Hub or its window is missing, if Accessibility access is denied, or if the hinge does not change within the timeout."
2252
+ )]
2253
+ async fn posture_set(
2254
+ &self,
2255
+ ::rmcp::handler::server::wrapper::Parameters(
2256
+ args,
2257
+ ): ::rmcp::handler::server::wrapper::Parameters<
2258
+ $crate::mcp::PostureSetArgs,
2259
+ >,
2260
+ ) -> ::std::result::Result<
2261
+ ::rmcp::model::CallToolResult,
2262
+ ::rmcp::model::ErrorData,
2263
+ > {
2264
+ let target = Self::dispatch_apple_target(
2265
+ $posture_set_name,
2266
+ args.simulator_name,
2267
+ args.simulator_udid,
2268
+ None,
2269
+ args.controlkit_port,
2270
+ )?;
2271
+ let (Some(simulator), controlkit) = Self::controlkit_for_target(&target)? else {
2272
+ return Err(::rmcp::model::ErrorData::invalid_request(
2273
+ concat!($posture_set_name, " only works on Apple simulators."),
2274
+ None,
2275
+ ));
2276
+ };
2277
+ let read_status = || async {
2278
+ let hinge = controlkit
2279
+ .call("device.hinge.get", ::serde_json::json!({}))
2280
+ .await
2281
+ .map_err(|error| {
2282
+ ::rmcp::model::ErrorData::internal_error(error.to_string(), None)
2283
+ })?;
2284
+ Ok::<_, ::rmcp::model::ErrorData>(
2285
+ hinge
2286
+ .get("status")
2287
+ .and_then(::serde_json::Value::as_str)
2288
+ .unwrap_or("unsupported")
2289
+ .to_string(),
2290
+ )
2291
+ };
2292
+
2293
+ let current = read_status().await?;
2294
+ if current == "unsupported" {
2295
+ return Err(::rmcp::model::ErrorData::invalid_request(
2296
+ format!("{} has no hinge, so it cannot be folded.", simulator.name),
2297
+ None,
2298
+ ));
2299
+ }
2300
+ if !args.posture.matches_hinge_status(¤t) {
2301
+ $crate::HostInspect::new()
2302
+ .set_posture(&simulator.name, args.posture.hinge_button())
2303
+ .map_err(|error| {
2304
+ ::rmcp::model::ErrorData::internal_error(error.to_string(), None)
2305
+ })?;
2306
+ let deadline = ::std::time::Instant::now() + ::std::time::Duration::from_secs(5);
2307
+ loop {
2308
+ ::tokio::time::sleep(::std::time::Duration::from_millis(250)).await;
2309
+ if args.posture.matches_hinge_status(&read_status().await?) {
2310
+ break;
2311
+ }
2312
+ if ::std::time::Instant::now() >= deadline {
2313
+ return Err(::rmcp::model::ErrorData::internal_error(
2314
+ format!(
2315
+ "Device Hub accepted the {} request but the hinge did not change within 5 seconds.",
2316
+ args.posture.as_str()
2317
+ ),
2318
+ None,
2319
+ ));
2320
+ }
2321
+ }
2322
+ }
2323
+ Ok(::rmcp::model::CallToolResult::success(vec![
2324
+ ::rmcp::model::ContentBlock::text(format!(
2325
+ "Set {} to {}.",
2326
+ simulator.name,
2327
+ args.posture.as_str()
2328
+ )),
2329
+ ]))
2330
+ }
2331
}
2332
};
2333
}
@@ -2059,7 +2352,9 @@ xcrs_mcp_tools!(
2352
"input_click",
2353
"input_spatial_tap",
2354
"orientation_get",
2062
- "orientation_set"
2355
+ "orientation_set",
2356
+ "posture_get",
2357
+ "posture_set"
2358
);
2359
2360
#[rmcp::tool_handler(router = Self::xcrs_tool_router())]
@@ -2098,6 +2393,73 @@ pub async fn serve() -> Result<()> {
2393
mod tests {
2394
use super::*;
2395
2396
+ fn duo_displays() -> Vec<crate::SimDisplay> {
2397
+ let display = |screen_id, role: &str, width, height| crate::SimDisplay {
2398
+ screen_id,
2399
+ name: String::new(),
2400
+ role: role.to_string(),
2401
+ width,
2402
+ height,
2403
+ };
2404
+ vec![
2405
+ display(1, "outer", 1398, 2034),
2406
+ display(3, "inner", 2007, 2853),
2407
+ ]
2408
+ }
2409
+
2410
+ #[test]
2411
+ fn live_display_follows_the_hinge_status() {
2412
+ let displays = duo_displays();
2413
+
2414
+ assert_eq!(
2415
+ live_display("closed", &displays).map(|d| d.screen_id),
2416
+ Some(1)
2417
+ );
2418
+ assert_eq!(
2419
+ live_display("fullyOpen", &displays).map(|d| d.screen_id),
2420
+ Some(3)
2421
+ );
2422
+ assert_eq!(
2423
+ live_display("partiallyOpen", &displays).map(|d| d.screen_id),
2424
+ Some(3)
2425
+ );
2426
+ assert!(live_display("unsupported", &displays).is_none());
2427
+ assert!(live_display("closed", &displays[1..]).is_none());
2428
+ }
2429
+
2430
+ #[test]
2431
+ fn postures_map_to_hinge_buttons_and_statuses() {
2432
+ assert_eq!(Posture::Folded.hinge_button(), "closed");
2433
+ assert_eq!(Posture::Unfolded.hinge_button(), "open");
2434
+ assert!(Posture::Folded.matches_hinge_status("closed"));
2435
+ assert!(!Posture::Folded.matches_hinge_status("fullyOpen"));
2436
+ assert!(Posture::Unfolded.matches_hinge_status("fullyOpen"));
2437
+ assert!(!Posture::Unfolded.matches_hinge_status("partiallyOpen"));
2438
+ }
2439
+
2440
+ #[test]
2441
+ fn posture_set_args_accept_folded_and_unfolded_only() {
2442
+ let args: PostureSetArgs =
2443
+ serde_json::from_value(json!({ "posture": "folded" })).expect("folded is valid");
2444
+ assert_eq!(args.posture, Posture::Folded);
2445
+ assert!(serde_json::from_value::<PostureSetArgs>(json!({ "posture": "bent" })).is_err());
2446
+ assert!(serde_json::from_value::<PostureSetArgs>(json!({})).is_err());
2447
+ }
2448
+
2449
+ #[test]
2450
+ fn screen_capture_advertises_the_display_argument() {
2451
+ let tools = XcrsMcpServer::xcrs_tool_router().list_all();
2452
+ let tool = tools
2453
+ .iter()
2454
+ .find(|tool| tool.name == "screen_capture")
2455
+ .expect("screen_capture is exposed");
2456
+ assert!(tool
2457
+ .input_schema
2458
+ .get("properties")
2459
+ .and_then(|p| p.get("display"))
2460
+ .is_some());
2461
+ }
2462
+
2463
#[test]
2464
fn legacy_discovery_probe_receives_method_not_found() {
2465
let response = legacy_discovery_response(
@@ -2120,9 +2482,9 @@ mod tests {
2482
assert!(response.is_none());
2483
}
2484
2123
- /// The 19 canonical tool names shared by the standalone and embedded
2485
+ /// The 21 canonical tool names shared by the standalone and embedded
2486
/// automation servers.
2125
- const CANONICAL_TOOL_NAMES: [&str; 19] = [
2487
+ const CANONICAL_TOOL_NAMES: [&str; 21] = [
2488
"device_list",
2489
"device_select",
2490
"device_capabilities",
@@ -2142,6 +2504,8 @@ mod tests {
2504
"input_spatial_tap",
2505
"orientation_get",
2506
"orientation_set",
2507
+ "posture_get",
2508
+ "posture_set",
2509
];
2510
2511
/// Fully-qualified tool names this crate used before the canonical-tool
@@ -2180,7 +2544,7 @@ mod tests {
2544
/// types it does not own, so they intentionally advertise no
2545
/// `output_schema`. Only `device_select` returns structured content built
2546
/// entirely from types this module owns, so it is the only tool with one.
2183
- const TOOLS_WITHOUT_OUTPUT_SCHEMA: [&str; 18] = [
2547
+ const TOOLS_WITHOUT_OUTPUT_SCHEMA: [&str; 20] = [
2548
"device_list",
2549
"device_capabilities",
2550
"app_install_launch",
@@ -2199,10 +2563,12 @@ mod tests {
2563
"input_spatial_tap",
2564
"orientation_get",
2565
"orientation_set",
2566
+ "posture_get",
2567
+ "posture_set",
2568
];
2569
2570
#[test]
2205
- fn tool_router_exposes_exactly_the_nineteen_canonical_tools() {
2571
+ fn tool_router_exposes_exactly_the_twentyone_canonical_tools() {
2572
let tools = XcrsMcpServer::xcrs_tool_router().list_all();
2573
2574
assert_eq!(
crates/xcrs/src/xcrs.rs
+388
-3
@@ -467,6 +467,182 @@ struct SimctlDevice {
467
state: String,
468
#[serde(rename = "isAvailable")]
469
is_available: Option<bool>,
470
+ #[serde(rename = "deviceTypeIdentifier")]
471
+ device_type_identifier: Option<String>,
472
+}
473
+
474
+#[derive(Debug, Deserialize)]
475
+struct SimctlDeviceTypeList {
476
+ devicetypes: Vec<SimctlDeviceType>,
477
+}
478
+
479
+#[derive(Debug, Deserialize)]
480
+struct SimctlDeviceType {
481
+ identifier: String,
482
+ #[serde(rename = "bundlePath")]
483
+ bundle_path: Option<String>,
484
+}
485
+
486
+/// An integrated display of a simulated device. Foldable devices such as the
487
+/// iPhone Duo have more than one.
488
+#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
489
+pub struct SimDisplay {
490
+ /// The ID `simctl io --display` accepts.
491
+ pub screen_id: u32,
492
+ /// The display name from the device type, for example `LCD`.
493
+ pub name: String,
494
+ /// `main` for a single-display device; `outer` and `inner` for a device with
495
+ /// two; `display` otherwise.
496
+ pub role: String,
497
+ pub width: u32,
498
+ pub height: u32,
499
+}
500
+
501
+impl SimDisplay {
502
+ /// `screen 1 outer 1398x2034`
503
+ pub fn describe(&self) -> String {
504
+ format!(
505
+ "screen {} {} {}x{}",
506
+ self.screen_id, self.role, self.width, self.height
507
+ )
508
+ }
509
+}
510
+
511
+/// Reads the integrated displays from a device type's `capabilities.plist`,
512
+/// already converted to JSON. A device type without a `displays` array falls
513
+/// back to the main screen dimensions as display 1.
514
+pub fn parse_integrated_displays(capabilities_json: &str) -> Result<Vec<SimDisplay>> {
515
+ let root: serde_json::Value = serde_json::from_str(capabilities_json)
516
+ .context("failed to parse device type capabilities JSON")?;
517
+ let capabilities = root
518
+ .get("capabilities")
519
+ .ok_or_else(|| anyhow!("device type capabilities have no 'capabilities' dictionary"))?;
520
+
521
+ // (screen ID, name, is primary, width, height)
522
+ let mut integrated: Vec<(u32, String, bool, u32, u32)> = capabilities
523
+ .get("displays")
524
+ .and_then(serde_json::Value::as_array)
525
+ .into_iter()
526
+ .flatten()
527
+ .filter(|display| {
528
+ display
529
+ .get("displayType")
530
+ .and_then(serde_json::Value::as_str)
531
+ == Some("integrated")
532
+ })
533
+ .filter_map(|display| {
534
+ let number = |key: &str| {
535
+ display
536
+ .get(key)?
537
+ .as_u64()
538
+ .and_then(|n| u32::try_from(n).ok())
539
+ };
540
+ Some((
541
+ number("screenID")?,
542
+ display
543
+ .get("displayName")
544
+ .and_then(serde_json::Value::as_str)
545
+ .unwrap_or_default()
546
+ .to_string(),
547
+ display
548
+ .get("deviceName")
549
+ .and_then(serde_json::Value::as_str)
550
+ == Some("primary"),
551
+ number("width")?,
552
+ number("height")?,
553
+ ))
554
+ })
555
+ .collect();
556
+
557
+ if integrated.is_empty() {
558
+ let main = |key: &str| {
559
+ capabilities
560
+ .get("ScreenDimensionsCapability")
561
+ .unwrap_or(capabilities)
562
+ .get(key)?
563
+ .as_u64()
564
+ .and_then(|n| u32::try_from(n).ok())
565
+ };
566
+ let (width, height) = main("main-screen-width")
567
+ .zip(main("main-screen-height"))
568
+ .ok_or_else(|| anyhow!("device type capabilities list no integrated display"))?;
569
+ integrated.push((1, String::new(), true, width, height));
570
+ }
571
+
572
+ integrated.sort_by_key(|(screen_id, ..)| *screen_id);
573
+ let count = integrated.len();
574
+ Ok(integrated
575
+ .into_iter()
576
+ .map(|(screen_id, name, primary, width, height)| SimDisplay {
577
+ screen_id,
578
+ name,
579
+ role: match (count, primary) {
580
+ (1, _) => "main",
581
+ (2, true) => "outer",
582
+ (2, false) => "inner",
583
+ _ => "display",
584
+ }
585
+ .to_string(),
586
+ width,
587
+ height,
588
+ })
589
+ .collect())
590
+}
591
+
592
+/// Resolves the `display` argument of `screen_capture` against a device's
593
+/// integrated displays.
594
+///
595
+/// `None` selects every display of a foldable device and the only display of any
596
+/// other, so an unfolded device never yields a lone black frame. `outer`,
597
+/// `inner` and `all` are matched case-insensitively; a number matches a screen ID.
598
+pub fn select_displays<'a>(
599
+ displays: &'a [SimDisplay],
600
+ selection: Option<&str>,
601
+) -> Result<Vec<&'a SimDisplay>> {
602
+ let selection = selection.map(str::trim).filter(|value| !value.is_empty());
603
+ match selection.map(str::to_ascii_lowercase).as_deref() {
604
+ None | Some("all") => Ok(displays.iter().collect()),
605
+ Some(role @ ("outer" | "inner")) => {
606
+ let found: Vec<_> = displays
607
+ .iter()
608
+ .filter(|display| display.role == role)
609
+ .collect();
610
+ if found.is_empty() {
611
+ Err(anyhow!(
612
+ "this device has no {role} display; its displays are: {}",
613
+ list_displays(displays)
614
+ ))
615
+ } else {
616
+ Ok(found)
617
+ }
618
+ }
619
+ Some(other) => {
620
+ let screen_id: u32 = other.parse().map_err(|_| {
621
+ anyhow!(
622
+ "unknown display '{other}': use outer, inner, all or a screen ID ({})",
623
+ list_displays(displays)
624
+ )
625
+ })?;
626
+ displays
627
+ .iter()
628
+ .find(|display| display.screen_id == screen_id)
629
+ .map(|display| vec![display])
630
+ .ok_or_else(|| {
631
+ anyhow!(
632
+ "this device has no display with screen ID {screen_id}; its displays are: {}",
633
+ list_displays(displays)
634
+ )
635
+ })
636
+ }
637
+ }
638
+}
639
+
640
+fn list_displays(displays: &[SimDisplay]) -> String {
641
+ displays
642
+ .iter()
643
+ .map(SimDisplay::describe)
644
+ .collect::<Vec<_>>()
645
+ .join(", ")
646
}
647
648
pub struct Simctl<'a> {
@@ -548,20 +724,88 @@ impl Simctl<'_> {
724
Ok(())
725
}
726
551
- pub fn screenshot(&self, udid: &str) -> Result<Vec<u8>> {
727
+ /// Captures a PNG of one display. `display` is a screen ID as `simctl io
728
+ /// --display` accepts it; `None` captures the primary display.
729
+ pub fn screenshot(&self, udid: &str, display: Option<&str>) -> Result<Vec<u8>> {
730
let temporary_directory = tempfile::tempdir()?;
731
let screenshot_path = temporary_directory.path().join("screenshot.png");
732
let screenshot_path_string = screenshot_path
733
.to_str()
734
.ok_or_else(|| anyhow!("screenshot path is not valid UTF-8"))?;
735
558
- self.tools
559
- .run_xcrun(["simctl", "io", udid, "screenshot", screenshot_path_string])?;
736
+ let mut args = vec!["simctl", "io", udid, "screenshot"];
737
+ let display_argument = display.map(|display| format!("--display={display}"));
738
+ if let Some(display_argument) = &display_argument {
739
+ args.push(display_argument);
740
+ }
741
+ args.push(screenshot_path_string);
742
+ self.tools.run_xcrun(args)?;
743
744
std::fs::read(&screenshot_path)
745
.with_context(|| format!("failed to read {}", screenshot_path.display()))
746
}
747
748
+ /// The integrated displays of a simulator, resolved from its device type and
749
+ /// cached per device type.
750
+ pub fn displays(&self, udid: &str) -> Result<Vec<SimDisplay>> {
751
+ static CACHE: std::sync::OnceLock<std::sync::Mutex<BTreeMap<String, Vec<SimDisplay>>>> =
752
+ std::sync::OnceLock::new();
753
+ let cache = CACHE.get_or_init(Default::default);
754
+
755
+ let devices: SimctlDeviceList = serde_json::from_str(
756
+ &self
757
+ .tools
758
+ .run_xcrun(["simctl", "list", "devices", "--json"])?,
759
+ )
760
+ .context("failed to parse simctl device list JSON")?;
761
+ let device_type = devices
762
+ .devices
763
+ .into_values()
764
+ .flatten()
765
+ .find(|device| device.udid == udid)
766
+ .ok_or_else(|| anyhow!("Apple simulator with UDID '{udid}' was not found"))?
767
+ .device_type_identifier
768
+ .ok_or_else(|| anyhow!("simctl reported no device type for simulator '{udid}'"))?;
769
+
770
+ if let Some(cached) = cache
771
+ .lock()
772
+ .ok()
773
+ .and_then(|cache| cache.get(&device_type).cloned())
774
+ {
775
+ return Ok(cached);
776
+ }
777
+
778
+ let device_types: SimctlDeviceTypeList = serde_json::from_str(&self.tools.run_xcrun([
779
+ "simctl",
780
+ "list",
781
+ "devicetypes",
782
+ "--json",
783
+ ])?)
784
+ .context("failed to parse simctl device type list JSON")?;
785
+ let bundle_path = device_types
786
+ .devicetypes
787
+ .into_iter()
788
+ .find(|candidate| candidate.identifier == device_type)
789
+ .and_then(|candidate| candidate.bundle_path)
790
+ .ok_or_else(|| anyhow!("simctl reported no bundle for device type '{device_type}'"))?;
791
+ let plist = Path::new(&bundle_path).join("Contents/Resources/capabilities.plist");
792
+ let json = run_command(
793
+ Path::new("/usr/bin/plutil"),
794
+ [
795
+ OsStr::new("-convert"),
796
+ OsStr::new("json"),
797
+ OsStr::new("-o"),
798
+ OsStr::new("-"),
799
+ plist.as_os_str(),
800
+ ],
801
+ )?;
802
+ let displays = parse_integrated_displays(&json)?;
803
+ if let Ok(mut cache) = cache.lock() {
804
+ cache.insert(device_type, displays.clone());
805
+ }
806
+ Ok(displays)
807
+ }
808
+
809
pub fn app_container_path(&self, udid: &str, bundle_id: &str) -> Result<PathBuf> {
810
let output =
811
self.tools
@@ -1602,6 +1846,26 @@ impl HostInspect {
1846
}
1847
}
1848
1849
+ /// Set a foldable simulator's hinge by pressing the matching button in its
1850
+ /// Device Hub window. `device` is the simulator's name, which is how Device Hub
1851
+ /// titles the window; `hinge` is `closed`, `partiallyOpen` or `open`. Device
1852
+ /// Hub must be running with that window open, and the process running xcrs
1853
+ /// needs the Accessibility permission.
1854
+ pub fn set_posture(&self, device: &str, hinge: &str) -> Result<()> {
1855
+ let invocation = self.invoke(&[
1856
+ "devicehub-posture".to_string(),
1857
+ "--device".to_string(),
1858
+ device.to_string(),
1859
+ "--posture".to_string(),
1860
+ hinge.to_string(),
1861
+ ])?;
1862
+ if invocation.succeeded() {
1863
+ Ok(())
1864
+ } else {
1865
+ Err(anyhow!("{}", invocation.message()))
1866
+ }
1867
+ }
1868
+
1869
/// List the host's on-screen windows, ordered by layer and then id.
1870
pub fn windows(&self, owner: Option<&str>) -> Result<HostWindowList> {
1871
let mut arguments = vec!["windows".to_string()];
@@ -2937,4 +3201,125 @@ SERIAL456 no permissions (user denied) usb:0-2 transport_id:3
3201
.expect_err("a non-finite position should be rejected");
3202
assert!(error.to_string().contains("must be a finite number"));
3203
}
3204
+
3205
+ /// The `capabilities.displays` array of the iPhone Duo device type, trimmed to
3206
+ /// the fields xcrs reads.
3207
+ const DUO_CAPABILITIES: &str = r#"{
3208
+ "capabilities": {
3209
+ "displays": [
3210
+ {"screenID": 1, "deviceName": "primary", "displayName": "LCD",
3211
+ "displayType": "integrated", "width": 1398, "height": 2034},
3212
+ {"screenID": 3, "deviceName": "primary-1", "displayName": "LCD-1",
3213
+ "displayType": "integrated", "width": 2007, "height": 2853},
3214
+ {"screenID": 2, "deviceName": "external-0", "displayName": "TV",
3215
+ "displayType": "tvOut", "width": 1920, "height": 1080},
3216
+ {"screenID": 4, "deviceName": "wireless0", "displayName": "CarPlay",
3217
+ "displayType": "carPlay", "width": 800, "height": 480}
3218
+ ]
3219
+ }
3220
+ }"#;
3221
+
3222
+ #[test]
3223
+ fn duo_capabilities_yield_two_integrated_displays() {
3224
+ let displays = parse_integrated_displays(DUO_CAPABILITIES).expect("displays should parse");
3225
+
3226
+ assert_eq!(
3227
+ displays,
3228
+ vec![
3229
+ SimDisplay {
3230
+ screen_id: 1,
3231
+ name: "LCD".to_string(),
3232
+ role: "outer".to_string(),
3233
+ width: 1398,
3234
+ height: 2034,
3235
+ },
3236
+ SimDisplay {
3237
+ screen_id: 3,
3238
+ name: "LCD-1".to_string(),
3239
+ role: "inner".to_string(),
3240
+ width: 2007,
3241
+ height: 2853,
3242
+ },
3243
+ ]
3244
+ );
3245
+ assert_eq!(displays[0].describe(), "screen 1 outer 1398x2034");
3246
+ assert_eq!(displays[1].describe(), "screen 3 inner 2007x2853");
3247
+ }
3248
+
3249
+ #[test]
3250
+ fn single_display_device_is_main_and_falls_back_to_screen_dimensions() {
3251
+ let with_display = r#"{"capabilities": {"displays": [
3252
+ {"screenID": 1, "deviceName": "primary", "displayName": "LCD",
3253
+ "displayType": "integrated", "width": 1206, "height": 2622}]}}"#;
3254
+ let displays = parse_integrated_displays(with_display).expect("displays should parse");
3255
+ assert_eq!(displays.len(), 1);
3256
+ assert_eq!(displays[0].role, "main");
3257
+
3258
+ let without_displays = r#"{"capabilities": {"ScreenDimensionsCapability": {
3259
+ "main-screen-width": 750, "main-screen-height": 1334}}}"#;
3260
+ let displays = parse_integrated_displays(without_displays).expect("fallback should parse");
3261
+ assert_eq!(displays.len(), 1);
3262
+ assert_eq!(
3263
+ (displays[0].screen_id, displays[0].width, displays[0].height),
3264
+ (1, 750, 1334)
3265
+ );
3266
+ }
3267
+
3268
+ #[test]
3269
+ fn capabilities_without_any_display_are_an_error() {
3270
+ assert!(parse_integrated_displays(r#"{"capabilities": {}}"#).is_err());
3271
+ assert!(parse_integrated_displays("not json").is_err());
3272
+ }
3273
+
3274
+ fn duo_displays() -> Vec<SimDisplay> {
3275
+ parse_integrated_displays(DUO_CAPABILITIES).expect("displays should parse")
3276
+ }
3277
+
3278
+ fn selected_ids(displays: &[SimDisplay], selection: Option<&str>) -> Vec<u32> {
3279
+ select_displays(displays, selection)
3280
+ .expect("selection should resolve")
3281
+ .into_iter()
3282
+ .map(|display| display.screen_id)
3283
+ .collect()
3284
+ }
3285
+
3286
+ #[test]
3287
+ fn display_selection_resolves_roles_ids_and_all() {
3288
+ let displays = duo_displays();
3289
+
3290
+ assert_eq!(selected_ids(&displays, Some("outer")), vec![1]);
3291
+ assert_eq!(selected_ids(&displays, Some("Inner")), vec![3]);
3292
+ assert_eq!(selected_ids(&displays, Some("all")), vec![1, 3]);
3293
+ assert_eq!(selected_ids(&displays, Some("3")), vec![3]);
3294
+ }
3295
+
3296
+ #[test]
3297
+ fn display_selection_defaults_to_all_only_with_more_than_one_display() {
3298
+ let duo = duo_displays();
3299
+ assert_eq!(selected_ids(&duo, None), vec![1, 3]);
3300
+ assert_eq!(selected_ids(&duo, Some(" ")), vec![1, 3]);
3301
+
3302
+ let single = parse_integrated_displays(
3303
+ r#"{"capabilities": {"displays": [
3304
+ {"screenID": 1, "deviceName": "primary", "displayName": "LCD",
3305
+ "displayType": "integrated", "width": 100, "height": 200}]}}"#,
3306
+ )
3307
+ .expect("displays should parse");
3308
+ assert_eq!(selected_ids(&single, None), vec![1]);
3309
+ assert!(select_displays(&single, Some("inner")).is_err());
3310
+ }
3311
+
3312
+ #[test]
3313
+ fn display_selection_rejects_unknown_values() {
3314
+ let displays = duo_displays();
3315
+
3316
+ let invalid = select_displays(&displays, Some("sideways")).expect_err("invalid name");
3317
+ assert!(invalid
3318
+ .to_string()
3319
+ .contains("outer, inner, all or a screen ID"));
3320
+ assert!(invalid.to_string().contains("screen 3 inner 2007x2853"));
3321
+
3322
+ let missing = select_displays(&displays, Some("7")).expect_err("no such screen");
3323
+ assert!(missing.to_string().contains("no display with screen ID 7"));
3324
+ }
3325
}
docs/controlkit.md
+58
-1
@@ -116,11 +116,22 @@ expose the same unprefixed tool names.
116
| `input_swipe` | iOS / tvOS, simulator or physical | Swipe between coordinates. |
117
| `input_button` | iOS / tvOS, simulator or physical | Press a Home or tvOS remote button. |
118
| `app_launch`/`app_terminate` | simulator or physical | Launch/terminate an app by bundle ID. On a physical device this calls the runner's `device.apps.launch`/`device.apps.terminate` RPC methods instead of `simctl`. |
119
-| `screen_capture` | Simulator or physical | Capture a PNG screenshot. Uses `simctl` for a simulator, or `devicectl device capture screenshot` when a `device` identifier is given. |
119
+| `screen_capture` | Simulator or physical | Capture a PNG screenshot. Uses `simctl` for a simulator, or `devicectl device capture screenshot` when a `device` identifier is given. `display` (`outer`, `inner`, `all` or a screen ID) picks a display of a foldable simulator; see [iPhone Duo](#iphone-duo-foldable-simulators). |
120
+| `posture_get` | iOS 27.1+ simulator | Read the hinge status and angle, and which display is live. |
121
+| `posture_set` | iOS 27.1+ simulator | Fold (`folded`) or unfold (`unfolded`) a foldable simulator. Needs Device Hub and the Accessibility permission. |
122
| `ui_describe` | All UI-test runners | Read the accessibility hierarchy for the supplied `bundle_id`. |
123
| `ui_element_list` | All UI-test runners | Read actionable elements with tap coordinates for the supplied `bundle_id`. |
124
| `ui_tap` | All UI-test runners | Resolve and activate an accessibility element by label, identifier, or value. |
125
126
+The runner implements these JSON-RPC methods: `device.capabilities`,
127
+`device.info`, `device.apps.launch`, `device.apps.terminate`, `device.io.tap`,
128
+`device.io.click`, `device.io.spatial.tap`, `device.io.swipe` (x1, y1, x2, y2,
129
+optional durationMs; iOS), `device.io.orientation.get`/`set` (`PORTRAIT` or
130
+`LANDSCAPE`; iOS), `device.io.text`, `device.io.button`, `device.dump.ui`,
131
+`device.ui.tap` and `device.hinge.get` (iOS 27.1 and later; the runner lists
132
+`hinge` in its capabilities only on a device that reports one). Calls that name
133
+an app that is not running fail with an error instead of stopping the runner.
134
+
135
UI introspection requires a recent ControlKit runner that implements
136
`device.dump.ui`. The `ui_tap` action requires a runner that implements
137
`device.ui.tap`; it resolves and activates the requested element inside the
@@ -131,6 +142,52 @@ navigate focus before calling `ui_tap`. All three UI tools require the target
142
app's `bundle_id`; this lets XCTest attach to an already-running app instead of
143
accidentally reading the ControlKit host app.
144
145
+### iPhone Duo (foldable simulators)
146
+
147
+The iPhone Duo simulator (Xcode 27.1, iOS 27.1) has two integrated displays,
148
+read from the device type's `capabilities.plist`:
149
+
150
+| Screen ID | Role | Size |
151
+| --- | --- | --- |
152
+| 1 | outer | 1398x2034 |
153
+| 3 | inner | 2007x2853 |
154
+
155
+Only the display in use shows content. The other one is solid black, so a
156
+plain capture of display 1 on an unfolded device returns a black frame.
157
+
158
+- `device_capabilities` adds `displays` and `foldable` (true when there is
159
+ more than one integrated display) for simulators.
160
+- `screen_capture` without `display` captures every display of a foldable
161
+ device: a text block (`screen 1 outer 1398x2034`) then an image for each.
162
+ Single-display devices return one image as before. `display` accepts `outer`,
163
+ `inner`, `all` or a screen ID.
164
+- `posture_get` calls the runner's `device.hinge.get` and reports `closed`,
165
+ `partiallyOpen` or `fullyOpen` with the angle in radians (0 closed, pi fully
166
+ open). `live_display` is the outer display when closed and the inner display
167
+ otherwise. A device without a hinge reports `unsupported`.
168
+- `posture_set` has no `simctl` or CoreSimulator equivalent. It presses the
169
+ `Closed` or `Open` button in the simulator's Device Hub window through macOS
170
+ accessibility (`xcrs-host-inspect devicehub-posture`), then polls the hinge
171
+ for up to 5 seconds. It does nothing when the hinge is already in the
172
+ requested state.
173
+
174
+Limits of `posture_set`:
175
+
176
+- Device Hub must be running with the simulator's window open. The window is
177
+ matched by the simulator's name (its title is `<name> – <runtime>`), so two
178
+ windows with the same name are an error.
179
+- The app that runs xcrs needs the Accessibility permission (System Settings,
180
+ Privacy & Security, Accessibility). Without it the call fails with a message
181
+ saying so.
182
+- The window does not need to be in front.
183
+- Device Hub has a `Partially Open` button too, but its angle is not
184
+ adjustable, so `posture_set` offers only `folded` and `unfolded`.
185
+- The helper binary is found as for the other host-inspect commands
186
+ (`XCRS_HOST_INSPECT`, next to `xcrs`, or `.build/` in `xcrs-controlkit`).
187
+
188
+Rotating the inner display and inspecting its UI use the same runner methods as
189
+any other device. Call `posture_get` first to know which display is live.
190
+
191
For a local macOS runner, omit simulator fields and pass `host` and
192
`controlkit_port` when needed:
193