iOS: final runner probe can replace an xcodebuild launch failure with a generic connect error

[emmanuelchucks]

Summary

On a physical iOS device, waitForRunner can return Runner did not accept connection when xcodebuild exits while the final runner endpoint probe is in flight. The actionable XCTest launch failure remains in runner.log, but it is absent from the returned error.

I observed this with an untrusted Developer App certificate, then reproduced the timing deterministically against current main without a device. In that window, the generic connect error also allows the prepare health path to start another runner even though the launch blocker is deterministic.

Physical-device observation

Environment:

• agent-device 0.20.6
• macOS 26.5.2 (25F84)
• Xcode 26.5 (17F42)
• Node.js 24.16.0
• physical iPhone on iOS 18.7.9

The relevant agent-device invocation was:

agent-device prepare ios-runner \
  --platform ios \
  --udid <physical-device-udid> \
  --timeout 240000

runner.log recorded the terminal cause:

Finished with error: The application could not be launched because the Developer App Certificate is not trusted.
...
Testing failed:
    The application could not be launched because the Developer App Certificate is not trusted.

The request diagnostic instead ended with:

Runner did not accept connection

The prepare health path then started a second runner, which failed with the same certificate-trust error before the request again returned the generic connect error.

Trusting the certificate resolved the physical-device blocker. The concern here is not that agent-device should bypass that requirement; it is that the existing actionable Xcode diagnostic was replaced at the command boundary.

Deterministic reproduction on current main

I added a focused regression test to runner-transport.test.ts at current main commit 5b1b1aad2eccd299d266315b438893b5e6ae5fb0.

The test starts with a live-looking runner child, makes the final usbmux probe set child.exitCode = 65 and fail with ECONNREFUSED, and gives testPromise the captured certificate-trust diagnostic. It expects the existing early-exit error and nested Xcode output:

Regression test

test('waitForRunner preserves xcodebuild diagnostics when the runner exits during the final probe', async () => {
  const session: RunnerSession = {
    sessionId: 'starting-device-session',
    device: xctestIosDevice,
    deviceId: xctestIosDevice.id,
    port: 8100,
    xctestrunPath: '/tmp/runner.xctestrun',
    jsonPath: '/tmp/runner.json',
    testPromise: Promise.resolve({
      exitCode: 65,
      stdout: '',
      stderr:
        'The application could not be launched because the Developer App Certificate is not trusted.',
    }),
    child: { pid: 1234, exitCode: null } as ExecBackgroundResult['child'],
    ready: false,
  };
  mockUsbmuxPostCommand.mockImplementation(async () => {
    (session.child as { exitCode: number | null }).exitCode = 65;
    throw new Error('ECONNREFUSED');
  });

  await assert.rejects(
    () =>
      waitForRunner(
        xctestIosDevice,
        8100,
        { command: 'uptime' },
        '/tmp/runner.log',
        100,
        session,
      ),
    (error: unknown) => {
      const appError = error as AppError;
      assert.equal(appError.message, 'Runner did not accept connection (xcodebuild exited early)');
      assert.match(
        String((appError.details?.xcodebuild as { stderr?: string } | undefined)?.stderr),
        /Developer App Certificate is not trusted/,
      );
      return true;
    },
  );
});

Command:

pnpm exec vitest run --project unit-core \
  src/platforms/apple/core/__tests__/runner-transport.test.ts

Result on unmodified current main:

AssertionError: Expected values to be strictly equal:
+ actual - expected

+ 'Runner did not accept connection'
- 'Runner did not accept connection (xcodebuild exited early)'

Test Files  1 failed (1)
Tests       1 failed | 14 passed (15)

As a mechanism check only, rechecking the runner child after retry completion made this focused suite pass 15/15. I reverted that probe and have not opened a PR; the appropriate precedence and final implementation are for maintainer triage.

Current control flow

ensureRunnerAttemptCanStart checks session.child.exitCode before each route probe. If the child exits during the final probe:

1. the route records its endpoint error and returns no response;
2. the retry loop exhausts;
3. waitForRunner leaves the loop;
4. the physical-device path calls buildRunnerConnectError without checking the child again.

That produces the deterministic test result above. The runner error table can classify an early-exit error once it receives one, but this timing window does not build that error.

Expected behavior

If the XCTest process has exited before runner connection returns, preserve the existing Runner did not accept connection (xcodebuild exited early) error and its nested xcodebuild output, reason, and hint. A known terminal launch failure should not be replaced by an endpoint error or trigger another runner-health attempt.

Related work

• fix(ios): diagnose a runner that cannot install, instead of blaming the screen #1529 added classification for actionable physical-runner installation failures, but this timing window prevents the Xcode output from reaching the early-exit classification path.
• refactor(ios): runner error classification as data; recovery tested through the AppleRunnerProvider seam #1631 / refactor(ios): runner error classification as data; recovery tested at the transport seam #1644 centralized runner error classification and recovery coverage; current main still reproduces this post-probe exit window.
• I found no open issue or PR matching xcodebuild exited early, Runner did not accept connection, or Developer App Certificate.

If this behavior and scope look right, I would be happy to prepare the focused regression and fix after maintainer triage.

[thymikee]

Confirmed against current main. waitForRunner checks session.child.exitCode before each attempt, but after the retry loop it sends the physical-device path directly to buildRunnerConnectError without rechecking the runner process. That makes the supplied final-probe race real and bypasses the existing buildRunnerEarlyExitError diagnostics/classification. This is sufficiently specified for agent pickup: preserve cancellation and usbmux-unattached precedence, recheck the child before the terminal generic connect error, surface the existing early-exit error with nested xcodebuild output, and add the supplied non-vacuous two-event regression (probe fails while marking exitCode 65). The physical observation already establishes practical relevance; no new device behavior is being invented.

[emmanuelchucks]

Thanks for confirming. I’m happy to contribute the focused fix and regression. Would you prefer that I open a PR, or is this already queued for an internal agent?

[thymikee]

Please go ahead and open the focused PR; there is no internal fixer queued for this issue. Keep the change to the terminal child-exit recheck plus the planted race regression, preserving cancellation/usbmux error precedence and the existing early-exit error shape. Link the PR to #1775 and include the red-before result from current main in its validation.