Skip to content

test(expo): post the evidence that shows the change, before and after - #10219

Closed
mikepitre wants to merge 3 commits into
mike/expo-verify-attachfrom
mike/expo-verify-evidence
Closed

mikepitre wants to merge 3 commits into
mike/expo-verify-attachfrom
mike/expo-verify-evidence

Conversation

@mikepitre

@mikepitre mikepitre commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Description

This is the same change as clerk/clerk-ios#688, which is merged, for the Expo package in integration/expo-native/. That description has the reasons, and it says what each new check in the publish script is for. clerk/clerk-android#1088 is the same change for clerk-android. This PR sits on #10163 and is the sixth of the stack. Its third commit is the fix of clerk/clerk-ios#689.

Before this PR, attach could post only the video and the screenshots of one passing run. A video opened on the screen that the run before it left, and the app restarted before the test began. After this PR:

  • attach posts only what its flags name. Pass --screenshot <label> once for each screenshot, --video, or both. With neither flag it fails with a usage error that says how to choose.
  • --before <run-id> adds a run of the same spec on the build without the change. That run can have failed.
  • The block is one result line and a table with a Before and an After column. A screenshot is 280 pixels wide.
  • run closes the app and starts the recorder when the first test is about to launch the app. A run in which no test calls host.launch has no video.
  • attach refuses a closed or merged pull request. It prints already posted only when the description holds the exact block it would write.
  • A hand-off carries the files of the before run in version 2 of the manifest, and scripts/verify-attach.mjs gains the checks that feat(verify): post the evidence that shows the change, before and after clerk-ios#688 lists.

The twelve files that integration/expo-native/src/core/MANIFEST lists, nine test files, src/platform/ios/local.ts, and the publish script are byte copies of the files that clerk/clerk-ios#688 put on main there, at c88bdf0af. publish.ts and test/evidence.test.ts are the files of clerk/clerk-ios#689. scripts/verify-attach.test.mjs differs from the file there in one line, which imports describe and it from vitest. verify-attach.yml does not change.

Four things differ from clerk/clerk-ios#688:

  • This package drives two platforms. A before run must be a run of the same platform, and attach fails with a usage error when it is not. The description still holds one block for each platform.
  • On Android the test app is a dev client, and the tests start it with adb. There the CLI closes the app and starts screenrecord, and then the test stops the app again and starts its activity. A retried open on that path repeats only the last step, so it does not signal a second time.
  • The skill, features/README.md, and references/remote.md say the same things in the words of this repository.
  • The second commit adds two tests that clerk-ios does not have yet. The review of feat(verify): post the evidence that shows the change, before and after clerk-ios#688 found that no test pinned a retried app open. One test shows that a retried open signals a second time, after the new sign-in ticket of that try. The other shows that the CLI then closes nothing and starts no second recorder.

The changeset is empty, as it is in the other PRs of the stack. No published package changes.

What this can break:

  • attach <run-id> --pr <n> with no other flag now fails.
  • The script and the manifest change version together in this PR, from 1 to 2. Once the stack is on main, the workflow refuses a hand-off from a branch whose verify code is older, and only the run of verify-attach shows the refusal.
  • verify-e2e.yml records video. A recorder that cannot start now fails each test that launches the app. Before this PR it stopped the run before any test.

Look hardest at these:

  1. scripts/verify-attach.mjs, with 203 lines added and 67 removed. It is the only code here that runs with the bot token. It is the file of feat(verify): post the evidence that shows the change, before and after clerk-ios#688, so one read covers both.
  2. run in integration/expo-native/src/core/verbs.ts. The recorder now starts in the middle of a test.

npm test in integration/expo-native/ passes 787 of 787 at 9354f2b8e7, where #10163 has 752, and npm run typecheck is clean. vitest run --root scripts verify-attach passes 190 of 190, where #10163 has 143. Each of 46 deliberate breaks of the code fails at least one test. They are the 43 of the last round of clerk/clerk-ios#688, two for the new tests, and one for the fix.

verify-e2e passed at 9354f2b8e7, and no test needed its retry. On iOS all 17 tests passed. On Android 15 passed, and the other two are for iOS only. The iOS video opens on the simulator's home screen for about two and a half seconds, and the Android video opens on the launcher for about a second and a half. Then the app opens.

On a local Android emulator, after one up, custom-flow-sign-in/complete ran twice with video and passed both times. Those runs came before the third commit, which changes only how attach compares a block. Frames of the first three seconds of the second run, three a second, show the splash screen of the app, the dev client as it loads the bundle, and then the test. No frame shows the launcher, and the app starts once. The block at the end of this description is attach --video of that run, from the CLI of this branch. Its last seconds show the address, the test code, and the user and session IDs of a test user. down has deleted that user with its application.

Not proven:

  • The verify-attach workflow with this script. GitHub starts it only from main.
  • A hand-off with before files through a real tunnel. Only unit tests cover it.
  • The video change on a borrowed device. No remote session was started. There the close and the start of the recorder cross the tunnel in the middle of a test, and the test waits at most 60 seconds.
  • A before and after pair of videos, and screenshots with a video in one block, on a real pull request.

The example on clerk/clerk-android#1088 showed one defect in the shared code. A second attach of the same run with --before uploaded the files again and printed posted, where it should print already posted. The third commit takes the fix from clerk/clerk-ios#689, with its test. No attach with --before was made on this pull request.

Checklist

  • pnpm test runs as expected.
  • pnpm build runs as expected.
  • (If applicable) JSDoc comments have been added or updated for any package exports
  • (If applicable) Documentation has been updated

Type of change

  • 🐛 Bug fix
  • 🌟 New feature
  • 🔨 Breaking change
  • 📖 Refactoring / dependency upgrade / documentation
  • other: test tooling

Verified on an Android emulator. Run r20261013-190853-8110 on verify-android-1 (emulator-5560) at 3e6d7798a8b3, 1 of 1 passed.

video
evidence-ef0a6ed02466e06c-1.mp4

🤖 Generated with Claude Code

mikepitre and others added 2 commits October 13, 2026 19:00
This is the change of clerk/clerk-ios#688 for the Expo package. The
twelve files that src/core/MANIFEST lists, nine test files, the iOS
backend, and the publish script are byte copies of the files there.

attach posts what its flags name: --screenshot <label> for each
screenshot, --video, or both, and --before <run-id> adds the run on the
build without the change. The block is one result line and a table with
280 px images. run closes the app and starts the recorder at the first
host.launch, so a video no longer opens on the screen that the run
before it left. attach refuses a closed or merged pull request, and
prints "already posted" only when the description holds the block it
would write.

A hand-off carries the before files in version 2 of the manifest, and
scripts/verify-attach.mjs checks each new thing it accepts. The fix
lines for a missing iOS template simulator now give simctl create.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A retried open signals that the app is about to launch a second time,
after the new sign-in ticket of that try. The CLI answers the second
signal from the recorder it already started: it closes nothing and
starts no second recorder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 9354f2b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
clerk-js-sandbox Ready Ready Preview Oct 13, 2026 11:22pm UTC
swingset Ready Ready Preview Oct 13, 2026 11:22pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

attach read the upload addresses back from the block in table order and
laid the block out again as if they were in upload order. With a before
run a row shows the before cell first, so the two never matched, and an
identical attach uploaded every file again. The comparison no longer
depends on the order: the block there must be the block attach would
write, with any upload address in each place.

publish.ts and the test are byte copies of clerk/clerk-ios#689.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pkg-pr-new

pkg-pr-new Bot commented Oct 13, 2026

Copy link
Copy Markdown

Open in StackBlitz

@clerk/astro

npm i https://pkg.pr.new/@clerk/astro@10219

@clerk/backend

npm i https://pkg.pr.new/@clerk/backend@10219

@clerk/chrome-extension

npm i https://pkg.pr.new/@clerk/chrome-extension@10219

@clerk/clerk-js

npm i https://pkg.pr.new/@clerk/clerk-js@10219

@clerk/electron

npm i https://pkg.pr.new/@clerk/electron@10219

@clerk/electron-passkeys

npm i https://pkg.pr.new/@clerk/electron-passkeys@10219

@clerk/eslint-plugin

npm i https://pkg.pr.new/@clerk/eslint-plugin@10219

@clerk/expo

npm i https://pkg.pr.new/@clerk/expo@10219

@clerk/expo-biometrics

npm i https://pkg.pr.new/@clerk/expo-biometrics@10219

@clerk/expo-google-signin

npm i https://pkg.pr.new/@clerk/expo-google-signin@10219

@clerk/expo-native-components

npm i https://pkg.pr.new/@clerk/expo-native-components@10219

@clerk/expo-passkeys

npm i https://pkg.pr.new/@clerk/expo-passkeys@10219

@clerk/express

npm i https://pkg.pr.new/@clerk/express@10219

@clerk/fastify

npm i https://pkg.pr.new/@clerk/fastify@10219

@clerk/hono

npm i https://pkg.pr.new/@clerk/hono@10219

@clerk/localizations

npm i https://pkg.pr.new/@clerk/localizations@10219

@clerk/mosaic

npm i https://pkg.pr.new/@clerk/mosaic@10219

@clerk/nextjs

npm i https://pkg.pr.new/@clerk/nextjs@10219

@clerk/nuxt

npm i https://pkg.pr.new/@clerk/nuxt@10219

@clerk/react

npm i https://pkg.pr.new/@clerk/react@10219

@clerk/react-router

npm i https://pkg.pr.new/@clerk/react-router@10219

@clerk/shared

npm i https://pkg.pr.new/@clerk/shared@10219

@clerk/tanstack-react-start

npm i https://pkg.pr.new/@clerk/tanstack-react-start@10219

@clerk/testing

npm i https://pkg.pr.new/@clerk/testing@10219

@clerk/ui

npm i https://pkg.pr.new/@clerk/ui@10219

@clerk/upgrade

npm i https://pkg.pr.new/@clerk/upgrade@10219

@clerk/vue

npm i https://pkg.pr.new/@clerk/vue@10219

commit: 9354f2b

@mikepitre

Copy link
Copy Markdown
Contributor Author

Replaced by the regrouped stack #10235, #10236, #10237 and #10238, which ends on the same code (tree c2fc7c19). The old stack grew one PR per port from clerk-ios and clerk-android; the new one groups what a reviewer reads together.

@mikepitre mikepitre closed this Oct 14, 2026

This branch was successfully deployed

2 active deployments
Preview – swingset — 9354f2b8 Deployed Oct 13, 2026 by vercel[bot]
Preview – clerk-js-sandbox — 9354f2b8 Deployed Oct 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant