Skip to content

test: remove simulator matrix - #326

Open
gmegidish wants to merge 1 commit into
mainfrom
test-remove-simulator-matrix
Open

test: remove simulator matrix#326
gmegidish wants to merge 1 commit into
mainfrom
test-remove-simulator-matrix

Conversation

@gmegidish

@gmegidish gmegidish commented Aug 1, 2026

Copy link
Copy Markdown
Member

Summary by CodeRabbit

  • Tests
    • Streamlined iOS simulator test coverage into a single, consolidated test suite.
    • Continued validation across screenshots, URLs, recordings, device behavior, apps, user interface interactions, lifecycle events, and filesystem operations.
    • Improved test diagnostics by displaying simulator logs during teardown.
    • Removed separate per-version test execution in favor of a simpler, more consistent validation process.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

The iOS simulator tests now run once in a shared suite. Per-version setup and execution were removed. Teardown logging remains. The screenshot test still references the removed iosVersion variable.

Changes

iOS simulator test consolidation

Layer / File(s) Summary
Shared simulator suite
test/simulator.spec.ts
The version-parameterized suites were replaced with one simulator suite. Existing simulator tests now execute once, with shared initialization and teardown. The screenshot filename references the undeclared iosVersion.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: removing the iOS simulator test matrix.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test-remove-simulator-matrix

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/simulator.spec.ts`:
- Around line 51-61: Update the screenshotPath construction in the “should take
screenshot” test to remove the deleted iosVersion reference, while preserving
the timestamp-based unique filename and existing screenshot verification flow.
- Around line 327-333: Update the removal assertion in the “should remove the
test directory from the app container” test to compare against the actual
basename of remoteDir, rather than the plain “mobilecli-test” prefix. Derive the
expected directory name from remoteDir using the existing path utilities, then
assert names does not contain that exact basename after fsRm.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 77da3240-d8e6-44d1-a75a-480997ce49a7

📥 Commits

Reviewing files that changed from the base of the PR and between 72b3349 and eb03d19.

📒 Files selected for processing (1)
  • test/simulator.spec.ts

Comment thread test/simulator.spec.ts
Comment on lines +51 to +61
test('should take screenshot', async () => {
test.skip(!simulatorId, 'simulator not found');

const screenshotPath = `/tmp/screenshot-ios${iosVersion}-${Date.now()}.png`;

takeScreenshot(simulatorId, screenshotPath);
verifyScreenshotFileWasCreated(screenshotPath);
verifyScreenshotFileHasValidContent(screenshotPath);

// console.log(`Screenshot saved at: ${screenshotPath}`);
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win

Remove the reference to the deleted iosVersion binding.

The version loop was removed, so iosVersion is no longer declared. Line 54 fails to compile under TypeScript and would throw a ReferenceError at runtime.

🐛 Proposed fix
-		const screenshotPath = `/tmp/screenshot-ios${iosVersion}-${Date.now()}.png`;
+		const screenshotPath = `/tmp/screenshot-ios-${Date.now()}.png`;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
test('should take screenshot', async () => {
test.skip(!simulatorId, 'simulator not found');
const screenshotPath = `/tmp/screenshot-ios${iosVersion}-${Date.now()}.png`;
takeScreenshot(simulatorId, screenshotPath);
verifyScreenshotFileWasCreated(screenshotPath);
verifyScreenshotFileHasValidContent(screenshotPath);
// console.log(`Screenshot saved at: ${screenshotPath}`);
});
test('should take screenshot', async () => {
test.skip(!simulatorId, 'simulator not found');
const screenshotPath = `/tmp/screenshot-ios-${Date.now()}.png`;
takeScreenshot(simulatorId, screenshotPath);
verifyScreenshotFileWasCreated(screenshotPath);
verifyScreenshotFileHasValidContent(screenshotPath);
// console.log(`Screenshot saved at: ${screenshotPath}`);
});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/simulator.spec.ts` around lines 51 - 61, Update the screenshotPath
construction in the “should take screenshot” test to remove the deleted
iosVersion reference, while preserving the timestamp-based unique filename and
existing screenshot verification flow.

Comment thread test/simulator.spec.ts
Comment on lines +327 to +333
test('should remove the test directory from the app container', async () => {
test.skip(!simulatorId, 'simulator not found');
fsRm(simulatorId, remoteDir, true);
const entries = fsList(simulatorId, `${containerPath}/Documents`);
const names = entries.map((e: any) => e.name);
expect(names).not.toContain('mobilecli-test');
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Strengthen the removal assertion.

remoteDir ends with a timestamp, so its basename is mobilecli-test-<timestamp>. expect(names).not.toContain('mobilecli-test') compares array elements for equality, so the plain string mobilecli-test never matches. The assertion passes even if fsRm leaves the directory in place. Assert on the actual basename instead.

♻️ Proposed fix
 		fsRm(simulatorId, remoteDir, true);
 		const entries = fsList(simulatorId, `${containerPath}/Documents`);
 		const names = entries.map((e: any) => e.name);
-		expect(names).not.toContain('mobilecli-test');
+		expect(names).not.toContain(path.basename(remoteDir));
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
test('should remove the test directory from the app container', async () => {
test.skip(!simulatorId, 'simulator not found');
fsRm(simulatorId, remoteDir, true);
const entries = fsList(simulatorId, `${containerPath}/Documents`);
const names = entries.map((e: any) => e.name);
expect(names).not.toContain('mobilecli-test');
});
test('should remove the test directory from the app container', async () => {
test.skip(!simulatorId, 'simulator not found');
fsRm(simulatorId, remoteDir, true);
const entries = fsList(simulatorId, `${containerPath}/Documents`);
const names = entries.map((e: any) => e.name);
expect(names).not.toContain(path.basename(remoteDir));
});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/simulator.spec.ts` around lines 327 - 333, Update the removal assertion
in the “should remove the test directory from the app container” test to compare
against the actual basename of remoteDir, rather than the plain “mobilecli-test”
prefix. Derive the expected directory name from remoteDir using the existing
path utilities, then assert names does not contain that exact basename after
fsRm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant