Skip to content

ci: add e2e-staging workflow for nightly staging tests #1253

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
piotr-iohk wants to merge 6 commits into
base: master
Choose a base branch
from
+263 −0
Open

ci: add e2e-staging workflow for nightly staging tests #1253

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
70c607d
ci: add e2e-staging workflow for nightly staging tests
cursoragent Sep 11, 2026
b3a5536
ci: fix notify-failure to cover upstream job failures
cursoragent Sep 11, 2026
e1b395c
ci: remove PR trigger and transfer shard from e2e-staging workflow
cursoragent Sep 11, 2026
97f0434
ci: add transfer_max shard to staging workflow
cursoragent Sep 11, 2026
3fa1500
ci: add notify_slack input to gate slack on dispatch
cursoragent Sep 11, 2026
2bcb369
ci: address e2e-staging workflow review feedback
cursoragent Sep 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
262 changes: 262 additions & 0 deletions .github/workflows/e2e-staging.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,262 @@
name: E2E Staging

on:
schedule:
- cron: "0 4 * * *"
workflow_dispatch:
inputs:
e2e_branch:
description: "Branch of synonymdev/bitkit-e2e-tests to use (main | default-feature-branch | custom branch name)"
required: false
default: "default-feature-branch"
post_to_slack:
description: "Post E2E staging summary to Slack (#bitkit-staging-nightly)"
type: boolean
required: false
default: false

env:
TERM: xterm-256color
FORCE_COLOR: 1

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name != 'schedule' }}

jobs:
build-staging:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v7

- name: Setup Java
uses: actions/setup-java@v6
with:
java-version: "17"
distribution: "temurin"

- name: Setup Gradle
uses: gradle/actions/setup-gradle@v6

- name: Decode google-services.json
run: |
set -euo pipefail
if [ -z "${GOOGLE_SERVICES_JSON_BASE64:-}" ]; then
echo "GOOGLE_SERVICES_JSON_BASE64 is empty; using checked-in placeholder."
exit 0
fi
mkdir -p app/src/debug
printf '%s' "$GOOGLE_SERVICES_JSON_BASE64" | base64 -d > app/src/debug/google-services.json
env:
GOOGLE_SERVICES_JSON_BASE64: ${{ secrets.GOOGLE_SERVICES_JSON_BASE64 }}

- name: Build debug app (regtest)
env:
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CHATWOOT_API: ${{ secrets.CHATWOOT_API }}
E2E: true
E2E_BACKEND: network
GEO: false
TREZOR_BRIDGE: true
TREZOR_BRIDGE_URL: http://10.0.2.2:21325
run: ./gradlew assembleDevDebug

- name: Rename APK
run: |
apk=$(find app/build/outputs/bitkit/devDebug -name 'bitkit-dev-debug-*-universal.apk')
mv "$apk" app/build/outputs/bitkit/devDebug/bitkit_e2e.apk

- name: Upload APK (regtest)
uses: actions/upload-artifact@v7
with:
name: bitkit-e2e-apk-staging_${{ github.run_number }}
path: app/build/outputs/bitkit/devDebug/bitkit_e2e.apk
Comment thread
cursor[bot] marked this conversation as resolved.

e2e-branch:
uses: synonymdev/bitkit-e2e-tests/.github/workflows/determine-e2e-branch.yml@main
with:
app_branch: ${{ github.head_ref || github.ref_name }}
e2e_branch_input: ${{ github.event.inputs.e2e_branch || 'default-feature-branch' }}

e2e-tests-staging:
Comment thread
cursor[bot] marked this conversation as resolved.
runs-on: ubuntu-latest
needs: [build-staging, e2e-branch]
timeout-minutes: 90

strategy:
fail-fast: false
matrix:
shard:
- { name: multi_address_2_regtest, grep: "@multi_address_2" }
- { name: pubky_paykit, grep: "@pubky" }
- { name: hardware_wallet, grep: "@hardware_wallet" }
- { name: transfer_max, grep: "@transfer_max" }

name: e2e-tests-staging - ${{ matrix.shard.name }}

steps:
- name: Show selected E2E branch
env:
E2E_BRANCH: ${{ needs.e2e-branch.outputs.branch }}
run: echo $E2E_BRANCH

- name: Clone E2E tests
uses: actions/checkout@v7
with:
repository: synonymdev/bitkit-e2e-tests
path: bitkit-e2e-tests
ref: ${{ needs.e2e-branch.outputs.branch }}

- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm

- name: AVD cache
uses: actions/cache@v6
id: avd-cache
with:
path: |
~/.android/avd/*
~/.android/adb*
key: avd-33-x86_64-pixel_6

- name: Create AVD and generate snapshot for caching
if: steps.avd-cache.outputs.cache-hit != 'true'
uses: reactivecircus/android-emulator-runner@v2
with:
profile: pixel_6
api-level: 33
arch: x86_64
avd-name: Pixel_6
force-avd-creation: false
emulator-options: -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-front none
disable-animations: false
script: echo "Generated AVD snapshot for caching."

- name: Download APK (regtest)
uses: actions/download-artifact@v8
with:
name: bitkit-e2e-apk-staging_${{ github.run_number }}
path: bitkit-e2e-tests/aut

- name: List APK directory contents
run: ls -l bitkit-e2e-tests/aut

- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 22

- name: Cache npm cache
uses: actions/cache@v6
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-

- name: Install dependencies
working-directory: bitkit-e2e-tests
run: npm ci

- name: Pull Trezor emulator image
if: matrix.shard.name == 'hardware_wallet'
working-directory: bitkit-e2e-tests
run: |
cd docker
docker compose --profile trezor pull --quiet trezor-user-env

- name: Run E2E Tests 1 (${{ matrix.shard.name }})
continue-on-error: true
id: test1
uses: reactivecircus/android-emulator-runner@v2
with:
profile: pixel_6
api-level: 33
arch: x86_64
avd-name: Pixel_6
force-avd-creation: false
emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-front none
script: cd bitkit-e2e-tests && ./ci_run_android.sh --mochaOpts.grep "${{ matrix.shard.grep }}"
env:
BACKEND: regtest
RECORD_VIDEO: true
ATTEMPT: 1

- name: Run E2E Tests 2 (${{ matrix.shard.name }})
continue-on-error: true
id: test2
if: steps.test1.outcome != 'success'
uses: reactivecircus/android-emulator-runner@v2
with:
profile: pixel_6
api-level: 33
arch: x86_64
avd-name: Pixel_6
force-avd-creation: false
emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-front none
script: cd bitkit-e2e-tests && ./ci_run_android.sh --mochaOpts.grep "${{ matrix.shard.grep }}"
env:
BACKEND: regtest
RECORD_VIDEO: true
ATTEMPT: 2

- name: Run E2E Tests 3 (${{ matrix.shard.name }})
id: test3
if: steps.test1.outcome != 'success' && steps.test2.outcome != 'success'
uses: reactivecircus/android-emulator-runner@v2
with:
profile: pixel_6
api-level: 33
arch: x86_64
avd-name: Pixel_6
force-avd-creation: false
emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-front none
script: cd bitkit-e2e-tests && ./ci_run_android.sh --mochaOpts.grep "${{ matrix.shard.grep }}"
env:
BACKEND: regtest
RECORD_VIDEO: true
ATTEMPT: 3

- name: Upload E2E Artifacts (${{ matrix.shard.name }})
if: failure()
uses: actions/upload-artifact@v7
with:
name: e2e-artifacts-staging_${{ matrix.shard.name }}_${{ github.run_number }}
path: bitkit-e2e-tests/artifacts/

- name: Dump docker logs on failure (${{ matrix.shard.name }})
if: failure()
uses: jwalton/gh-docker-logs@v2

slack-report:
if: >-
always() &&
(github.event_name == 'schedule' ||
(github.event_name == 'workflow_dispatch' && inputs.post_to_slack == true))
needs: [build-staging, e2e-branch, e2e-tests-staging]
runs-on: ubuntu-latest
steps:
- name: Checkout E2E tests
uses: actions/checkout@v7
with:
repository: synonymdev/bitkit-e2e-tests
ref: ${{ needs.e2e-branch.outputs.branch }}

@jvsena42 jvsena42 Sep 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low: this checks out the resolved e2e feature branch, but scripts/slack_summary.py exists only on main — so on a feature-branch dispatch the report step dies and no Slack message goes out at all.

I enumerated it rather than assuming: of the 9 branches in synonymdev/bitkit-e2e-tests, only main carries scripts/slack_summary.py. It landed there today in 6fbd12e, so every branch cut before that lacks it — including codex/pubky-ring-signup, which exists in the e2e repo and is a live app-side branch right now (#1224 and synonymdev/bitkit-ios#724).

Scenario: you dispatch E2E Staging from codex/pubky-ring-signup with post_to_slack: true. determine-e2e-branch resolves default-feature-branch to the same-named e2e branch, this line checks it out, and the next step dies with python3: can't open file '.../scripts/slack_summary.py'. Red slack-report job, zero Slack output — the exact reporting loss this job exists to prevent.

The scheduled nightly is safe today, which is why this is LOW: on schedule, github.ref_name is master, no master branch exists in the e2e repo, so it falls back to main. The window also self-heals as branches rebase past 6fbd12e.

Pin this one checkout to ref: main. The script is CI infrastructure, not test code — pinning it also stops a feature branch from shipping a broken slack_summary.py into the nightly report. The test-job checkout must keep tracking the resolved branch; that one is test code.


- name: Post Slack summary
run: python3 scripts/slack_summary.py e2e-staging

@jvsena42 jvsena42 Sep 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low, upstream: the shared script fails open, so a broken webhook leaves the nightly green and silent.

In synonymdev/bitkit-e2e-tests@main:scripts/slack_summary.py, post_to_slack returns False on a non-200 or a URLError, and returns True while printing to stderr when SLACK_WEBHOOK_URL is empty — but main() discards the value:

        post_to_slack(message)
        return 0

Scenario: the #bitkit-staging-nightly incoming webhook is revoked or rotated (routine when a Slack app is reinstalled). At 04:00 a shard fails, this step runs, Slack answers 404 invalid_token, the script prints one stderr line and exits 0, and the job goes green. The nightly silently stops reporting, and now that it posts on success too, "no message" is indistinguishable from "nothing ran". The slackapi/slack-github-action this replaced surfaced that as a red step.

Not this PR's defect and not blocking it — but both platforms now depend on that script, so it is worth a one-liner upstream: return 0 if post_to_slack(message) else 1.

I checked the adjacent risk and it is not live: SLACK_WEBHOOK_URL_STAGING does exist as a repo secret, so the empty-webhook silent-skip path isn't reachable today.

env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL_STAGING }}
PLATFORM: android
STAGING_RESULT: ${{ needs.build-staging.result == 'success' && needs.e2e-branch.result == 'success' && needs.e2e-tests-staging.result == 'success' && 'success' || 'failure' }}
BUILD_RESULT: ${{ needs.build-staging.result }}
E2E_BRANCH_RESULT: ${{ needs.e2e-branch.result }}
E2E_TESTS_RESULT: ${{ needs.e2e-tests-staging.result }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
RUN_ATTEMPT: ${{ github.run_attempt }}
E2E_TESTS_REF: ${{ needs.e2e-branch.outputs.branch }}
1 change: 1 addition & 0 deletions app/src/test/java/to/bitkit/build/BuildOutputContractTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ class BuildOutputContractTest {
"Justfile",
".github/workflows/e2e.yml",
".github/workflows/e2e_migration.yml",
".github/workflows/e2e-staging.yml",
".github/workflows/release.yml",
".github/workflows/release-internal.yml",
".agents/commands/release.md",
Expand Down
Loading