Skip to content

[worker] Wire SSH session into build - #4031

Open
gwdp wants to merge 6 commits into
gwdp/eng-24547-build-tools-ssh-sessionfrom
gwdp/eng-24547-worker-ssh-phase
Open

[worker] Wire SSH session into build#4031
gwdp wants to merge 6 commits into
gwdp/eng-24547-build-tools-ssh-sessionfrom
gwdp/eng-24547-worker-ssh-phase

Conversation

@gwdp

@gwdp gwdp commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Why

Wiring SSH into the worker build lifecycle: when a job has SSH enabled, open the session in SSH_SESSION and keep the VM up until supervision says it is safe to tear down.

How

  • Start the session early in build(), pass a hasJobFinished signal into the supervisor, and await teardown in finally
  • Leave SSH_SESSION open in the log UI for the life of the tunnel (doNotMarkEnd, emit END on teardown)
  • Opening the session is best-effort: failure warns and the job continues without SSH
  • Upterm comes from PATH (image bake) or a runtime download from gs://turtle-v2/upterm via build-tools; this PR does not vendor binaries into the worker package

Test Plan

Worker unit tests for the SSH phase. CI

@linear-code

linear-code Bot commented Jul 16, 2026

Copy link
Copy Markdown

ENG-24547

gwdp commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.97%. Comparing base (37ebacc) to head (0c6efc0).

Additional details and impacted files
@@                            Coverage Diff                             @@
##           gwdp/eng-24547-build-tools-ssh-session    #4031      +/-   ##
==========================================================================
+ Coverage                                   63.95%   63.97%   +0.03%     
==========================================================================
  Files                                        1033     1034       +1     
  Lines                                       47747    47778      +31     
  Branches                                    10043    10046       +3     
==========================================================================
+ Hits                                        30530    30561      +31     
  Misses                                      17116    17116              
  Partials                                      101      101              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from 38b2178 to d311a40 Compare July 16, 2026 19:05
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch 2 times, most recently from 534b4f1 to a7b6154 Compare July 16, 2026 22:18
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from d311a40 to 1bf156f Compare July 16, 2026 22:18
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch from a7b6154 to c9e54b9 Compare July 16, 2026 22:20
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch 2 times, most recently from c1bfc6f to f7e016d Compare July 17, 2026 02:37
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch 2 times, most recently from 12f0a14 to 12feb7a Compare July 21, 2026 02:08
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from f7e016d to c590f2f Compare July 21, 2026 02:08
@gwdp gwdp changed the title [worker] wire ssh session into build [worker] Wire SSH session into build Jul 21, 2026
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch from 12feb7a to cb90fa2 Compare July 24, 2026 00:52
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from c590f2f to f04922c Compare July 24, 2026 00:52
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch from cb90fa2 to 9409c52 Compare July 24, 2026 00:58
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from f04922c to 1c89f46 Compare July 24, 2026 00:58
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch from 9409c52 to daee76d Compare July 24, 2026 03:10
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from 1c89f46 to 3e17793 Compare July 24, 2026 03:10
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch from daee76d to 7154d05 Compare July 24, 2026 03:13
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from 3e17793 to 1e58c28 Compare July 24, 2026 03:13
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch from 7154d05 to 420bbef Compare July 24, 2026 03:28
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from 1e58c28 to aff1e33 Compare July 24, 2026 03:28
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch from 420bbef to 3dc29fa Compare July 24, 2026 03:37
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from aff1e33 to 8fe0c84 Compare July 24, 2026 03:37
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch from de1bff6 to f6de25e Compare July 25, 2026 04:41
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch 2 times, most recently from 0297f39 to 0859dec Compare July 25, 2026 04:42
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch from d5d5f1a to c4bce9c Compare July 27, 2026 00:23
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from 0859dec to bb0577f Compare July 27, 2026 00:24
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch from c4bce9c to 2aa0559 Compare July 27, 2026 00:49
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from bb0577f to 81a5ffa Compare July 27, 2026 00:49
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from 81a5ffa to 0eabaa0 Compare July 28, 2026 01:05

@sjchmiela sjchmiela left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh sorry i had this review from 4 days ago not sent

Comment thread packages/worker/package.sh Outdated
Comment thread packages/worker/package.sh Outdated
Comment thread packages/worker/package.sh Outdated
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch 4 times, most recently from 7149fa3 to 62efd17 Compare July 28, 2026 23:56
@gwdp
gwdp requested a review from sjchmiela July 29, 2026 00:11
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch from 62efd17 to 7b94ba2 Compare July 29, 2026 00:22
@gwdp
gwdp force-pushed the gwdp/eng-24547-build-tools-ssh-session branch 2 times, most recently from 2e5bca4 to 61e8f23 Compare August 3, 2026 20:12
@gwdp
gwdp force-pushed the gwdp/eng-24547-worker-ssh-phase branch 4 times, most recently from 4206350 to 2dad43c Compare August 3, 2026 23:00

@sjchmiela sjchmiela left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is good, i think the code layout due to the stack nature of the feature has grown a bit too wide, but it looks great

Comment thread packages/worker/src/sshSession.ts Outdated
done = (async () => {
let result = BuildPhaseResult.SUCCESS;
try {
await TurtleSshSession.superviseSshSessionAsync({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe the supervisesshsessionasync should be inlined here?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or at least in this file

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaning toward keeping it as its own function on purpose. The full supervise loop has its own unit tests but also would bloat this file. Let me know if you are soft on this 🙇‍♂️

@github-actions

Copy link
Copy Markdown

✅ Thank you for adding the changelog entry!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants