Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions submitqueue/extension/buildrunner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,11 @@ See [`doc/rfc/submitqueue/build-runner.md`](../../../doc/rfc/submitqueue/build-r
2. Map the `base` and `head` change slices onto the backend's build primitives (apply `base`, apply `head`, validate the result).
3. Map the runner's lifecycle states down to the `BuildStatus` values: `Accepted` (accepted for execution), `Running` (executing), and the terminal `Succeeded` / `Failed` / `Cancelled`.
4. Implement internal reconnect / retry so transient failures surface as plain errors without blocking the caller.

## Backends

- `fake`: local-development backend; every build succeeds unless a head change URI carries a failure marker (see `fake` package doc).
- `githubactions`: proof-of-architecture backend that dispatches a GitHub
Actions workflow. See [`githubactions/README.md`](githubactions/README.md)
for the workflow inputs and example orchestrator environment variables.
- `buildkite`: Buildkite-backed backend.
30 changes: 30 additions & 0 deletions submitqueue/extension/buildrunner/githubactions/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
load("@rules_go//go:def.bzl", "go_library", "go_test")

go_library(
name = "githubactions",
srcs = [
"client.go",
"githubactions.go",
],
importpath = "github.com/uber/submitqueue/submitqueue/extension/buildrunner/githubactions",
visibility = ["//visibility:public"],
deps = [
"//submitqueue/entity",
"//submitqueue/extension/buildrunner",
"@org_uber_go_zap//:zap",
],
)

go_test(
name = "githubactions_test",
srcs = ["githubactions_test.go"],
embed = [":githubactions"],
deps = [
"//core/httpclient",
"//submitqueue/entity",
"//submitqueue/extension/buildrunner",
"@com_github_stretchr_testify//assert",
"@com_github_stretchr_testify//require",
"@org_uber_go_zap//:zap",
],
)
91 changes: 91 additions & 0 deletions submitqueue/extension/buildrunner/githubactions/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# GitHub Actions BuildRunner

`githubactions` implements `buildrunner.BuildRunner` with GitHub Actions
`workflow_dispatch`. It is intended to prove the SubmitQueue BuildRunner
architecture against a common CI system without adding local state.

## How it works

1. `Trigger` dispatches the configured workflow on a trusted ref, usually
`main`, and returns GitHub's workflow run ID as the SubmitQueue build ID.
2. SubmitQueue passes these workflow inputs:
- `sq_base_uris`
- `sq_head_uris`
- `sq_queue`
- `sq_metadata`
3. `Status` calls GitHub's get-workflow-run endpoint with that run ID.
4. `Cancel` calls GitHub's cancel-workflow-run endpoint with that run ID.

## Minimal workflow

Create a workflow on the target repository's default branch:

```yaml
name: SubmitQueue CI
run-name: SubmitQueue ${{ inputs.sq_queue }}

on:
workflow_dispatch:
inputs:
sq_base_uris:
required: true
type: string
sq_head_uris:
required: true
type: string
sq_queue:
required: true
type: string
sq_metadata:
required: false
type: string

permissions:
contents: read

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Inspect SubmitQueue payload
run: |
echo '${{ inputs.sq_base_uris }}'
echo '${{ inputs.sq_head_uris }}'
echo '${{ inputs.sq_queue }}'
# Prototype: add a script here that applies sq_base_uris, then
# sq_head_uris, then runs the repository's real CI command.
```

The workflow definition should live on a trusted ref. The untrusted changes
should be represented by `sq_base_uris` and `sq_head_uris` and applied inside
the job.

## Integrator configuration

A server wiring this backend should provide the GitHub API client and runner
configuration equivalent to:

```sh
BUILD_RUNNER=githubactions
GITHUB_BASE_URL=https://api.github.com
GITHUB_TOKEN=<token with actions:read/actions:write>
Comment thread
albertywu marked this conversation as resolved.
GITHUB_ACTIONS_OWNER=uber
GITHUB_ACTIONS_REPO=submitqueue
GITHUB_ACTIONS_WORKFLOW=submitqueue-ci.yml
GITHUB_ACTIONS_REF=main
GITHUB_ACTIONS_EXTRA_INPUTS=runner=ubuntu-latest,test_command=make test
```

`GITHUB_ACTIONS_REF` defaults to `main`. `GITHUB_ACTIONS_EXTRA_INPUTS` is
optional comma-separated `key=value` data copied into every dispatch request;
use it for workflow-specific knobs like runner labels or test commands.

## Practical setup notes

- Use a trusted workflow definition from the target repository's default branch.
- Keep workflow permissions minimal. The job may apply untrusted changes before
running tests, so avoid broad secrets in this workflow.
- The backend proves the BuildRunner architecture. It does not prescribe how
your workflow materializes `sq_base_uris` and `sq_head_uris`; wire that to the
repository's existing patch/PR application logic.
178 changes: 178 additions & 0 deletions submitqueue/extension/buildrunner/githubactions/client.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,178 @@
// Copyright (c) 2025 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package githubactions

import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
)

const githubAPIVersion = "2026-03-10"

// client is a thin wrapper around the GitHub Actions REST endpoints that
// BuildRunner needs: workflow dispatch, get workflow run, and cancel run.
type client struct {
httpClient *http.Client
owner string
repo string
workflowID string
}

type dispatchWorkflowRequest struct {
Ref string `json:"ref"`
ReturnRunDetails bool `json:"return_run_details,omitempty"`
Inputs map[string]string `json:"inputs,omitempty"`
}

type dispatchWorkflowResponse struct {
WorkflowRunID int64 `json:"workflow_run_id"`
RunURL string `json:"run_url"`
HTMLURL string `json:"html_url"`
}

// workflowRun is the subset of a GitHub Actions workflow run object the
// runner needs.
type workflowRun struct {
ID int64 `json:"id"`
Name string `json:"name"`
DisplayTitle string `json:"display_title"`
Status string `json:"status"`
Conclusion string `json:"conclusion"`
HTMLURL string `json:"html_url"`
RunAttempt int `json:"run_attempt"`
Event string `json:"event"`
HeadBranch string `json:"head_branch"`
CreatedAt string `json:"created_at"`
}

func (c *client) dispatchWorkflow(ctx context.Context, req dispatchWorkflowRequest) (dispatchWorkflowResponse, error) {
body, err := json.Marshal(req)
if err != nil {
return dispatchWorkflowResponse{}, fmt.Errorf("marshal request: %w", err)
}
var resp dispatchWorkflowResponse
if err := c.do(ctx, http.MethodPost, c.workflowPath("dispatches"), body, &resp); err != nil {
return dispatchWorkflowResponse{}, err
}
return resp, nil
}

func (c *client) getRun(ctx context.Context, runID int64) (workflowRun, error) {
var run workflowRun
if err := c.do(ctx, http.MethodGet, c.runPath(runID), nil, &run); err != nil {
return workflowRun{}, err
}
return run, nil
}

func (c *client) cancelRun(ctx context.Context, runID int64) error {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.runPath(runID)+"/cancel", nil)
if err != nil {
return fmt.Errorf("create request: %w", err)
}
c.setHeaders(req)

resp, err := c.httpClient.Do(req)
if err != nil {
return fmt.Errorf("send request: %w", err)
}
defer resp.Body.Close()
_, _ = io.Copy(io.Discard, resp.Body)

switch resp.StatusCode {
case http.StatusAccepted, http.StatusOK, http.StatusCreated, http.StatusNoContent:
return nil
case http.StatusConflict, http.StatusUnprocessableEntity:
// Already terminal or otherwise not cancellable: no-op per
// BuildRunner.Cancel contract.
return nil
case http.StatusNotFound:
return fmt.Errorf("workflow run not found")
default:
return fmt.Errorf("unexpected status %d from cancel", resp.StatusCode)
Comment thread
albertywu marked this conversation as resolved.
}
Comment thread
albertywu marked this conversation as resolved.
Comment thread
albertywu marked this conversation as resolved.
}
Comment thread
albertywu marked this conversation as resolved.

func (c *client) workflowPath(suffix string) string {
path := fmt.Sprintf(
"/repos/%s/%s/actions/workflows/%s",
url.PathEscape(c.owner),
url.PathEscape(c.repo),
url.PathEscape(c.workflowID),
)
if suffix != "" {
path += "/" + suffix
}
return path
}

func (c *client) runPath(runID int64) string {
return fmt.Sprintf(
"/repos/%s/%s/actions/runs/%s",
url.PathEscape(c.owner),
url.PathEscape(c.repo),
url.PathEscape(strconv.FormatInt(runID, 10)),
)
}

func (c *client) do(ctx context.Context, method, rawURL string, body []byte, out any) error {
var bodyReader io.Reader
if body != nil {
bodyReader = bytes.NewReader(body)
}

req, err := http.NewRequestWithContext(ctx, method, rawURL, bodyReader)
if err != nil {
return fmt.Errorf("create request: %w", err)
}
c.setHeaders(req)

resp, err := c.httpClient.Do(req)
if err != nil {
return fmt.Errorf("send request: %w", err)
}
defer resp.Body.Close()

respBody, err := io.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("read response: %w", err)
}

if resp.StatusCode == http.StatusNotFound {
return fmt.Errorf("GitHub API returned 404 for %s %s", method, rawURL)
}
Comment thread
albertywu marked this conversation as resolved.
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("API returned status %d: %s", resp.StatusCode, respBody)
}

if out != nil && len(respBody) > 0 {
if err := json.Unmarshal(respBody, out); err != nil {
return fmt.Errorf("unmarshal response: %w", err)
}
}
return nil
}

func (c *client) setHeaders(req *http.Request) {
req.Header.Set("Accept", "application/vnd.github+json")
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-GitHub-Api-Version", githubAPIVersion)
}
Loading
Loading