-
Notifications
You must be signed in to change notification settings - Fork 0
ci: standardize release process #7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| name: Check release metadata | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'pyproject.toml' | ||
| - 'CHANGELOG.md' | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| check: | ||
| name: Verify changelog matches version bump | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 | ||
| with: | ||
| python-version: '3.12' | ||
|
|
||
| - name: Check release metadata | ||
| run: python scripts/check-release.py |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| name: GitHub Release | ||
|
|
||
| on: | ||
| push: | ||
| tags: | ||
| - 'v[0-9]*' | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| jobs: | ||
| release: | ||
| name: Create GitHub Release | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | ||
|
|
||
| - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 | ||
| with: | ||
| python-version: '3.12' | ||
|
|
||
| - name: Read package metadata | ||
| id: meta | ||
| run: | | ||
| pkg_name=$(python -c "import tomllib,pathlib; print(tomllib.loads(pathlib.Path('pyproject.toml').read_text())['project']['name'])") | ||
| pkg_version="${GITHUB_REF_NAME#v}" | ||
| echo "name=${pkg_name}" >> "$GITHUB_OUTPUT" | ||
| echo "version=${pkg_version}" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Extract changelog notes | ||
| id: notes | ||
| run: | | ||
| set -euo pipefail | ||
| version="${GITHUB_REF_NAME#v}" | ||
| if [[ -f CHANGELOG.md ]]; then | ||
| body="$(python scripts/extract-changelog.py "$version")" | ||
| else | ||
| body="Release ${version}." | ||
| fi | ||
| delimiter="EOF_${RANDOM}_${RANDOM}" | ||
| { | ||
| echo "body<<${delimiter}" | ||
| echo "$body" | ||
| echo "${delimiter}" | ||
| } >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Create GitHub Release | ||
| uses: softprops/action-gh-release@1e812e8210a4a8a0b23075e5795f2a4e2b2a0b7 # v2.2.2 | ||
| with: | ||
| tag_name: ${{ github.ref_name }} | ||
| name: ${{ steps.meta.outputs.name }} ${{ steps.meta.outputs.version }} | ||
| body: ${{ steps.notes.outputs.body }} | ||
| generate_release_notes: false | ||
| make_latest: true | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| # Releasing | ||
|
|
||
| Every release uses `./scripts/release.sh`. Do not bump versions, tag, or create GitHub Releases manually. | ||
|
|
||
| ## One-time setup | ||
|
|
||
| - Install [GitHub CLI](https://cli.github.com/) (`gh`) and authenticate. | ||
| - Ensure PyPI [trusted publishing](https://docs.pypi.org/trusted-publishers/) is configured for this repo (`publish.yml` uses the `pypi` GitHub environment). | ||
|
|
||
| ## Release steps | ||
|
|
||
| 1. Add user-facing notes under `## [Unreleased]` in `CHANGELOG.md`. | ||
| 2. Prepare the release PR: | ||
|
|
||
| ```bash | ||
| ./scripts/release.sh prepare patch # or minor | major | 1.2.3 | ||
| ``` | ||
|
|
||
| 3. Merge the PR after CI passes (including the changelog check). | ||
| 4. Publish from a clean default branch checkout: | ||
|
|
||
| ```bash | ||
| git checkout main # or master for hotdata-marimo | ||
| git pull | ||
| ./scripts/release.sh publish | ||
| ``` | ||
|
|
||
| ## What happens automatically | ||
|
|
||
| Pushing a `vX.Y.Z` tag triggers two workflows: | ||
|
|
||
| | Workflow | Purpose | | ||
| |----------|---------| | ||
| | `publish.yml` | Build wheel/sdist and publish to PyPI | | ||
| | `release.yml` | Create the GitHub Release with notes from `CHANGELOG.md` | | ||
|
|
||
| ## Enforcement | ||
|
|
||
| - **PR check** (`check-release.yml`): if `pyproject.toml` version changes, `CHANGELOG.md` must contain a matching `## [X.Y.Z]` section. | ||
| - **Tag check** (`publish.yml`): the tag (without `v`) must match `[project].version` in `pyproject.toml`. | ||
| - **Publish guard** (`release.sh publish`): refuses to tag if the changelog section is missing. | ||
|
|
||
| Together, these make it hard to ship a version without changelog notes or a GitHub Release. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| #!/usr/bin/env python3 | ||
| """Fail CI when pyproject.toml version changes without a matching CHANGELOG entry.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import re | ||
| import subprocess | ||
| import sys | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. super nit: (not blocking) |
||
| from pathlib import Path | ||
|
|
||
|
|
||
| def git_show(path: str, ref: str) -> str: | ||
| try: | ||
| return subprocess.check_output(["git", "show", f"{ref}:{path}"], text=True) | ||
| except subprocess.CalledProcessError: | ||
| return "" | ||
|
|
||
|
|
||
| def read_version(text: str) -> str: | ||
| match = re.search(r'(?m)^version = "([^"]+)"', text) | ||
| if not match: | ||
| raise SystemExit("could not read version from pyproject.toml") | ||
| return match.group(1) | ||
|
|
||
|
|
||
| def has_changelog_section(version: str) -> bool: | ||
| changelog = Path("CHANGELOG.md") | ||
| if not changelog.exists(): | ||
| return False | ||
| return bool(re.search(rf"^## \[{re.escape(version)}\]", changelog.read_text(), re.M)) | ||
|
|
||
|
|
||
| def main() -> None: | ||
| base = "origin/main" | ||
| for candidate in ("origin/main", "origin/master"): | ||
| if subprocess.call(["git", "rev-parse", "--verify", candidate], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) == 0: | ||
| base = candidate | ||
| break | ||
|
|
||
| current = Path("pyproject.toml").read_text() | ||
| previous = git_show("pyproject.toml", base) | ||
| if not previous: | ||
| print("skip: no base pyproject.toml to compare") | ||
| return | ||
|
|
||
| old_version = read_version(previous) | ||
| new_version = read_version(current) | ||
| if old_version == new_version: | ||
| print(f"version unchanged ({new_version})") | ||
| return | ||
|
|
||
| if not has_changelog_section(new_version): | ||
| raise SystemExit( | ||
| f"pyproject.toml version bumped to {new_version} but CHANGELOG.md " | ||
| f"has no '## [{new_version}]' section" | ||
| ) | ||
|
|
||
| print(f"release metadata ok for {new_version}") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| #!/usr/bin/env python3 | ||
| """Print the Keep a Changelog section for a release version.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import re | ||
| import sys | ||
| from pathlib import Path | ||
|
|
||
|
|
||
| def extract(changelog: str, version: str) -> str: | ||
| pattern = rf"^## \[{re.escape(version)}\].*$" | ||
| match = re.search(pattern, changelog, re.M) | ||
| if not match: | ||
| raise SystemExit(f"no changelog section for {version}") | ||
|
|
||
| start = match.start() | ||
| rest = changelog[match.end() :] | ||
| next_heading = re.search(r"^## \[", rest, re.M) | ||
| end = match.end() + (next_heading.start() if next_heading else len(rest)) | ||
| section = changelog[start:end].strip() | ||
| title, _, body = section.partition("\n") | ||
| return body.strip() or f"Release {version}." | ||
|
|
||
|
|
||
| def main() -> None: | ||
| if len(sys.argv) != 2: | ||
| raise SystemExit("usage: extract-changelog.py VERSION") | ||
|
|
||
| version = sys.argv[1] | ||
| changelog = Path("CHANGELOG.md").read_text() | ||
| print(extract(changelog, version)) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| #!/usr/bin/env bash | ||
| # Generate publish.yml for a package. Usage: publish-workflow.sh hotdata-runtime | ||
| set -euo pipefail | ||
| pkg="${1:?package name}" | ||
| cat <<EOF | ||
| name: Publish to PyPI | ||
|
|
||
| on: | ||
| push: | ||
| tags: | ||
| - 'v[0-9]*' | ||
|
|
||
| concurrency: | ||
| group: pypi-publish-\${{ github.ref_name }} | ||
| cancel-in-progress: false | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| build: | ||
| name: Build distribution | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | ||
|
|
||
| - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 | ||
| with: | ||
| python-version: '3.12' | ||
|
|
||
| - name: Install build tooling | ||
| run: python -m pip install --upgrade build twine | ||
|
|
||
| - name: Verify tag matches pyproject version | ||
| run: | | ||
| if [[ ! "\$GITHUB_REF_NAME" =~ ^v[0-9] ]]; then | ||
| echo "Release tag '\$GITHUB_REF_NAME' must start with 'v' followed by a digit (e.g. v1.0.0)" >&2 | ||
| exit 1 | ||
| fi | ||
| tag="\${GITHUB_REF_NAME#v}" | ||
| pkg_version=\$(python -c "import tomllib,pathlib; print(tomllib.loads(pathlib.Path('pyproject.toml').read_text())['project']['version'])") | ||
| if [ "\$tag" != "\$pkg_version" ]; then | ||
| echo "Release tag (\$tag) does not match pyproject.toml version (\$pkg_version)" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: Build sdist and wheel | ||
| run: python -m build | ||
|
|
||
| - name: Check distribution metadata | ||
| run: python -m twine check --strict dist/* | ||
|
|
||
| - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5 | ||
| with: | ||
| name: dist | ||
| path: dist/ | ||
|
|
||
| publish: | ||
| name: Publish to PyPI | ||
| needs: build | ||
| runs-on: ubuntu-latest | ||
| environment: | ||
| name: pypi | ||
| url: https://pypi.org/p/${pkg} | ||
| permissions: | ||
| id-token: write | ||
| steps: | ||
| - uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5 | ||
| with: | ||
| name: dist | ||
| path: dist/ | ||
|
|
||
| - name: Publish via Trusted Publishing | ||
| uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0 | ||
| EOF |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: (not blocking) the
EOFheredoc delimiter is collidable. If a future changelog entry contains a line that's exactlyEOF, parsing of$GITHUB_OUTPUTwill terminate early and corrupt the body. GitHub's docs recommend a random delimiter, e.g.:Unlikely to bite, but trivially cheap to harden.