Migrate repository structure, tooling, CI/CD and docs to the uv zensical stack by raederan · Pull Request #129 · OpenSemanticLab/osw-python

raederan · 2026-07-07T14:30:35Z

⚠️ Attention

Before any merges, please create a baseline release beyond v1.1.2 including all latest PR merges before refactoring structure hard like this!

Target #127

Modernize osw-python to the layout already validated on
OO-LD/oold-python (based on the
cookiecutter-uv template): uv + hatchling/hatch-vcs packaging, ruff + ty +
deptry tooling, a single uv-based CI pipeline, PyPI trusted publishing and
zensical documentation with mike-based versioning (as on
OO-LD/oold-schema).

Changes

Packaging: PEP 621 [project] table in pyproject.toml;
setuptools/setuptools-scm replaced by hatchling + hatch-vcs (tag-derived
versioning preserved via no-guess-dev); committed uv.lock;
setup.py, setup.cfg and tox.ini removed.
Dev tooling: black/isort/flake8/autoflake replaced by ruff
(check + format); ty type checking and deptry dependency audit added;
Makefile with install/check/test/build/docs targets;
pre-commit rewritten on ruff-pre-commit.
Dependency fixes (surfaced by the new tooling): requests and
httpx declared (were transitive-only); missing
backports.strenum added for Python 3.10 (latent ImportError); eight
unused runtime dependencies removed (ruamel.yaml, pandas, chardet,
matplotlib, scipy, sphinx, stdlib-shadowing pathlib/asyncio); the
testing/dev extras replaced by a PEP 735 dev dependency group
(lock: 220 -> 187 packages).
CI: single main.yml (quality gate via make check + 8-leg test
matrix: ubuntu 3.10-3.13, macos/windows on 3.10+3.13) on a composite
setup-python-env action (setup-uv + uv sync --frozen); coverage
moved from Coveralls to Codecov (org-level token, informational
statuses); integration tests deselected by default (no secrets needed).
Release: tag-triggered on-release-main.yml publishes to PyPI via
OIDC trusted publishing (uv publish --trusted-publishing always,
environment pypi, no stored token).
Docs: dual Sphinx (ReadTheDocs) + MkDocs stack consolidated into
zensical; hero landing page, About/Get-Started guides and grouped API
reference (oold-python layout); versioned deployment with the
zensical-compatible mike fork to the gh-pages branch
(main -> dev, tag -> X.Y.Z + latest).
Housekeeping: README slimmed (pitch, install, quickstart, links);
CONTRIBUTING simplified with a tabbed development guide in the docs;
CHANGELOG frozen with a pointer to GitHub Releases; stray root
__init__.py and a sphinx-era docs/api gitignore rule removed.

Out of Scope

python-semantic-release: automated changelog/release-notes
generation is planned but deliberately not part of this migration.
Type-debt cleanup: ty runs green via a documented first-adoption
ruleset ([tool.ty.rules] downgrades) and ruff carries a documented
legacy-debt ignore block (typing modernization such as PEP 585/604
annotations); both are meant to be tightened incrementally.
Notebook rendering: docs/tutorials/basics.ipynb ships as a static
file (as before); rendered notebook docs are a possible follow-up.
Integration tests in CI: they no longer run in CI (previously ran
on push with secrets); a manual/scheduled workflow can be added later.

Further Steps

After merge, in order:

Confirm the org-level CODECOV_TOKEN secret is visible to this repo.
Configure the PyPI trusted publisher for project osw
(owner OpenSemanticLab, repo osw-python, workflow
on-release-main.yml, environment pypi); revoke PYPI_TOKEN after
the first successful OIDC release.
Let docs.yml run once so mike populates gh-pages (dev +
versions.json), remove the legacy mkdocs files from the gh-pages
root, then switch Pages source to "Deploy from a branch: gh-pages".
Disable the ReadTheDocs project.
Dry-run the release pipeline with a pre-release tag (e.g. v1.1.3rc1)
and verify PyPI upload + versioned docs deploy end to end.

Move project metadata and dependencies from setup.cfg into a PEP 621 [project] table, swap the setuptools/setuptools-scm backend for hatchling + hatch-vcs (version_scheme preserved as no-guess-dev), and add a committed uv.lock. setup.py and tox.ini are retained until their replacements are proven.

Add a composite setup-python-env action (setup-uv + uv sync --frozen) and a main.yml workflow with a quality job (make check) and an 8-leg test matrix (ubuntu 3.10-3.13, macos/windows on 3.10+3.13) running pytest and ty. Coverage goes to codecov via the org-level CODECOV_TOKEN with informational statuses. ci.yml is replaced by on-release-main.yml (interim token publish) so the workflow filename and pypi environment already match the upcoming OIDC trusted publisher configuration

Replace the dual sphinx (readthedocs) + mkdocs (gh-pages) stack with zensical: new zensical.toml (material theme, mkdocstrings on src), rewritten landing page, real docs/docs-test Makefile targets, a check-docs CI job, and a Pages-artifact deploy workflow. Remove conf.py, mkdocs.yml, .readthedocs.yml, docs/requirements.txt and the sphinx-only include shim pages

Enable the mike version provider in zensical.toml and deploy docs to the gh-pages branch: main pushes publish the dev version (docs.yml), release tags publish X.Y.Z with the latest alias and root redirect (on-release-main.yml). The zensical-compatible mike fork is pinned in the dev dependency group and commit-hash locked in uv.lock

Delete setup.py, setup.cfg, tox.ini and the stray root __init__.py. Remove eight unused runtime dependencies (ruamel.yaml, pandas, chardet, matplotlib, scipy, sphinx, and the stdlib-shadowing pathlib/asyncio), drop the testing extra in favor of the dev dependency group, and trim the deptry ignores accordingly (lock: 220 to 187 packages). Update README, CONTRIBUTING, docs/dev.md and test docstrings to the uv/make workflow; mark CHANGELOG as release-notes-based until python-semantic-release lands

Rebuild the site after the oold-python layout: hero landing page with logo and call-to-action buttons, about and get-started pages (tabbed uv/pip install, extras table, quickstart from real examples), API reference grouped under docs/api (fixing a sphinx-era gitignore rule that hid it), and dev.md as a tabbed make/manual mirror of the simplified CONTRIBUTING. Slim README down to pitch, install, quickstart and links

Give the live upload tests unique per-run file names and content so leftover pages on the shared test wiki can no longer trigger fileexists-no-change errors, and move the wiki-side cleanup of the target-fpt test into a finally block so failed runs do not leak state.

CredentialManager resolves credentials from memory, an existing file (read-only), the OSW_USERNAME/OSW_PASSWORD environment variables (e.g. via .env), or an interactive prompt, and no longer persists them: OswExpress drops the save-path prompt and auto-save, the prompt result stays in memory, save_credentials_to_file becomes an explicit deprecated opt-in and stops editing .gitignore. Update tests to the new contract and document the resolution order. Also keep the entity model pristine after tests and clean up warnings: snapshot and restore src/osw/model/entity.py around integration test sessions (fetch_schema regenerates it into the source tree by design), remove a self-referential deprecation on StoreOntologiesParam, make FileResult.close() a silent no-op on closed files like io streams, mark deliberately triggered warnings as expected per test, and filter non-actionable third-party warnings narrowly (122 -> 1 on the integration suite). BREAKING CHANGE: credentials are no longer written to disk automatically; rely on OSW_USERNAME/OSW_PASSWORD (or a .env file), an in-memory CredentialManager, or a self-managed credentials file.

Analyze conventional commits on every push to main: when a release is due, semantic-release updates the changelog (insertion flag, history preserved), commits, tags vX.Y.Z and creates the GitHub release; the same workflow run then builds, publishes to PyPI via trusted publishing and deploys the versioned docs. hatch-vcs derives the package version from the tag, so no version file is bumped. Replaces the manual tag-push release flow.

simontaurus · 2026-07-09T01:46:23Z

+"""Working-directory isolation for the integration tests.
+
+The tests and the library defaults create files relative to the current
+working directory - among them credential files in CLEAR TEXT


still the case with the use of env vars?

simontaurus · 2026-07-09T01:46:59Z

Not critical but there no obvious reason why the depr. marker was removed

simontaurus · 2026-07-09T01:54:15Z

        _deploy(param=deploy_param)


 @pytest.mark.skip(reason="Not deployable in test environment")


General comment: consider running local prefect server and run integration tests against it

simontaurus · 2026-07-09T02:12:51Z

@raederan @LukasGold : Since merging this to main would auto generate a new release we should first collect more potential breaking changes on a dev branch first
Suggestion for the general workflow: feature-branches => dev-branch => main

raederan · 2026-07-09T08:53:39Z

@simontaurus @LukasGold

Collecting breaking changes has downsides

One major bundling several breaks loses traceability (which change broke me?) and delays feedback
It turns small incremental upgrades into one big-bang migration
dev drifts from main and needs syncing; if the promotion squashes, it corrupts the version bump (semantic-release reads individual commits)

Going `feature -> main` wins us

A. One break per release, version maps 1:1 to change, clean to document
B. majors v2 -> v3 -> v4 in a row is fine
C. main always means releasable, no half-finished work parked anywhere

Recommendation

I would go with feature -> main, add a required-reviewer protection rule on the pypi GitHub environment (which cuts the release), and wire CI so integration tests run on the PR and the resulting version preview is commented right in it.

Alternative (not automated)

If you still want to stick with collecting breaking changes (which I would not recommend), we could stop the auto-release on push to main and collect changes there instead, so we still avoid the drift and maintenance of a separate dev branch: merges to main publish nothing, the release becomes an explicit manual trigger (a one-click "Run workflow") that cuts everything accumulated since the last tag, while integration tests and the version preview still run and comment on the PR exactly as in Option I.

raederan added 11 commits July 7, 2026 13:46

chore: ign local files

df14ee5

fix: typo

a4bb8a4

chore: replace black/isort/flake8 with ruff and add ty + deptry

cf21071

style: apply ruff format and lint auto-fixes

39e83f0

ci: switch release to pypi trusted publishing (oidc)

5e9ce62

raederan requested a review from simontaurus July 7, 2026 14:30

raederan self-assigned this Jul 7, 2026

raederan added documentation Improvements or additions to documentation enhancement New feature or request labels Jul 7, 2026

raederan added 2 commits July 7, 2026 16:36

fix: resolve environment-dependent ty failure on python 3.10

85fd9cc

fix: resolve environment-dependent ty failure and deptry warnings

13968ad

raederan requested a review from LukasGold July 7, 2026 14:38

raederan mentioned this pull request Jul 7, 2026

Cookiecutter UV Migration #127

Open

raederan added 3 commits July 8, 2026 15:09

simontaurus reviewed Jul 9, 2026

View reviewed changes

Comment thread src/osw/ontology.py

simontaurus Jul 9, 2026

Copy link
Copy Markdown

Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not critical but there no obvious reason why the depr. marker was removed

simontaurus reviewed Jul 9, 2026

View reviewed changes

simontaurus approved these changes Jul 9, 2026

View reviewed changes

raederan added the question Further information is requested label Jul 9, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Migrate repository structure, tooling, CI/CD and docs to the uv zensical stack#129

Migrate repository structure, tooling, CI/CD and docs to the uv zensical stack#129
raederan wants to merge 16 commits into
mainfrom
uv-zensical-migration

raederan commented Jul 7, 2026 •

edited

Loading

Uh oh!

simontaurus Jul 9, 2026

Uh oh!

simontaurus Jul 9, 2026

Uh oh!

simontaurus Jul 9, 2026

Uh oh!

simontaurus commented Jul 9, 2026

Uh oh!

raederan commented Jul 9, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		_deploy(param=deploy_param)


		@pytest.mark.skip(reason="Not deployable in test environment")

Uh oh!

Conversation

raederan commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Attention

Target #127

Changes

Out of Scope

Further Steps

Uh oh!

simontaurus Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

simontaurus Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

simontaurus Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

simontaurus commented Jul 9, 2026

Uh oh!

raederan commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Collecting breaking changes has downsides

Going feature -> main wins us

Recommendation

Alternative (not automated)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

raederan commented Jul 7, 2026 •

edited

Loading

raederan commented Jul 9, 2026 •

edited

Loading

Going `feature -> main` wins us