Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 33 additions & 4 deletions .github/workflows/publish-node.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ on:
tags:
- "node-v*"
workflow_dispatch:
inputs:
dry_run:
description: Build and verify packages without publishing
type: boolean
default: true

permissions:
contents: read
Expand Down Expand Up @@ -34,6 +39,7 @@ jobs:
build_args: ""
steps:
- uses: actions/checkout@v4
- run: python3 scripts/check-release.py node
- uses: dtolnay/rust-toolchain@stable
with:
toolchain: 1.88.0
Expand All @@ -50,6 +56,14 @@ jobs:
name: bindings-${{ matrix.target }}
path: bindings/node/datafog.*.node
if-no-files-found: error
- uses: actions/upload-artifact@v4
if: matrix.target == 'x86_64-unknown-linux-gnu'
with:
name: node-wrapper
path: |
bindings/node/native.js
bindings/node/index.d.ts
if-no-files-found: error

publish:
name: Publish to npm
Expand All @@ -58,25 +72,40 @@ jobs:
environment: npm
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24
registry-url: https://registry.npmjs.org
cache: npm
cache-dependency-path: bindings/node/package-lock.json
- run: npm install --global npm@11.17.0
- run: npm ci --prefix bindings/node
- uses: actions/download-artifact@v4
with:
pattern: bindings-*
path: bindings/node/artifacts
- uses: actions/download-artifact@v4
with:
name: node-wrapper
path: bindings/node
- name: Assemble platform packages
working-directory: bindings/node
run: |
npx napi create-npm-dirs
npx napi artifacts --output-dir artifacts
npx napi prepublish -t npm --no-gh-release --skip-optional-publish
- name: Verify assembled package and native dependency
working-directory: bindings/node
run: npm run test:package -- --prebuilt --platform-package npm/linux-x64-gnu
- name: Check all package contents without publishing
working-directory: bindings/node
run: |
for package in npm/*; do
(cd "$package" && npm publish --dry-run --ignore-scripts)
done
npm publish --dry-run --ignore-scripts
- name: Publish
if: github.event_name == 'push' || !inputs.dry_run
working-directory: bindings/node
run: npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
2 changes: 2 additions & 0 deletions .github/workflows/publish-python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ jobs:
target: x86_64-pc-windows-msvc
steps:
- uses: actions/checkout@v4
- run: python3 scripts/check-release.py python
- uses: PyO3/maturin-action@v1
with:
command: build
Expand All @@ -47,6 +48,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: python3 scripts/check-release.py python
- uses: PyO3/maturin-action@v1
with:
command: sdist
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/publish-rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ jobs:
environment: crates-io
steps:
- uses: actions/checkout@v4
- run: python3 scripts/check-release.py rust
- uses: dtolnay/rust-toolchain@stable
- run: cargo publish --package datafog-core --locked
env:
Expand Down
20 changes: 15 additions & 5 deletions .github/workflows/publish-wasm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ on:
tags:
- "wasm-v*"
workflow_dispatch:
inputs:
dry_run:
description: Build and verify the package without publishing
type: boolean
default: true

permissions:
contents: read
Expand All @@ -14,8 +19,12 @@ jobs:
name: Publish @datafog/wasm to npm
runs-on: ubuntu-latest
environment: npm
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v4
- run: python3 scripts/check-release.py wasm
- uses: dtolnay/rust-toolchain@stable
with:
toolchain: 1.88.0
Expand All @@ -24,12 +33,13 @@ jobs:
with:
node-version: 24
registry-url: https://registry.npmjs.org
cache: npm
cache-dependency-path: bindings/wasm/package-lock.json
- run: npm install --global npm@11.17.0
- run: cargo install wasm-bindgen-cli --version 0.2.127 --locked
- run: npm ci --prefix bindings/wasm
- run: npm run build --prefix bindings/wasm
- run: npx --prefix bindings/wasm playwright install --with-deps chromium
- run: npm run test:package --prefix bindings/wasm
- run: npm publish --dry-run --ignore-scripts
working-directory: bindings/wasm
- run: npm publish
if: github.event_name == 'push' || !inputs.dry_run
working-directory: bindings/wasm
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
8 changes: 4 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ detection settings remain separate from transformation policy.
DataFog Core is a separate distribution and canonical API, not a drop-in
replacement for the established `datafog` Python package.

| DataFog Python 4.8.x | DataFog Core 0.2.x |
| DataFog Python 4.8.x | DataFog Core 0.3.x |
| --- | --- |
| `pip install datafog` | `pip install datafog-core` |
| `from datafog.engine import ...` | `from datafog_core import ...` |
Expand Down
2 changes: 1 addition & 1 deletion bindings/node/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "datafog-node"
version = "0.2.0"
version = "0.3.0"
edition = "2024"
rust-version = "1.88"
description = "Node.js bindings for datafog-core"
Expand Down
4 changes: 2 additions & 2 deletions bindings/node/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion bindings/node/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@datafog/node",
"version": "0.2.0",
"version": "0.3.0",
"description": "Fast structured PII detection for Node.js, powered by Rust",
"license": "MIT",
"repository": {
Expand Down
2 changes: 1 addition & 1 deletion bindings/python/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "datafog-core-python"
version = "0.2.0"
version = "0.3.0"
edition = "2024"
rust-version = "1.88"
description = "Python bindings for datafog-core"
Expand Down
2 changes: 1 addition & 1 deletion bindings/python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "maturin"

[project]
name = "datafog-core"
version = "0.2.0"
version = "0.3.0"
description = "Fast structured PII detection implemented in Rust"
readme = "../../README.md"
license = "MIT"
Expand Down
2 changes: 1 addition & 1 deletion bindings/wasm/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "datafog-wasm"
version = "0.2.0"
version = "0.3.0"
edition = "2024"
rust-version = "1.88"
description = "WebAssembly bindings for datafog-core"
Expand Down
4 changes: 2 additions & 2 deletions bindings/wasm/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion bindings/wasm/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@datafog/wasm",
"version": "0.2.0",
"version": "0.3.0",
"description": "Fast structured PII detection for browsers, powered by Rust and WebAssembly",
"license": "MIT",
"repository": {
Expand Down
2 changes: 1 addition & 1 deletion crates/core/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "datafog-core"
version = "0.2.0"
version = "0.3.0"
edition = "2024"
rust-version = "1.88"
description = "Fast structured PII detection library"
Expand Down
5 changes: 3 additions & 2 deletions docs/bookkeeping-performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

This is a focused follow-up to [Slice Ten](finding-selection-performance.md).
The baseline is the merge of PR #13, commit
`083eaaab6bcfcfba9a31bc8ce1cfc99f1c2a3cae`. The change is a candidate for the
planned 0.3.0 release; package versions stay at 0.2.0 until release preparation.
`083eaaab6bcfcfba9a31bc8ce1cfc99f1c2a3cae`. The change is included in the
0.3.0 release. The measurements below compare the two implementations before
the release version bump.

## Scope and behavior

Expand Down
49 changes: 49 additions & 0 deletions docs/development.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,52 @@ bindings/wasm/ Browser WebAssembly binding
fixtures/ Shared conformance fixtures
docs/ Architecture records and Mintlify documentation
```

## Publish a release from GitHub Actions

Keep the four Cargo package versions, Python project version, both npm
package versions, and their lockfiles aligned. The publishing workflows run
`python3 scripts/check-release.py <runtime>` and reject inconsistent versions
or a release tag that does not match the package version.

Run the quality gates and installed-package tests above on the release commit.
Merge the release preparation PR, then create these tags on that same commit
for version 0.3.0:

| Tag | Workflow | Destination |
| --- | --- | --- |
| `rust-v0.3.0` | `publish-rust.yml` | crates.io |
| `python-v0.3.0` | `publish-python.yml` | PyPI |
| `node-v0.3.0` | `publish-node.yml` | npm: Node and five native platform packages |
| `wasm-v0.3.0` | `publish-wasm.yml` | npm: browser WASM |

Each tag push starts its publishing workflow. Check all four runs and install
the published versions before announcing the combined release. Registry
publication is not atomic across packages; inspect any partial result before
retrying. Never move a tag that has already published a package.

The npm workflows also accept a manual `dry_run` input, which defaults to
`true`. A dry run builds and verifies package contents without publishing.
For an actual manual publish, select the intended release tag and set
`dry_run` to `false`.

### npm trusted publisher setup

Configure [npm trusted publishing](https://docs.npmjs.com/trusted-publishers/)
once in each package's npm settings. Use GitHub organization **DataFog**,
repository **datafog-core**, environment **npm**, and allow direct
`npm publish`:

| Packages | Workflow filename |
| --- | --- |
| `@datafog/node`, `@datafog/node-darwin-arm64`, `@datafog/node-darwin-x64`, `@datafog/node-linux-x64-gnu`, `@datafog/node-linux-arm64-gnu`, `@datafog/node-win32-x64-msvc` | `publish-node.yml` |
| `@datafog/wasm` | `publish-wasm.yml` |

The publish jobs request GitHub's OIDC identity with `id-token: write` and use
npm 11.17.0 on Node 24. They do not use `NPM_TOKEN` or a local machine's npm
session. npm generates provenance for these public packages automatically.

The Node workflow builds all five native binaries, transfers the generated
JavaScript loader and types to the publish job, and runs the installed-package
suite against the assembled package and its Linux native dependency before
publishing. The WASM workflow runs the browser package suite before publishing.
4 changes: 4 additions & 0 deletions docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@
"reference/errors"
]
},
{
"group": "Releases",
"pages": ["releases/0-3-0"]
},
{
"group": "Contributing",
"pages": ["development"]
Expand Down
4 changes: 2 additions & 2 deletions docs/finding-selection-performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
This change addresses [issue #10](https://github.com/DataFog/datafog-core/issues/10)
in the shared Rust transformation path, following the structured PERSON work.
Public APIs, policies, detection coverage, offsets, and package versions are
unchanged. PERSON and this optimization are intended for a combined 0.3.0
release after review; this change does not publish packages.
unchanged. PERSON and this optimization are included in the combined 0.3.0
release; the original feature PR did not publish packages.

## Algorithm and compatibility

Expand Down
3 changes: 3 additions & 0 deletions docs/get-started/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ icon: "download"

## Package availability

The structured PERSON APIs require version **0.3.0 or newer** in every
binding. See the [0.3.0 release notes](/releases/0-3-0).

| Runtime | Distribution | Import | Availability |
| --- | --- | --- | --- |
| Rust | `datafog-core` | `datafog_core` | Published on crates.io |
Expand Down
Loading
Loading