Skip to content

[Rust][FFI] Add Avro C ABI (avro feature) - #830

Draft
irinatomic-db wants to merge 2 commits into
mainfrom
irina-tomic_data/avro/ffi-c-abi
Draft

[Rust][FFI] Add Avro C ABI (avro feature)#830
irinatomic-db wants to merge 2 commits into
mainfrom
irina-tomic_data/avro/ffi-c-abi

Conversation

@irinatomic-db

@irinatomic-db irinatomic-db commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

PR stack

What changes are proposed in this pull request?

Exposes the Rust SDK's Avro support across the C FFI, behind the off-by-default avro feature (Beta). Additive only — existing C signatures and CStreamConfigurationOptions are byte-for-byte unchanged, so Go/C++/.NET keep building against the current ABI.

  • ffi/Cargo.toml: new avro feature forwarding to databricks-zerobus-ingest-sdk/avro.
  • ffi/cbindgen.toml: [defines] "feature = avro" = "ZEROBUS_AVRO", so the avro declarations are emitted in zerobus.h under #if defined(ZEROBUS_AVRO). The header is identical with or without the feature (freshness check stays green).
  • ffi/src/stream.rs:
    • New create fns zerobus_sdk_create_avro_stream (+ _async / _with_headers_provider _with_headers_provider_async), taking avro_schema_json as a null-terminated C string in the descriptor slot.
    • New ingest fns zerobus_stream_ingest_avro_record[s] (+ _async / _nowait), building EncodedRecord::Avro (raw pre-encoded datums, mirroring the proto path).
    • build_stream_from_parts gained an avro_schema_json param (forces RecordType::Avro.avro(schema)); the EncodedRecord→C-array match gained a gated Avro arm.
  • Regenerated ffi/zerobus.h (avro surface under #if defined(ZEROBUS_AVRO)).
  • ffi/NEXT_CHANGELOG.md entry; ci-rust.yml gains an --features avro build/test step.

Consumers build the FFI with --features avro and compile with -DZEROBUS_AVRO. Ephemeral streams only; server support is pending.

Once the avro gate is removed the new CI job will be removed as well.

How is this tested?

cd rust

# With the feature
cargo build -p zerobus-ffi --features avro
cargo test  -p zerobus-ffi --features avro
cargo clippy -p zerobus-ffi --features avro --all-targets

# Default (feature off) — no regressions
cargo build -p zerobus-ffi
cargo test  -p zerobus-ffi

# Whole workspace builds/tests
cargo build --workspace
cargo test  --workspace --no-run

cargo fmt --all --check

# Header stability: additive-only and identical with/without the feature,
# so the zerobus.h freshness check stays green.
cargo build -p zerobus-ffi && cp ffi/zerobus.h /tmp/h.h
cargo build -p zerobus-ffi --features avro && diff /tmp/h.h ffi/zerobus.h
git diff ffi/zerobus.h    # purely additive

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant