Skip to content

Add an annotation framework for typed table properties - #2955

Draft
blackmwk wants to merge 10 commits into
apache:mainfrom
blackmwk:ir-2877
Draft

Add an annotation framework for typed table properties#2955
blackmwk wants to merge 10 commits into
apache:mainfrom
blackmwk:ir-2877

Conversation

@blackmwk

@blackmwk blackmwk commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

What changes are included in this PR?

  • Add the iceberg-property-macro crate with a Properties derive macro.
  • Define table-property keys, defaults, documentation, and custom converters using field annotations.
  • Generate Default, public getters, string-map parsing, and JSON serialization/deserialization.
  • Keep TableProperties fields private.
  • Change write.format.default from String to the typed DataFileFormat.
  • Update existing consumers to use the generated getters.
  • Update the public API snapshot.

Are these changes tested?

Yes.

  • Added unit tests for default and overridden property values.
  • Added JSON serialization and deserialization round-trip tests.
  • Added coverage for optional properties and custom parsing behavior.
  • Ran the Iceberg and DataFusion unit test suites.
  • Ran Clippy with warnings denied.
  • Verified formatting and the public API snapshot.

Comment thread crates/property-macro/src/lib.rs Outdated
Comment thread crates/iceberg/src/spec/table_properties.rs Outdated
Comment thread crates/iceberg/src/spec/table_properties.rs
Comment thread crates/iceberg/src/spec/table_properties.rs
@blackmwk
blackmwk marked this pull request as draft August 4, 2026 09:49
Comment thread crates/property-macro/Cargo.toml Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/compression.rs Outdated
Comment thread crates/iceberg/src/compression.rs Outdated
Comment thread crates/iceberg/src/compression.rs Outdated
Comment thread crates/iceberg/src/lib.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
Comment thread crates/iceberg/src/spec/parsed_table_prop.rs Outdated
};

let json = serde_json::to_value(&properties).unwrap();
let expected_parts = [

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect. I expect that if the value is same as default, we skip when serializing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Type safe TableProperties parsing.

1 participant