Skip to content

Promote manifest bounds after schema evolution - #85

Merged
osipovartem merged 1 commit into
embucket-sync-df55.0.0from
df55-manifest-bounds-type-promotion
Sep 22, 2026
Merged

osipovartem merged 1 commit into
embucket-sync-df55.0.0from
df55-manifest-bounds-type-promotion

Conversation

@osipovartem

Copy link
Copy Markdown

Summary

  • promote existing int lower/upper bounds to long when a manifest is rewritten under an evolved schema
  • promote existing float bounds to double through the same metadata-only path
  • apply promotion in both existing-manifest rewrite constructors and cover lower/upper bounds with a focused unit test

Why

Iceberg allows int -> long, float -> double, and decimal precision widening without rewriting data files. When an append rewrites existing manifest entries under the current schema, old four-byte int/float bounds must be promoted before the new manifest is serialized. Otherwise readers decode those bytes using the widened schema and fail with could not convert slice to array. Decimal bounds need no byte rewrite because their minimal two's-complement encoding is precision-independent.

This is a manifest-metadata path only; it adds no per-row conversion or query execution overhead.

Validation

  • cargo +1.95.0 test -p iceberg-rust-spec promotes_manifest_bounds_to_evolved_schema
  • cargo +1.95.0 check -p iceberg-rust
  • cargo +1.95.0 clippy -p iceberg-rust-spec -p iceberg-rust --all-targets -- -D warnings
  • Rustice integration: append before and after int -> long, float -> double, and decimal precision widening; reload catalog; read both files; exercise predicate pruning

@osipovartem
osipovartem merged commit 4f91770 into embucket-sync-df55.0.0 Sep 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant