Skip to content

feat: DH-23168: Coerce parquet uint64 column - #8390

Merged
devinrsmith merged 1 commit into
deephaven:rc/v42.xfrom
robbcamera:rjc_42.x_DH-23168
Aug 21, 2026
Merged

feat: DH-23168: Coerce parquet uint64 column#8390
devinrsmith merged 1 commit into
deephaven:rc/v42.xfrom
robbcamera:rjc_42.x_DH-23168

Conversation

@robbcamera

Copy link
Copy Markdown
Contributor

Cherry-pick of #8350

Cherry-pick of deephaven#8350

Co-authored-by: Jianfeng Mao <4297243+jmao-denver@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings August 18, 2026 15:32
@github-actions

github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Deploying docs previews for ac82ec8 (available for 14 days)

Python
Groovy

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds configurable Parquet UINT_64 reading while preserving values accurately.

Changes:

  • Defaults UINT_64 columns to BigInteger.
  • Adds checked and signed long coercion modes.
  • Adds Java/Python tests and documentation.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
py/server/tests/test_parquet.py Tests Python coercion modes.
py/server/deephaven/parquet.py Exposes Python configuration API.
extensions/parquet/table/src/test/java/io/deephaven/parquet/table/TestParquetSchemaReader.java Tests schema inference.
extensions/parquet/table/src/test/java/io/deephaven/parquet/table/ParquetTableReadWriteTest.java Tests end-to-end reads.
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/ParquetSchemaReader.java Infers target column types.
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/ParquetInstructions.java Adds coercion instructions.
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/pagestore/topage/ToLongPage.java Supports checked long conversion.
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/pagestore/topage/ToBigIntegerPage.java Supports exact conversion.
extensions/parquet/table/src/main/java/io/deephaven/parquet/table/location/ParquetColumnLocation.java Selects appropriate materializers.
extensions/parquet/table/build.gradle Adds test dependency.
extensions/parquet/base/src/main/java/io/deephaven/parquet/base/materializers/LongFromUnsignedLongMaterializer.java Implements checked conversion.
extensions/parquet/base/src/main/java/io/deephaven/parquet/base/materializers/BigIntegerFromUnsignedLongMaterializer.java Implements exact conversion.
docs/python/reference/data-import-export/Parquet/readTable.md Documents read configuration.
docs/python/reference/data-import-export/Parquet/ColumnInstruction.md Documents Python option.
docs/python/how-to-guides/data-import-export/parquet-formats.md Explains Python coercions.
docs/groovy/reference/data-import-export/Parquet/readTable.md Adds related documentation.
docs/groovy/how-to-guides/data-import-export/parquet-instructions.md Documents Java instructions.
docs/groovy/how-to-guides/data-import-export/parquet-formats.md Explains Groovy coercions.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

if ci.codec_name:
builder.addColumnCodec(ci.column_name, ci.codec_name, ci.codec_args)
builder.useDictionary(ci.column_name, ci.use_dictionary)
if ci.unsigned_long_target is not None:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pydoc for the field says "it is ignored when writing". that's not entirely true at the moment. it should say "it is rejected when writing". i would push back against silently discarding the passed-in option because that feels gross. if they passed in something invalid, rejection is the right answer

so if anything, that one-word change in the pydoc would be my vote. i wouldn't think it's highly critical or anything, though

@rcaudy rcaudy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code owner approval for .py files.

@devinrsmith
devinrsmith merged commit 18db214 into deephaven:rc/v42.x Aug 21, 2026
34 of 37 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 21, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants