Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion mkdocs/docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -2123,7 +2123,7 @@ PyIceberg integrates with [Apache DataFusion](https://datafusion.apache.org/) th

The integration has a few caveats:

- Only works with `datafusion == 51`, aligns with the version used in `pyiceberg-core`
- Only works with `datafusion == 53`, aligns with the version used in `pyiceberg-core`
- Depends directly on `iceberg-rust` instead of PyIceberg's implementation
- Has limited features compared to the full PyIceberg API

Expand Down
7 changes: 5 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,11 @@ sql-sqlite = ["sqlalchemy>=2.0.18,<3"]
gcsfs = ["gcsfs>=2023.1.0"]
rest-sigv4 = ["boto3>=1.24.59"]
hf = ["huggingface-hub>=0.24.0"]
pyiceberg-core = ["pyiceberg-core>=0.5.1,<0.10.0"]
datafusion = ["datafusion>=52,<53"]
# `pyiceberg-core` bundles a `datafusion-ffi` build, and the DataFusion FFI ABI is not stable

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get rid of this comment? Not sure if it's valuable.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather keep it while pins have to match exactly? Otherwise this knowledge is implicit and not written down anywhere (besides Github issues)

# across major versions (https://github.com/apache/datafusion/issues/17374). The two ranges below
# must name the same DataFusion major version: pyiceberg-core 0.10.x bundles datafusion-ffi 53.x.
pyiceberg-core = ["pyiceberg-core>=0.10.1,<0.11.0"]
datafusion = ["datafusion>=53,<54"]
gcp-auth = ["google-auth>=2.4.0"]
entra-auth = ["azure-identity>=1.25.1"]
geoarrow = ["geoarrow-pyarrow>=0.2.0"]
Expand Down
32 changes: 16 additions & 16 deletions uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading