Search before asking
Motivation
Search before asking
Motivation
PartitionsRead.withFilter and FilesRead.withFilter are currently no-op
stubs marked // TODO. As a result, predicates on system tables such as
SELECT * FROM t$files WHERE schema_id = 0 rely entirely on the upstream
engine to filter the result, even though the source could prune rows
much earlier.
Solution
Apply the predicate as a post-filter on the materialized row before projection.
For FilesRead, leaves referencing partition / bucket / level need to
be stripped because they are already pushed down by FilesScan — re-evaluating
them on the materialized row may throw ClassCastException due to type mismatch
(e.g. a String literal vs the BinaryString in the row).
PartitionsRead keeps the predicate as-is because PartitionsScan does not
push anything down and PartitionsTable schema does not contain bucket
/ level columns.
POC ready (already implemented locally with tests, will open PR once assigned).
Anything else?
No.
Are you willing to submit a PR?
Solution
No response
Anything else?
No response
Are you willing to submit a PR?
Search before asking
Motivation
Search before asking
Motivation
PartitionsRead.withFilterandFilesRead.withFilterare currently no-opstubs marked
// TODO. As a result, predicates on system tables such asSELECT * FROM t$files WHERE schema_id = 0rely entirely on the upstreamengine to filter the result, even though the source could prune rows
much earlier.
Solution
Apply the predicate as a post-filter on the materialized row before projection.
For
FilesRead, leaves referencingpartition/bucket/levelneed tobe stripped because they are already pushed down by
FilesScan— re-evaluatingthem on the materialized row may throw ClassCastException due to type mismatch
(e.g. a String literal vs the BinaryString in the row).
PartitionsReadkeeps the predicate as-is becausePartitionsScandoes notpush anything down and
PartitionsTableschema does not containbucket/
levelcolumns.POC ready (already implemented locally with tests, will open PR once assigned).
Anything else?
No.
Are you willing to submit a PR?
Solution
No response
Anything else?
No response
Are you willing to submit a PR?