Skip to content

HADOOP-19930. ObjectWritable to validate class on load - #8640

Open
steveloughran wants to merge 1 commit into
apache:trunkfrom
steveloughran:pr/HADOOP-19864-writable-load-validation
Open

HADOOP-19930. ObjectWritable to validate class on load#8640
steveloughran wants to merge 1 commit into
apache:trunkfrom
steveloughran:pr/HADOOP-19864-writable-load-validation

Conversation

@steveloughran

@steveloughran steveloughran commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Validate class instantiation through ObjectWritable and elsewhere to make sure
the class is of the requested type before the class is instantiated

New reflection validation in ReflectionUtils
-Pulls up class type checking to before actual class load; fails fast -Add check of array length.
-Limit recursive depth to 100.

This is #8569 on a new branch to get yetus to pick it up

Contains content generated by Claude.ai

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

New reflection validation in ReflectionUtils
-Pulls up class type checking to before actual class load; fails fast
-Add check of array length.
-Limit recursive depth to 100.

+Sequence file logic.

Conf loading does force classloading, so move sequence
file logic out of it.

This does remove it from the conf file weak hash map of classes, but
the jvm does its own class map here. The real feature of the conf
map is the caching of cache misses, which would actually surface
as errors on reading files, so you aren't saving anything performance-wise.

Contains content generated by Claude.ai

Contributed by Steve Loughran.

# Conflicts:
#	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java
@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 6 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 2m 0s Maven dependency ordering for branch
+1 💚 mvninstall 43m 22s trunk passed
+1 💚 compile 16m 7s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 compile 16m 47s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 checkstyle 5m 55s trunk passed
+1 💚 mvnsite 4m 11s trunk passed
+1 💚 javadoc 3m 19s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 3m 14s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
-1 ❌ spotbugs 1m 7s /branch-spotbugs-hadoop-tools_hadoop-federation-balance-warnings.html hadoop-tools/hadoop-federation-balance in trunk has 12 extant spotbugs warnings.
+1 💚 shadedclient 28m 56s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 26s Maven dependency ordering for patch
+1 💚 mvninstall 2m 18s the patch passed
+1 💚 compile 15m 26s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javac 15m 26s the patch passed
+1 💚 compile 16m 38s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 javac 16m 38s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 5m 34s root: The patch generated 0 new + 431 unchanged - 1 fixed = 431 total (was 432)
+1 💚 mvnsite 4m 10s the patch passed
+1 💚 javadoc 3m 18s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 3m 15s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 6m 51s the patch passed
+1 💚 shadedclient 29m 5s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 23m 38s hadoop-common in the patch passed.
+1 💚 unit 8m 46s hadoop-mapreduce-client-core in the patch passed.
+1 💚 unit 2m 57s hadoop-federation-balance in the patch passed.
+1 💚 asflicense 1m 11s The patch does not generate ASF License warnings.
262m 48s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8640/1/artifact/out/Dockerfile
GITHUB PR #8640
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux bde632a7b1d2 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / df7fca6
Default Java Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8640/1/testReport/
Max. process+thread count 1577 (vs. ulimit of 10000)
modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-federation-balance U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8640/1/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants