Blackbird: ClassFile generation of unrolled codec class, optimize record / builder / field

[stevenschlansker]

This PR replaces the Blackbird engine. The module coordinates, the BlackbirdModule API and the registration do not change. The LambdaMetafactory accessors are gone. For each bean type, Blackbird now generates one hidden class per direction with the JDK ClassFile API (JEP 484), and installs it as the deserializer and the serializer of that bean.

The branch is rebased on current 3.x as 9 commits. Each commit compiles and passes its tests, so the series can be reviewed one commit at a time:

1. Build Blackbird on JDK 25 as a multi-release jar
2. Replace the Blackbird deserializer engine with generated per-bean readers (POJOs)
3. Generate readers for records and builder-based beans
4. Replace the Blackbird serializer engine with generated per-bean writers
5. Remove the rest of the LambdaMetafactory engine
6. Add differential tests of Blackbird against stock databind
7. Update the Blackbird benchmarks for the generated-codec engine
8. Document the new Blackbird engine
9. Add the databind test corpus as a manual check for Blackbird changes

This version also addresses the review of 2026-09-23 and the UnsupportedClassVersionError report in FasterXML/jackson-future-ideas#108. I reply to each point separately.

JDK requirement

Blackbird needs Java 25 or later at run time. The code generators use the ClassFile API (final in Java 24) and compile at release 25, the current LTS release. The other modules of this repository keep the Java 17 baseline.

The jar is a multi-release jar. The two generators (BeanReaderGenerator, BeanWriterGenerator) compile at release 25 into META-INF/versions/25. All other classes, module-info and the ServiceLoader entry compile at release 17. On Java 17 to 24:

• Module discovery (MapperBuilder.findAndAddModules()) skips Blackbird with one logged warning, and it registers every other module. In the previous version of this PR, the class file 69 entry stopped the ServiceLoader loop with UnsupportedClassVersionError for all modules.
• An explicit addModule(new BlackbirdModule()) throws IllegalStateException, with a message that names the Java version. It also throws when findAndAddModules() runs before or after the explicit registration.
• On the module path, the JVM starts normally.

A BOM upgrade on Java 17 or 21 thus does not break an application that has Blackbird on its path. Users who need Blackbird acceleration on Java 17 to 24 can pin jackson-module-blackbird 3.2.x.

The build needs JDK 25 for Blackbird. On an older JDK, the reactor leaves out blackbird, blackbird-tests and blackbird-jpms-tests, and the other modules build as before.

The release must be built on JDK 25. A release build on an older JDK would publish no Blackbird artifact. To prevent that, a release profile in the root POM has an enforcer rule for JDK [25,). maven-release-plugin passes -Prelease (from oss-parent), so a release on an older JDK fails. The CI release and deploy job now runs on JDK 25.

Design notes

Stock base classes. A generated reader extends GeneratedReaderBase, which extends stock BeanDeserializer (GeneratedBuilderBase extends BuilderBasedDeserializer, and GeneratedWriterBase extends BeanSerializer). The codec is a copy of the resolved stock (de)serializer, and the generated class overrides deserialize() or serialize(). A writer also overrides serializeWithType() when the bean has no @JsonTypeId property, and the codec of a bean that declares views overrides _computeViewMask(). Each input that the generated loop does not model goes to the stock implementation of the same instance. Other modifiers, which often cast to BeanDeserializer, and the instanceof checks in databind see the type they expect. The placeholder that stands in from the modifier to resolution is also a stock subclass. This design also removed StockDeserializerProbe, StockSerializerProbe and the ThreadLocal of the previous version. Two small nested probes remain for protected members: BBWriterFactory.WriterProbe (fields of BeanPropertyWriter) and ReaderSupport.InstantiatorProbe (StdValueInstantiator.rewrapCtorProblem).

One rule for gates. The generated code handles a bean, or a property of it, only where that code does what stock databind does, and a differential test shows it. Everything else keeps stock code: the whole bean keeps the stock (de)serializer (a gate), or one property keeps its stock SettableBeanProperty or PropertyWriter inside the codec. This covers polymorphic values and default typing, custom null providers and null serializers, property wrappers (managed and back references, wrappers from other modifiers), external type ids, and a delegating creator next to a default creator. Decisions that stock databind already made (ignored and included names, aliases, case-insensitive matching, the selected creator, the build method) come from the resolved stock deserializer. Blackbird does not compute them again. For records, Blackbird builds its own name matcher, but it puts in only the names and aliases for which the stock lookup (PropertyBasedCreator.findCreatorProperty) finds the component. Failures go through the stock helpers (wrapAndThrow, wrapInstantiationProblem, the instantiator failure handling), so the exception class, the message and the reference path match stock, also for "Infinite recursion". A generated method over 7000 bytes is discarded: HotSpot does not compile methods over 8000 bytes.

Discovery provider. The ServiceLoader entry is internal.DiscoveredBlackbirdModule, a subclass of BlackbirdModule. Only this class is lenient on Java 17 to 24. The module can tell discovery from an explicit registration without a stack walk. Databind keeps one module for each registration id, the one added last. On Java 25, the provider has the registration id of BlackbirdModule, so discovery plus an explicit registration installs Blackbird once. On Java 17 to 24, the provider has its own id, so it cannot replace an explicit registration, which must fail.

Release guard. See above. PackagingTest checks that the root POM keeps the rule.

Exports. The JPMS module and the OSGi bundle export only tools.jackson.module.blackbird. The codecs are defined in Blackbird's own module (MethodHandles.lookup() of the generators), so no user module links to the internal packages. The bundle declares osgi.ee JavaSE 17 explicitly, because bnd derives it from the newest class file and maps class file 69 to UNKNOWN. The module no longer requires java.logging: generation failures log at DEBUG through System.Logger.

Member access. Generated code reaches members through MethodHandle constants that Blackbird unreflects with the public lookup after databind ran fixAccess. Blackbird thus reaches exactly what stock databind can invoke, and grants no access of its own. A user-supplied lookup is not necessary any more. The BlackbirdModule constructors and the findLookup() overrides stay for compatibility, unused. Hidden classes are defined without ClassOption.STRONG, so codecs unload with the mapper (#147).

Shared, padded write helpers. Scalar writes go through small static helpers in one generated class, WriteHelpers, that all writers share. The helpers are padded with nops past FreqInlineSize (325 bytes, C2) and C1MaxInlineSize (35 bytes, C1), so that neither compiler inlines them into serialize(). On aarch64, C2 copies of writeName and the NumberOutput digit code inlined into writers that hold list loops spill loop state to the stack. With the padding, RecordBenchmark.recordSer is 24-45% faster and PojoBenchmark.mediaItemSer 12-31% faster on aarch64 (JDK 25 and 27-ea, 5 forks, padding on/off/on/off), and BuilderBenchmark.builderSer 23% faster on x86_64. An earlier version of this PR emitted a copy of the helpers into each writer class. Each copy compiled separately, and on x86_64 the extra machine code caused instruction-cache misses (perfnorm: the same instruction count, 7% more cycles, IPC 3.7 against 4.0). The shared class makes recordSer 12% and mediaItemSer 7% faster on x86_64, and it removes about 2 KB of nops from each writer class, which lowers the cold-start cost per type by about 0.1 ms. -XX:+PrintInlining confirms that the helpers stay out of line. Graal uses other inlining rules. The comment at BeanWriterGenerator.INLINE_PAD records this.

Core and databind versions. Generated readers call JsonParser.nextNameMatchAndToken. Stock databind 3.3 calls it too, so Blackbird needs jackson-core and jackson-databind of the same version, as the README says.

Tests

Run on JDK 25:

./mvnw -B clean verify -Dblackbird.test.oldJdks=<jdk-17-home>,<jdk-21-home>

• blackbird (455 tests): the existing suite, plus engagement, demotion and behavior tests per feature. The suite runs with tools.jackson.module.blackbird.failOnCodegenError=true, so a generator bug fails the test instead of a silent fallback to stock.
• blackbird-tests (82 tests, classpath mode): engagement for beans in the unnamed module; differential tests (each scenario runs through a stock mapper and a Blackbird mapper, compares the value or the exception class, message and reference path, and asserts which path Blackbird took); and the packaged jar in child JVMs on JDK 25 and on each JDK that -Dblackbird.test.oldJdks lists (discovery on the class path and the module path with a second module provider on the path, and explicit registration with discovery before it, after it and without it), the module descriptor, the bundle exports, the release guard, and an isolated class loader. Without -Dblackbird.test.oldJdks, the old-JDK cases are skipped. CI passes Temurin 17 and 21.
• blackbird-jpms-tests (2 tests): a foreign named module that requires Blackbird, with package-private beans, on the module path.

Each finding of the review with testable behavior maps to a test that fails on the previous head and passes now. Our final review of this version found a builder setter, declared to return the builder, that returns null: stock keeps the current builder, and the generated codec failed. That case is fixed, and ReaderErrorsDiffTest.builderSetterReturningNullKeepsBuilder fails without the fix. The OSGi fix has a check of the manifest only, without an OSGi framework.

The databind test corpus is a manual check, not part of the build. blackbird-tests/corpus runs the whole jackson-databind test suite with Blackbird registered in every mapper, through a 10-line patch to databind's ObjectMapper(MapperBuilder) constructor. The blackbird-tests README says how to run it. Maintainers run it before they submit a Blackbird change. With databind f75dc9039fe0: 7476 tests, 7468 pass, 1 is skipped (as without Blackbird), and 7 fail. The 7 assert the exact list or number of registered modules, which the harness changes by one. The previous head had 72 failures. The corpus is manual because databind publishes no test jar and the hook patches databind main code, so a CI job would break when databind changes that constructor. If databind adds a supported test hook (a system property that names extra modules to register), a CI job for Blackbird and Afterburner becomes simple.

Benchmarks

The suite is in blackbird/benchmarks (records, builders, setter POJOs, 40- and 200-property beans, cold start). A classic-blackbird profile builds the same benchmarks against Blackbird 3.2.2. Throughput relative to stock databind on JDK 25.0.4, x86_64 (Ryzen 7 5800X3D). Each ratio is from one JMH run that also measured stock, and the 99.9% confidence intervals are 1 to 6% of the scores:

Benchmark	Classic 3.2.2	This PR
Record read	1.40x	2.32x
Record write	1.08x	1.22x
Builder read	1.01x	1.26x
Builder write	1.21x	1.44x
Setter POJO read (media item)	0.97x	1.22x
Setter POJO write (media item)	1.01x	1.19x
40-property bean write	1.25x	1.72x
200-property bean write	1.14x	2.07x

Two rows are slower than classic, and in both this PR keeps the stock deserializer. A POJO with a property-based @JsonCreator stays on stock (1.02x, classic 1.26x), because classic accelerated that creator. The 200-property read stays on stock (0.99x, classic 1.04x), because its generated reader would be over 7000 bytes, and HotSpot interpreted that reader at 0.27x of stock before the size limit. Cold start costs about 1 ms more per bean type than stock (stock 0.83 ms, this PR 1.85 ms, classic 1.82 ms). The README has the full table, and blackbird/benchmarks/README.md has the raw scores and the commands.

Compatibility notes

• Removed public types (listed in the release notes): deser.CreatorOptimizer, deser.ObjBooleanConsumer, the function interfaces of BBDeserializerModifier, ser.ToBooleanFunction, and the util package. BBDeserializerModifier and BBSerializerModifier take only the lookup function. The package-private CrossLoaderAccess is removed (closes Remove CrossLoaderAccess companion-class slow path from Blackbird (deprecated in 3.2) #350; its methods were deprecated for removal since 3.2).
• Fixes WrongMethodTypeException with blackbird #142: the affected code path is gone.
• Native-image detection is unchanged: the module does nothing in a native image.
• To see the generated code, run with -Dblackbird.debug.dumpDir=<dir> and decompile the class files. -Dblackbird.debug.codegen=true prints each gate decision.

The release target (3.3.0 or 3.4) is your decision. The release notes in release-notes/VERSION are under 3.3.0 for now.

[cowtowncoder]

Ok: is JDK-25 hard requirement for building/releasing Blackbird module? JDK 21 already needed for building jackson-core 3.1+, everything else built with 17.

EDIT: yes, should have read properly; Blackbird does rely on JDK 25 after this rewrite.
Since you are module owner @stevenschlansker you decide the baseline; I just want to figure out how to coordinate this, both wrt my release process and documentation side for users -- anyone with pre-JDK-25 will be unable to continue using Blackbird.
So basically: do we want this for Jackson 3.3.0, or for 3.4. I don't have release date for 3.3 so there is time, we just have to decide.

[cowtowncoder]

Code review (Claude Code)

I found 10 issues and one minor note, each checked against stock jackson-databind behavior. Nothing was run, and none of these cases is covered by the existing blackbird tests.

High

1. Beans that stock builds a special way go through the generated path anyway (BBReaderFactory.java:102). The check is only canCreateUsingDefault(), which misses two cases:
   • An EXTERNAL_PROPERTY type id: stock resolve() removes that property from _beanProperties, so the codec sends both the property and its type id to the unknown-property handler. Reading fails with UnrecognizedPropertyException, or the value is silently dropped.
   • A delegating @JsonCreator alongside a no-arg constructor: the codec uses the constructor plus setters instead of the delegate.
2. Polymorphic properties skip deserializeWithType (BBReaderFactory.java:378, and the record path at :311). A property with a value type deserializer still becomes a CHILD arm and calls plain deserialize(). This breaks @JsonTypeInfo(use=NAME) on concrete bases and activateDefaultTyping(NON_FINAL): the subtype is never built and the type id is read as an unknown property.

Medium

3. @JsonManagedReference is broken (BBReaderFactory.java:371, classify). The store handle comes from prop.getMember() whatever the property's class is. ManagedReferenceProperty (and any SettableBeanProperty.Delegating wrapper) forwards getMember(), so its deserializeAndSet is bypassed and the back-reference (child.parent) stays null.
4. Missing record components come out as null or 0 (BeanReaderGenerator.java:793). Stock PropertyValueBuffer._findMissing uses NullValueProvider.getAbsentValue and then deser.getAbsentValue. For example, record R(String name, Optional<String> nick) read from {"name":"x"} gives Optional.empty() in stock but null here. That null also wrongly trips FAIL_ON_NULL_CREATOR_PROPERTIES.
5. Constructor and build() exceptions aren't wrapped (BeanReaderGenerator.java:414, :852, :624). The no-arg constructor, the record canonical constructor and the builder build() are called through raw invokeExact:
   • A record whose compact constructor throws IllegalArgumentException leaks that raw exception instead of ValueInstantiationException.
   • DeserializationProblemHandler.handleInstantiationProblem is never called for beans that have properties.
   • The existing BustedCtor test only passes because that bean has no properties and stays on the stock path.
6. Names ignored because of @JsonIgnoreType are lost (BBDeserializerModifier.java:149). The codec's ignore list is rebuilt from annotations and misses names the stock factory adds with builder.addIgnorable (BeanDeserializerFactory.filterBeanProps). JSON containing those properties now fails in handleUnknownProperty instead of being skipped. Fix: read _ignorableProps, _includableProps and _ignoreAllUnknown from the resolved stock deserializer.
7. Custom null serializers are ignored (BBWriterFactory.java:118). STRING and CHILD arms write writeNull() directly, bypassing the property's _nullSerializer (for example @JsonSerialize(nullsUsing=...)).
8. The CHILD writer arm skips three things stock BeanPropertyWriter.serializeAsProperty does (BBWriterFactory.java:143):
   • The property's _typeSerializer, so type ids are dropped under USE_STATIC_TYPING with @JsonTypeInfo, or with final polymorphic types.
   • The self-reference check, so FAIL_ON_SELF_REFERENCES becomes a StackOverflowError.
   • _applyView.
9. Generated serialize / serializeWithType lose error wrapping (BeanWriterGenerator.java:307, :369). There is no equivalent of stock's catch (Exception) → wrapAndThrow or catch (StackOverflowError) → "Infinite recursion". A throwing getter escapes as a raw (possibly checked) exception with no path, and a cyclic object graph ends in a raw StackOverflowError instead of a DatabindException.
10. Blackbird now requires Java 25 (blackbird/pom.xml:30). It compiles at release 25 for the ClassFile API and only builds under the JDK 25+ profile, while the project baseline is Java 17:
    • Users on JDK 17–24 who upgrade get UnsupportedClassVersionError.
    • With the jar on the classpath of a JDK 17–24 app that uses findModules(), ServiceLoader iteration fails with ServiceConfigurationError, which breaks auto-registration of every module.

Low

• module-info.java:12 now exports tools.jackson.module.blackbird.internal. That goes against the project's rule not to export internal packages, and it looks unnecessary because the hidden classes are defined inside blackbird's own module.

Summary

The most urgent are #10, which affects every user below Java 25, and #1–2, which lose data or fail on reads that work today. Most of the rest come from the generated code skipping things stock BeanPropertyWriter / SettableBeanProperty do. Falling back to the stock path whenever a property has a type deserializer or serializer, a custom null handler, or a wrapper property class would close most of them.

🤖 Generated with Claude Code

[stevenschlansker]

@cowtowncoder Thank you for the review, I will do another iteration on this soon to address the findings.
Regarding baseline, yes this is a requirement for this approach because the ClassFile API is only JDK 24+ (and 24 is EOL, so does not make much sense as a baseline). I agree this is unfortunate for users who want the best performance but cannot upgrade.

That said, the last Blackbird release still works same as it always has - such users could pin it, at least for now until Jackson API drifts too far. (I expect not many changes will hit 2.x at least at this point?). On top of that, with modern reflection and Jackson improvements, the classic Blackbird hardly wins in a meaningful way anymore.

I have never run the Jackson release process so I want to make very sure I will not break anything for you. The goal here is that only the build JDK must be 25, all other modules still run fine with their existing 17 baseline, but I am not 100% sure how to test every path you use.

I am of course always excited to "ship it" but I do not want to push anything too fast or break everything all at once. I see the risk as manageable since there is always the "bug found, disable blackbird for now" break-glass option. Regarding 3.3.0 vs 3.4, I will accept your judgement as wise and all-powerful maintainer :) I will try to remain responsive to your concerns as we decide whether to merge before 3.3 is cut or sometime later. Once I address feedback, my next step will be to run this in test and then production on our own platform, and I will report the experience back here.

[cowtowncoder]

@stevenschlansker Excellent, sounds like a plan -- we are in full agreement here.

Performance improvement improvement looks really good so looking forward to getting this merged.

[stevenschlansker]

I refreshed the PR with fixes to all problems reported so far. Working on production testing now (have to upgrade our app to Jackson 3 first...)

[stevenschlansker]

The new refresh also includes some manual tests (not hooked up to CI currently) to re-run the full Jackson test suite with Blackbird force-installed. This helps makes sure we do not break any core behavior.