Generate class transformers ahead of time

.gitignore

@@ -8,3 +8,4 @@ infection.log
 infection.html
 .phpbench/
 docs_php/
+var

Makefile

@@ -34,6 +34,10 @@ static: phpstan cs                                               				## run stat
 
 test: phpunit                                                                   ## run tests
 
+.PHONY: snapshot
+snapshot: vendor                                                                ## regenerate the snapshots of the generated transformers
+	UPDATE_SNAPSHOTS=1 vendor/bin/phpunit --no-coverage --filter testGeneratedCodeMatchesSnapshot
+
 .PHONY: benchmark
 benchmark: vendor                                                               ## run benchmarks
 	vendor/bin/phpbench run tests/Benchmark --report=default

docs/extensions.md

@@ -23,13 +23,14 @@ $hydrator = (new StackHydratorBuilder())
 ```
 ## Built-in extensions
 
-The library ships with three extensions out of the box:
+The library ships with four extensions out of the box:
 
 | Extension | Purpose |
 | --- | --- |
 | `LifecycleExtension` | [Lifecycle hooks](lifecycle-hooks.md), run code before and after the extract and hydrate process. |
 | `CryptographyExtension` | [Cryptography](cryptography.md), encrypt and decrypt sensitive data with crypto-shredding. |
 | `UpcastExtension` | [Upcasting](upcasting.md), reshape outdated stored data while it is hydrated. |
+| `GeneratedTransformerExtension` | [Generated code](generated-code.md), generated mapping code instead of reflection. |
 
 ## Middleware
 
@@ -156,7 +157,7 @@ serializable.
 At the end of the stack a `ClassTransformer` turns the array into the object
 and back. By default this is the `ReflectionTransformer`, which sets and reads
 the properties with reflection. A transformer factory can provide another
-transformer for a class, for example generated code.
+transformer for a class, for example [generated code](generated-code.md).
 
 ```php
 use Patchlevel\Hydrator\Metadata\ClassMetadata;

docs/generated-code.md

@@ -0,0 +1,98 @@
+# Generated Code
+
+By default the `ReflectionTransformer` maps arrays to objects with reflection:
+it iterates over the metadata of a class, sets each property through a
+`ReflectionProperty` and calls the [normalizers](normalizer.md) dynamically.
+This is fast enough for most applications, but when you hydrate millions of
+objects the reflection overhead adds up. The `GeneratedTransformerExtension`
+uses generated plain PHP code instead, one class per hydrated class.
+
+## Enable the extension
+
+Register the `GeneratedTransformerExtension` with the directory where the
+generated transformers are stored.
+
+```php
+use Patchlevel\Hydrator\Extension\Generated\GeneratedTransformerExtension;
+use Patchlevel\Hydrator\StackHydratorBuilder;
+
+$hydrator = (new StackHydratorBuilder())
+    ->useExtension(new GeneratedTransformerExtension(__DIR__ . '/var/cache/hydrator'))
+    ->build();
+```
+The extension registers a [transformer factory](extensions.md#transformer-factories),
+nothing else changes: all middlewares run as before and the generated code
+takes the place of reflection at the end of the stack. If no middleware has to
+run for a class, the hydrator calls the generated code directly.
+
+:::success
+The generated code behaves exactly like the `ReflectionTransformer`: objects are
+created without calling the constructor, missing fields keep their promoted
+default value, private and readonly properties are supported and the same
+exceptions are thrown. You can switch between both without changing your
+classes.
+:::
+
+## Warmup
+
+Code is never generated while the hydrator is used. The extension only loads
+transformers which already exist, so the cache path can be read-only at
+runtime. Generate them ahead of time with the `GeneratedTransformerWarmer`, for
+example when the application is deployed. Pass the hydrator itself, so the code
+is generated with the guessers and enrichers of all extensions.
+
+```php
+use Patchlevel\Hydrator\Extension\Generated\GeneratedTransformerExtension;
+use Patchlevel\Hydrator\Extension\Generated\GeneratedTransformerWarmer;
+use Patchlevel\Hydrator\StackHydratorBuilder;
+
+$hydrator = (new StackHydratorBuilder())
+    ->useExtension(new GeneratedTransformerExtension(__DIR__ . '/var/cache/hydrator'))
+    ->build();
+
+$warmer = new GeneratedTransformerWarmer($hydrator, __DIR__ . '/var/cache/hydrator');
+$warmer->warmup([
+    ProfileCreated::class,
+    NameChanged::class,
+]);
+```
+Classes referenced by these classes, such as nested value objects, are
+discovered and generated automatically.
+
+:::note
+Classes without generated code are transformed with reflection. A missing
+warmup makes the hydrator slower, but never breaks it.
+:::
+
+## Outdated code
+
+The generated code is tied to the [metadata](caching.md) and the definition of
+your classes. The name of a generated transformer contains a fingerprint of
+the class and its nested classes: their properties, field names, normalizers,
+visibility and defaults. If one of them changes, the existing code no longer
+matches and the class is transformed with reflection until the warmup runs
+again. Old files are never used again, you can delete them when you deploy.
+
+## Nested objects
+
+The biggest speedup comes from inlining nested objects: instead of going through
+the hydrator again for every nested object, the generated code hydrates and
+extracts them directly. Inlining a class would hide it from the middlewares,
+so it only happens when every middleware is a `SkippableMiddleware` which
+[skips](extensions.md#skipping-middlewares) that class. The decision is made
+per class and per direction: with the [cryptography](cryptography.md) extension
+for example, value objects without sensitive data are inlined while classes
+with sensitive data still go through the stack.
+
+:::note
+Nested classes with a class level normalizer, [lazy](lazy.md) classes and
+nested classes which changed since the warmup are never inlined, they always
+take the regular path through the hydrator. The same applies if you wrap the
+hydrator in your own `Hydrator`, so your wrapper also sees nested objects.
+:::
+
+## Learn more
+
+* [How to build the hydrator](hydrator.md)
+* [How to cache the metadata](caching.md)
+* [How to write your own extension](extensions.md)

docs/project.json

@@ -18,7 +18,8 @@
       "subEntries": [
         { "title": "Lifecycle Hooks", "file": "lifecycle-hooks.md" },
         { "title": "Cryptography", "file": "cryptography.md" },
-        { "title": "Upcasting", "file": "upcasting.md" }
+        { "title": "Upcasting", "file": "upcasting.md" },
+        { "title": "Generated Code", "file": "generated-code.md" }
       ]
     },
     {

phpcs.xml.dist

@@ -10,6 +10,8 @@
 
     <file>src</file>
     <file>tests</file>
+    <!-- generated code, kept as snapshot -->
+    <exclude-pattern>tests/Unit/Extension/Generated/Snapshot/*</exclude-pattern>
 
     <rule ref="PatchlevelCodingStandard">
         <exclude name="Generic.CodeAnalysis.UselessOverridingMethod.Found"/>

phpstan.neon.dist

@@ -8,6 +8,8 @@ parameters:
 	paths:
 		- src
 		- tests
+	excludePaths:
+		- tests/Unit/Extension/Generated/Snapshot
 	reportUnmatchedIgnoredErrors: false
 
 services:
@@ -19,3 +21,7 @@ services:
 		class: Patchlevel\Hydrator\Tests\Architecture\ExceptionImplementsHydratorExceptionTest
 		tags:
 			- phpat.test
+	-
+		class: Patchlevel\Hydrator\Tests\Architecture\NoCodeGenerationAtRuntimeTest
+		tags:
+			- phpat.test

src/Extension/Generated/ClassFingerprint.php

@@ -0,0 +1,125 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Patchlevel\Hydrator\Extension\Generated;
+
+use Patchlevel\Hydrator\Metadata\ClassMetadata;
+use Patchlevel\Hydrator\Metadata\ClassNotFound;
+use Patchlevel\Hydrator\Metadata\MetadataFactory;
+use Patchlevel\Hydrator\Normalizer\ArrayNormalizer;
+use Patchlevel\Hydrator\Normalizer\InvalidType;
+use Patchlevel\Hydrator\Normalizer\Normalizer;
+use Patchlevel\Hydrator\Normalizer\ObjectNormalizer;
+
+use function array_pop;
+use function hash;
+use function ksort;
+use function serialize;
+use function substr;
+
+/**
+ * Fingerprint of everything the generated code of a class depends on: the properties with their field names,
+ * normalizers, visibility, declaring class and inlined defaults. The fingerprints of a class and its nested classes are
+ * part of the name of the generated transformer, and nested classes are only inlined if their fingerprint still matches.
+ *
+ * @internal
+ */
+final class ClassFingerprint
+{
+    /**
+     * @param ClassMetadata<T> $metadata
+     *
+     * @template T of object
+     */
+    public static function of(ClassMetadata $metadata): string
+    {
+        $defaults = $metadata->promotedConstructorDefaults();
+        $shape = [$metadata->reflection->isFinal()];
+
+        foreach ($metadata->properties as $property) {
+            $reflection = $property->reflection;
+            $default = null;
+
+            if ($reflection->isPromoted() && isset($defaults[$property->propertyName])) {
+                // defaults which can not be inlined are resolved at runtime, their value does not matter here
+                $default = Literal::export($defaults[$property->propertyName]->getDefaultValue()) ?? 'runtime';
+            }
+
+            $declaringClass = $reflection->getDeclaringClass()->getName();
+
+            $shape[] = [
+                $property->propertyName,
+                $property->fieldName,
+                $property->normalizer !== null,
+                $reflection->getModifiers(),
+                // inherited properties may need the scope of their declaring class
+                $declaringClass === $metadata->className ? null : $declaringClass,
+                $default,
+            ];
+        }
+
+        return substr(hash('sha256', serialize($shape)), 0, 16);
+    }
+
+    /**
+     * Fingerprint of the class together with every class reachable through object normalizers, since the generated
+     * code of a class contains the inlined code of its nested classes.
+     *
+     * @param ClassMetadata<T> $metadata
+     *
+     * @template T of object
+     */
+    public static function ofGraph(ClassMetadata $metadata, MetadataFactory $metadataFactory): string
+    {
+        /** @var array<class-string, string> $fingerprints */
+        $fingerprints = [];
+        $queue = [$metadata];
+
+        while ($queue !== []) {
+            $current = array_pop($queue);
+
+            if (isset($fingerprints[$current->className])) {
+                continue;
+            }
+
+            $fingerprints[$current->className] = self::of($current);
+
+            foreach ($current->properties as $property) {
+                $nested = self::nestedClass($property->normalizer);
+
+                if ($nested === null || isset($fingerprints[$nested])) {
+                    continue;
+                }
+
+                try {
+                    $queue[] = $metadataFactory->metadata($nested);
+                } catch (ClassNotFound) {
+                    continue;
+                }
+            }
+        }
+
+        ksort($fingerprints);
+
+        return substr(hash('sha256', serialize($fingerprints)), 0, 16);
+    }
+
+    /** @return class-string|null */
+    private static function nestedClass(Normalizer|null $normalizer): string|null
+    {
+        if ($normalizer instanceof ArrayNormalizer) {
+            $normalizer = $normalizer->innerNormalizer();
+        }
+
+        if (!$normalizer instanceof ObjectNormalizer) {
+            return null;
+        }
+
+        try {
+            return $normalizer->className();
+        } catch (InvalidType) {
+            return null;
+        }
+    }
+}

src/Extension/Generated/ClassNotGeneratable.php

@@ -0,0 +1,18 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Patchlevel\Hydrator\Extension\Generated;
+
+use InvalidArgumentException;
+use Patchlevel\Hydrator\HydratorException;
+
+use function sprintf;
+
+final class ClassNotGeneratable extends InvalidArgumentException implements HydratorException
+{
+    public function __construct(string $class, string $reason)
+    {
+        parent::__construct(sprintf('A transformer for class "%s" can not be generated: %s', $class, $reason));
+    }
+}

src/Extension/Generated/ClassPlan.php

@@ -0,0 +1,47 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Patchlevel\Hydrator\Extension\Generated;
+
+use Patchlevel\Hydrator\Metadata\ClassMetadata;
+
+/**
+ * A class handled by the generated transformer together with the plans of its properties.
+ *
+ * @internal
+ */
+final class ClassPlan
+{
+    /**
+     * @param list<PropertyPlan> $properties
+     * @param bool               $scopedHydrate hydrate code must run in the scope of the class
+     * @param bool               $scopedExtract extract code must run in the scope of the class
+     */
+    public function __construct(
+        public readonly int $index,
+        public readonly ClassMetadata $metadata,
+        public readonly array $properties,
+        public readonly bool $scopedHydrate,
+        public readonly bool $scopedExtract,
+    ) {
+    }
+
+    /** @return class-string */
+    public function className(): string
+    {
+        return $this->metadata->className;
+    }
+
+    /** Classes without normalizers can never be part of a circular reference. */
+    public function leaf(): bool
+    {
+        foreach ($this->properties as $property) {
+            if ($property->kind !== ValueKind::Raw) {
+                return false;
+            }
+        }
+
+        return true;
+    }
+}