Feat: Dynamic plugin loading via Rust cdylibs

[terylt]

Summary

Adds cpex-dynamic-plugin, a new crate that lets operators load Rust plugins at runtime from .so / .dylib /
.dll files without rebuilding the host. Implements the design from docs/specs/cpex-rust-spec.md §17.

What's in

Single-plugin cdylibs. Plugin authors write a normal Plugin + HookHandler impl, slap the
cpex_dynamic_plugin! macro on it, build as cdylib. Operators load via URL-shaped kind:

plugins:                                                                                                           
  - name: my-plugin                     
    kind: "lib:/opt/plugins/my_plugin.so"                                                                          
    hooks: [cmf.tool_pre_invoke]                                                                                   
    config: { ... } 

Multi-handler cdylibs. One plugin struct can register handlers for multiple hooks. Operators select a specific handler via the URL fragment:

kind: "lib:/opt/plugins/auth.so#identity.resolve"

Multi-plugin cdylibs. Several distinct plugins can ship in one shared library via the cpex_dynamic_plugins! macro. Operators select an entry via the URL query parameter:

kind: "lib:/opt/plugins/multi.so?entry=rate_limiter"

Multi-plugin cdylibs also export an optional cpex_plugin_list discovery symbol; the host reads it to validate ?entry= against available entries up-front and produces friendly diagnostics ("no entry 'foo'. Available: [allow, deny]") instead of raw dlsym errors.

Key design decisions

• Same-version-only Rust ABI. Arc crosses the boundary as the stable vtable type — no serialization of payloads, extensions, or results. The ABI assumes plugin and host were built against the exact same cpex-core version with the same toolchain. A runtime ABI_VERSION handshake catches obvious mismatches loudly. See the "potential extension" section below for the looser-ABI alternative.
• Box::leak on the library handle. Arc vtables point into the cdylib's text section; unloading the library while any Arc is live would SIGSEGV. We keep loaded libraries mapped for the rest of the process — same model as Bevy and most Rust plugin frameworks. No hot reload in this slice.
• URL-shaped kind. :[?entry=][#handler]. The loader's concerns (path, entry, handler filter) live in the kind URL; the plugin's config: block stays purely the plugin's own settings. Scheme dispatch was added to PluginFactoryRegistry so other future schemes (e.g., wasm:) can register independently.
• catch_unwind at the FFI boundary. Plugin panics during construction are caught and reported as EntryPointResult::Panic — unwinding across extern "C" is UB, so we never let it happen.
• Plugin-only builds stay light. The crate gates libloading behind a host feature. Plugin authors depending on this crate get just the ABI types and helper macros; libloading is only pulled when the host opts in.

Test coverage

• 35 unit tests in cpex-dynamic-plugin: ABI dispatch, kind-URL parser, entry-name validation.
• 12 integration tests in tests/dlopen_e2e.rs (gated on --features host): single-plugin happy path, error diagnostics, multi-handler load + filter, multi-plugin ?entry= happy paths, unknown-entry diagnostics, multi-cdylib coexistence, single-plugin backward-compat regression check.

Three reference example cdylibs live alongside the crate at
crates/cpex-dynamic-plugin/examples/{single-plugin,multi-handler,multi-plugin}/.

Out of scope

• Hot reload (would need refcounted library wrapper coordinated with all derived Arcs).
• Sandbox (plugins run in-process with full host privileges; operators vet plugins before deploying).
• WASM plugins (separate scheme entirely).
• A cpex inspect <path.so> CLI to enumerate cpex_plugin_list for ops folks (≈50 LOC; nice-to-have, not blocking).

---

Potential extension: cross-version ABI via abi_stable

The same-version-only constraint is a deliberate v1 choice — simple to reason about, no perf cost, no large dep surface — but it forces operators to rebuild plugins whenever they upgrade the host.

The abi_stable crate offers an escape hatch. It provides:

• Stable wrapper types (RString, RVec, RArc, RBox, ROption) with #[repr(C)] layouts pinned across compiler versions.
• #[sabi_trait] macro that emits trait objects with stable, structurally-described vtables.
• Runtime layout validation — at dlopen, the host fetches the plugin's embedded layout metadata and structurally compares it field-by-field against its own expected layout. Mismatches surface as readable errors ("field handlers
  at offset 24: type mismatch in inner generic") rather than UB.

With abi_stable in place, an operator could in principle run a 1.74 host against a plugin built with 1.78, or pin a plugin to cpex-core 0.2 while the host moves to 0.3, as long as the published stable interface hasn't changed in incompatible ways.

The cost is substantial: every type crossing the boundary (Plugin, AnyHookHandler, MessagePayload, Extensions, PluginContext, PluginConfig, PluginResult, …) needs to become StableAbi, async trait support is ergonomically rougher, and there's a real per-call perf hit from the stable wrappers. The cleanest migration path is a dedicated
cpex-dynamic-plugin-stable adapter layer that mirrors cpex-core's types in R*-wrapped form, with conversion shims threading them into the real types at load time.

When this would be worth doing: if the deployment model evolves toward third-party plugin authors distributing binaries independently of host releases (a marketplace shape, à la Bevy plugins). For internal-only plugins where operators control both the host build and the plugins they load, the same-version constraint is the right tradeoff— one rebuild per host upgrade is cheap compared to maintaining a stable-ABI adapter layer and validating every cross-version combination in CI.

Potential extension: per-version plugin build container

A lower-cost alternative to a stable ABI is to make same-version builds trivially reproducible. Each CPEX release ships a cpex/plugin-builder:vX.Y.Z container image with the exact rustc, cpex-core, cpex-dynamic-plugin, and cargo features pinned. Plugin authors don't need to think about the toolchain at all:

cargo cpex plugin new my-plugin --version v0.5.0   # scaffolds with pinned deps                                    
cargo cpex plugin build                            # docker run cpex/plugin-builder:v0.5.0 cargo build 

The output .so is guaranteed to ABI-match the v0.5.0 host. The runtime ABI_VERSION handshake stays in place as a backstop for builds that escape the container.

Versus abi_stable: the container approach attacks the same operational pain (plugin/host version drift) from a different angle — instead of making the runtime tolerant of mismatches, it makes the build environment trivially reproducible so mismatches don't happen. Zero runtime cost, zero changes to cpex-core types, plugin authors keep writing idiomatic Rust (Arc, Vec, async_trait) with no R*-wrapper ceremony. The tradeoff: you can't deliberately
mix versions (host v0.5 + plugin built for v0.4), which abi_stable would allow. For internal deployments that constraint is a feature; for a third-party plugin marketplace it isn't.

Costs to plan for: one image per release to publish and maintain, multi-arch matrix (linux/amd64, linux/arm64, darwin/arm64), and image size (~1 GB for a Rust toolchain image, mitigable via slim base + cargo-chef caching). A
lower-effort variant skips Docker entirely and just publishes a compatibility manifest (rustc version, cpex-core git SHA, feature flags) that a cargo cpex plugin check subcommand validates against the dev's local toolchain — trades determinism for portability.