Skip to content

feat(eval): make regex support explicit - #656

Merged
nahime0 merged 4 commits into
mainfrom
feat/eval-regex-capability
Aug 4, 2026
Merged

feat(eval): make regex support explicit#656
nahime0 merged 4 commits into
mainfrom
feat/eval-regex-capability

Conversation

@nahime0

@nahime0 nahime0 commented Jul 30, 2026

Copy link
Copy Markdown
Member

Summary

  • Decouple dynamic eval() support from PCRE2 so Magician-only binaries no longer require a managed native project or link regex code.
  • Add --with-regex for regex calls that exist only inside opaque evaluated source, while keeping normal automatic detection for statically visible regex usage.
  • Register managed PCRE2 with Magician through a versioned provider ABI only when regex support is enabled. Without a provider, evaluated code does not expose preg_* or mb_ereg_match() and fails only if one of those unavailable functions is executed.
  • Print a post-build reminder when dynamic eval is compiled without regex support, including the commands needed to declare PCRE2 and enable the capability.
  • Move regex usage out of the general examples/eval/ program into a dedicated examples/eval_regex/ project with a managed PCRE2 manifest and lockfile.
  • Restore the Magician benchmark runner to dependency-free isolated fixtures, removing the PCRE2 project staging that eval-only benchmarks no longer need.

Behavior

Program shape Result
Dynamic eval without regex Compiles without a native project or PCRE2, links Magician only, and emits a reminder after a successful build.
Statically visible regex usage Keeps existing auto-detection, links managed PCRE2, and makes the provider available to dynamic eval too.
Regex used only inside opaque eval source Requires a declared pcre2 native dependency and an explicit --with-regex build.

Testing

  • cargo test --bin elephc with_regex_records_runtime_capability
  • cargo test --lib runtime_features::tests::test_eval
  • cargo test --test codegen_tests test_dynamic_eval_without_regex_needs_no_native_project
  • cargo test --test codegen_tests test_dynamic_eval_regex_without_capability_fails_at_runtime
  • cargo test --test codegen_tests test_dynamic_eval_with_regex_uses_managed_provider
  • cargo test --test codegen_tests test_static_regex_detection_enables_dynamic_eval_regex

@github-actions github-actions Bot added area:codegen Touches target-aware assembly or backend lowering. area:magician Touches eval, include execution, or elephc-magician. area:tooling-ci Touches CI, development tooling, Docker, or repository scripts. scope:multi-area Touches more compiler areas than the automatic area-label cap. size:l Large pull request. type:feature Introduces new user-visible behavior or capabilities. labels Jul 30, 2026
@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown

Greptile Summary

This PR decouples dynamic eval() regex support from PCRE2 so Magician-only binaries no longer require a managed native project or link regex code. A versioned provider ABI (elephc_pcre2_v1_*) is registered via __elephc_eval_register_regex_provider only when regex is explicitly required, and --with-regex is introduced as a new non-bridge CLI capability alongside the existing bridge --with-<crate> flags.

  • New regex_provider module in elephc-magician: introduces a OnceLock<RegexProvider> for process-wide idempotent provider registration; production builds use the managed shim callbacks, unit-test builds use a host-PCRE2 adapter wired in via #[cfg(test)].
  • build.rs link-directive switch: changes from cargo:rustc-link-lib to cargo:rustc-link-arg so PCRE2 link requirements stay private to this package's own test binaries and do not propagate into downstream consumers of the elephc-magician rlib/staticlib.
  • pcre2_shim.c recipe bump (revision 1 → 2): adds REG_STARTEND offset pre-reading before the output array is zeroed, fixing the search-within-a-byte-range path needed by exec_at; a new check_startend_offsets C-level test verifies the behavior.

Confidence Score: 5/5

Safe to merge — the change correctly decouples Magician from PCRE2 with well-tested boundaries, the provider ABI is versioned, and the recipe revision bump invalidates stale shim builds as intended.

The architectural change is sound: OnceLock registration is thread-safe and idempotent, build.rs correctly switches to cargo:rustc-link-arg so PCRE2 stays out of downstream consumers, the shim REG_STARTEND fix is validated by a new C-level test, and every changed dispatch path has corresponding integration tests. No correctness bugs or unsafe invariant violations were found.

Files Needing Attention: No files require special attention. The most sensitive paths — regex_provider.rs, engine.rs, and pcre2_shim.c — all have explicit guard checks and new test coverage.

Important Files Changed

Filename Overview
crates/elephc-magician/src/regex_provider.rs New module — OnceLock for versioned provider ABI registration; #[cfg(test)] host-PCRE2 adapter correctly isolated from the staticlib; all unsafe blocks have null-guard checks.
crates/elephc-magician/src/interpreter/builtins/regex/engine.rs Engine rewritten to use opaque provider callbacks; offsets() preserves the semantics of the old to_offsets() (negative → None); capture_slots == 0 compile error guards against zero-length slot vectors.
crates/elephc-magician/build.rs Switches from cargo:rustc-link-lib to cargo:rustc-link-arg so PCRE2 link requirements remain private to this package's test binaries and do not propagate to downstream consumers.
src/native_deps/recipes/pcre2_shim.c Recipe revision 2: reads REG_STARTEND offset pair before zeroing the output array, with bounds validation; new shim test check_startend_offsets verifies absolute-offset preservation.
src/codegen/lower_inst/builtins/eval.rs Adds register_eval_regex_provider emitted inside the lazy eval-context init guard; registration is idempotent via OnceLock, correctly placed before eval context init.
src/cli.rs Adds --with-regex as a known non-bridge capability through RUNTIME_CAPABILITY_FLAGS; unknown flag rejection is preserved; new test with_regex_records_runtime_capability verifies the flag is stored.
src/pipeline.rs Sets runtime_features.regex from --with-regex after IR optimization; dynamic_eval_capability_warning emits a post-build reminder when eval is compiled without regex.
src/codegen_support/runtime_features.rs Decouples pcre2 native package requirement from eval_bridge; PCRE2 is now only required when regex is true; new test verifies eval-alone produces only the magician bridge requirement.
src/native_deps/recipe.rs Introduces BuiltInRecipe enum and built_in_recipe() dispatcher; adds tests that verify the current catalog revision has a dispatcher and that the old revision 1 is rejected.
crates/elephc-magician/src/interpreter/builtins/registry/mod.rs Adds regex-provider availability as a second filter in eval_declared_builtin_spec; builtin_is_available is a pure function testable without global state, with a new unit test verifying the filter logic.
tests/codegen/eval.rs Adds four new integration tests covering: eval without native project, runtime failure for missing regex capability, --with-regex managed provider path, and static-regex auto-detection propagation.
tests/eval_string_interpolation_tests.rs Removes PCRE2 provisioning and shared cache root; replaces with per-test isolated XDG_CACHE_HOME; elephc_diagnostics filter updated to explicitly exclude the new eval capability reminder.

Sequence Diagram

sequenceDiagram
    participant CLI as elephc CLI
    participant Pipeline as pipeline.rs
    participant Codegen as eval.rs (codegen)
    participant Binary as Generated Binary
    participant Shim as elephc_pcre2_v1_*
    participant Magician as __elephc_eval_*

    CLI->>Pipeline: --with-regex flag
    Pipeline->>Pipeline: "ir_module.required_runtime_features.regex = true"
    Pipeline->>Pipeline: link_requirements: NativePackage(pcre2) + Bridge(elephc_magician)
    Pipeline->>Codegen: "generate() with regex=true"

    Note over Codegen: ensure_eval_context (lazy init)
    Codegen->>Binary: emit: load elephc_pcre2_v1_compile ptr to arg0
    Codegen->>Binary: emit: load elephc_pcre2_v1_exec ptr to arg1
    Codegen->>Binary: emit: load elephc_pcre2_v1_free ptr to arg2
    Codegen->>Binary: emit: call __elephc_eval_register_regex_provider

    Binary->>Magician: __elephc_eval_register_regex_provider(compile, exec, free)
    Magician->>Magician: REGEX_PROVIDER.set(RegexProvider) [OnceLock, idempotent]

    Binary->>Magician: eval(preg_match(...))
    Magician->>Magician: "eval_declared_builtin_spec(preg_match) -> regex_provider_available() = true"
    Magician->>Shim: (provider.compile)(pattern, flags)
    Shim-->>Magician: opaque handle
    Magician->>Shim: (provider.exec)(handle, subject, slots, offsets, REG_STARTEND)
    Shim-->>Magician: offset pairs
    Magician-->>Binary: captures / match result
Loading

Reviews (5): Last reviewed commit: "fix(tests): align eval interpolation wit..." | Re-trigger Greptile

@nahime0
nahime0 requested a review from Guikingone July 31, 2026 07:10
@nahime0 nahime0 moved this from Backlog to In review in Elephc Release Track Aug 4, 2026
@nahime0
nahime0 force-pushed the feat/eval-regex-capability branch from e83f60f to 30d375c Compare August 4, 2026 15:12
@nahime0
nahime0 merged commit 968e0aa into main Aug 4, 2026
117 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in Elephc Release Track Aug 4, 2026
@nahime0
nahime0 deleted the feat/eval-regex-capability branch August 4, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:codegen Touches target-aware assembly or backend lowering. area:magician Touches eval, include execution, or elephc-magician. area:tooling-ci Touches CI, development tooling, Docker, or repository scripts. scope:multi-area Touches more compiler areas than the automatic area-label cap. size:l Large pull request. type:feature Introduces new user-visible behavior or capabilities.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants