Skip to content

Update JUnit4 instrumentation to fire test suite events for Bazel#11322

Draft
daniel-mohedano wants to merge 3 commits intomasterfrom
daniel.mohedano/bazel-run-notifier-instrumentation
Draft

Update JUnit4 instrumentation to fire test suite events for Bazel#11322
daniel-mohedano wants to merge 3 commits intomasterfrom
daniel.mohedano/bazel-run-notifier-instrumentation

Conversation

@daniel-mohedano
Copy link
Copy Markdown
Contributor

@daniel-mohedano daniel-mohedano commented May 8, 2026

What Does This Do

  • Introduces proper handling for Bazel's JUnit tests runner suite events by:
    • Removing the previous temporary solution of lazy starting suites.
    • Introduces a new instrumentation for fireTestSuiteStarted and fireTestSuiteFinished methods to unwrap the notifiers and propagate the firing to our tracing listener.

Motivation

Bazel's default JUnit4 configuration for running test suites introduces changes that mess with our instrumentation:

  1. Bazel's JUnit entry-point is through BazelTestRunner.main(), which builds a JUnit4Runner (does not inherit org.junit.runner.Runner) and calls run().
  2. JUnit4Runner.run() instantiates JUnitCore, registers its run listeners and calls core.run(cancellableRequest).
  3. JUnitCore creates a fresh RunNotifier (from now on A) and registers the listeners on it.
  4. The request is run with a custom CancellableRunner (extends org.junit.runner.Runner). Its run(A) does currentNotifier = new ThreadSafeRunNotifier(A); delegate.run(currentNotifier);.
  5. ThreadSafeRunNotifier extends RunNotifierWrapper extends RunNotifier, so currentNotifier (from now on B) has its own private inherited listeners field.
  6. The real runner.run(B) executes.
  7. Our instrumentation modifies the call to do B.addListener(tracingListener), but Bs implementation overrides the method to call delegate.addListener(tracingListener); so our listener lands on A.listeners.
  8. The real runner fires events such as test suite starts with B.fireTestSuiteStarted(...).
  9. The wrapper does not override test suite methods, so it is resolved as RunNotifier.fireTestSuiteStarted(...) invoked with this = B. This iterates over B.listeners, which is empty.
  10. The wrapper does forward regular test events with delegate.fireTest...(...) by overriding the methods

All this caused test suite events to be dropped.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@daniel-mohedano daniel-mohedano added type: bug Bug report and fix comp: ci visibility Continuous Integration Visibility labels May 8, 2026
@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - sbt-scalatest

Job Status: success

Scenario Overhead (%)
agent 56.98
agentEvpProxy 56.37

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - nebula-release-plugin

Job Status: success

Scenario Overhead (%)
agent 37.94
agentless 36.40
agentlessCodeCoverage 43.61
agentlessLineCoverage 76.60

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - netflix-zuul

Job Status: success

Scenario Overhead (%)
agent 88.39
agentless 81.20
agentlessCodeCoverage 95.83
agentlessLineCoverage 112.65

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - pass4s

Job Status: success

Scenario Overhead (%)
agent 14.32
agentless 11.41
agentlessCodeCoverage 22.27

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - reactive-streams-jvm

Job Status: success

Scenario Overhead (%)
agent 21.87
agentless 18.54
agentlessCodeCoverage 19.48
agentlessLineCoverage 29.68

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - sonar-kotlin

Job Status: success

Scenario Overhead (%)
agent 13.86
agentless 12.42
agentlessCodeCoverage 15.76
agentlessLineCoverage 18.84

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - jolokia

Job Status: success

Scenario Overhead (%)
agent 94.94
agentless 90.16
agentlessCodeCoverage 97.43
agentlessLineCoverage 99.99

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - okhttp

Job Status: success

Scenario Overhead (%)
agent 20.51
agentless 19.78
agentlessCodeCoverage 23.38
agentlessLineCoverage 45.82

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - spring_boot

Job Status: success

Scenario Overhead (%)
agent 16.25
agentless 9.68
agentlessCodeCoverage 12.93
agentlessLineCoverage 33.86

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 8, 2026

Test Environment - sonar-java

Job Status: success

Scenario Overhead (%)
agent 13.06
agentless 14.07
agentlessCodeCoverage 76.97
agentlessLineCoverage 176.85

@daniel-mohedano daniel-mohedano changed the title fix: proper handling of bazel test runner suite events Update JUnit4 instrumentation to fire test suite events for Bazel May 11, 2026
@daniel-mohedano daniel-mohedano added type: enhancement Enhancements and improvements and removed type: bug Bug report and fix labels May 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility type: enhancement Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant