perf(perf-map-agent): index the git root once instead of walking it per source lookup

[not-matthias]

Problem

Walltime flamegraphs for JVM benchmarks spend a large share of samples in:

start_thread -> Thread::call_run -> JavaThread::thread_main_inner
  -> ServiceThread::service_thread_entry
    -> JvmtiExport::post_compiled_method_load
      -> on_compiled_method_load
        -> resolve_source_file
          -> find_file_recursive (recursing)

These are not leftover warmup frames. resolve_source_file in the perf-map JVMTI agent walked the whole git root recursively for every distinct class-relative source path. The result was cached per path, but a miss — every JDK and dependency class, which can never be on disk under the repo — paid a full tree walk first.

CompiledMethodLoad keeps firing during the measured phase (tier transitions, late compiles, deopt/recompile), and it is delivered on the JVM service thread concurrently with the benchmark. Walltime sampling captures all threads, so those walks land in the flamegraph.

Measured with a JIT-heavy workload (~3.5 s) in a ~6500-file checkout, strace -f -c:

	no agent	before	after
getdents64	4	129,180	968
newfstatat	279	448,644	286
openat	2,510	67,101	2,995
syscall time	0.007 s	2.29 s	0.016 s
Service Thread CPU	0 s	1.03 s	0.03 s

132 distinct source paths were requested, all of them misses in that run — i.e. 132 full tree walks.

Change

Walk the git root exactly once, on the first lookup, into a basename-keyed hash index (4096 chained buckets, FNV-1a). Each resolution is then one bucket scan for a path ending in /<relative_path>.

• Same lookup semantics, same skip rules (.-prefixed, build, target, node_modules), same perf-map output format.
• The walk uses d_type instead of a stat() per entry, falling back to stat() only for symlinks and DT_UNKNOWN (so symlinks are still followed as before).
• The index lives for the process lifetime and is freed in Agent_OnUnload, alongside the existing per-path cache.

Verification

• ./gradlew :jmh-fork:jmh-core:native-perf-map-agent:linkRelease — clean, no warnings.
• ./gradlew :jmh-fork:jmh-core:test --tests "io.codspeed.*" — 22 tests, 0 skipped, 0 failures. PerfMapAgentTest runs the real agent in a child JVM and asserts both the resolved <abs path>.java::<class>.<method> form and the no-source fallback form.
• A/B numbers above come from re-running the same measurement script against the pre- and post-change libperf_map_agent.so; perf map line count and the set of distinct source paths are unchanged.

Closes COD-3562.

[greptile-apps]

Confidence Score: 4/5

The PR should not merge until duplicate source-path resolution preserves the previous first-discovered selection behavior.

Findings

1.  Duplicate paths resolve differently ▶

Fix with agent prompt

### Issue 1
jmh-fork/jmh-core/native-perf-map-agent/src/main/c/perf_map_agent.c:175-176
If a multi-module checkout has the same package-relative file in two source roots, such as `com/example/Foo.java`, prepending each entry reverses their traversal order. `index_find` therefore returns the last discovered match, while the previous recursive search returned the first. The perf map can consequently associate a compiled method with a different source file.

```suggestion
  e->next = NULL;
  index_entry_t **tail = &source_index[bucket];
  while (*tail) {
    tail = &(*tail)->next;
  }
  *tail = e;
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR replaces repeated recursive source-file searches with a process-lifetime, basename-keyed index built during the first source lookup.

• Walks the enclosing Git repository once while preserving existing directory exclusions.
• Uses dirent.d_type to avoid most per-entry stat() calls.
• Caches source resolutions and releases the index during agent unload.
• The new head-inserted bucket chains reverse duplicate-path selection relative to the previous traversal.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[First source lookup] --> B[Find Git root]
  B --> C[Walk repository once]
  C --> D[Hash regular files by basename]
  D --> E[Source index]
  F[CompiledMethodLoad] --> G[Build class-relative source path]
  G --> H{Resolution cache hit?}
  H -->|Yes| I[Emit perf-map symbol]
  H -->|No| J[Hash source basename]
  J --> K[Scan matching bucket]
  K --> L[Cache resolved path or miss]
  L --> I

Loading

_{Reviews (1) · Last reviewed commit: "perf(perf-map-agent): index the git root..."}

[greptile-apps]

Duplicate paths resolve differently

If a multi-module checkout has the same package-relative file in two source roots, such as com/example/Foo.java, prepending each entry reverses their traversal order. index_find therefore returns the last discovered match, while the previous recursive search returned the first. The perf map can consequently associate a compiled method with a different source file.

Suggested change

	e->next = source_index[bucket];
	source_index[bucket] = e;
	e->next = NULL;
	index_entry_t **tail = &source_index[bucket];
	while (*tail) {
	tail = &(*tail)->next;
	}
	*tail = e;

Prompt To Fix With AI

This is a comment left during a code review.
Path: jmh-fork/jmh-core/native-perf-map-agent/src/main/c/perf_map_agent.c
Line: 175-176

Comment:
**Duplicate paths resolve differently**

If a multi-module checkout has the same package-relative file in two source roots, such as `com/example/Foo.java`, prepending each entry reverses their traversal order. `index_find` therefore returns the last discovered match, while the previous recursive search returned the first. The perf map can consequently associate a compiled method with a different source file.

```suggestion
  e->next = NULL;
  index_entry_t **tail = &source_index[bucket];
  while (*tail) {
    tail = &(*tail)->next;
  }
  *tail = e;
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[codspeed]

Hooray! CodSpeed harness just leveled up!

The base and head of this comparison were measured with different runner settings, so their benchmark values are not directly comparable.

What changed between base and head:

• CodSpeed runner v5 changed how benchmarks are measured (base 4.17.1 → head 5.3.1). View release notes

Re-run the base with the same settings to get a valid performance comparison.

---

_{Comparing cod-3562-investigate-servicethreads-find_file_recursive-frames (2340073) with main (6333555)}