Skip to content

test(evals): dedupe test names across test modules - #247

Merged
lihbr merged 3 commits into
mainfrom
lh/evals
Jul 31, 2026
Merged

test(evals): dedupe test names across test modules#247
lihbr merged 3 commits into
mainfrom
lh/evals

Conversation

@lihbr

@lihbr lihbr commented Jul 31, 2026

Copy link
Copy Markdown
Member

Resolves: n/a

Description

  • Dedupe eval names by keying them to their module ID (relative file path)

Checklist

  • If my changes require tests, I added them.
  • If my changes affect backward compatibility, it has been discussed.
  • If my changes require an update to the CONTRIBUTING.md guide, I updated it.

Preview

n/a

How to QA 1


Note

Low Risk
Test-reporting-only change; committed results.json key names will change on the next full eval run.

Overview
Eval JSON output (results.json / results.local.json) now keys each trial under {relativeModuleId} / {test name} instead of the bare test title.

That stops collisions when multiple *.eval.ts files reuse the same case name (e.g. several supports --help cases), so each module’s results stay separate in the report and in npm run evals:report.

Reviewed by Cursor Bugbot for commit 5beb5a6. Bugbot is set up for automated code reviews on this repo. Configure here.

Footnotes

  1. Please use these labels when submitting a review:
    ❓ #ask: Ask a question.
    💡 #idea: Suggest an idea.
    ⚠️ #issue: Strongly suggest a change.
    🎉 #nice: Share a compliment.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7e0cf21. Configure here.

Comment thread evals/reporter.ts Outdated
@lihbr lihbr changed the title test(evals): skip throwing tests test(evals): dedupe test names across test modules Jul 31, 2026
@lihbr
lihbr merged commit 28d167d into main Jul 31, 2026
60 of 70 checks passed
@lihbr
lihbr deleted the lh/evals branch July 31, 2026 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants