Skip to content

chore(test-specs): avoid silent mistakes when writing tests - #3510

Merged
fselmo merged 11 commits into
ethereum:forks/amsterdamfrom
fselmo:chore/avoid-silent-mistakes-test-writing
Sep 2, 2026
Merged

chore(test-specs): avoid silent mistakes when writing tests#3510
fselmo merged 11 commits into
ethereum:forks/amsterdamfrom
fselmo:chore/avoid-silent-mistakes-test-writing

Conversation

@fselmo

@fselmo fselmo commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Description

On a review of #3486, I asked one of the Claude agents to sweep the related framework for gotchas that could lead to silent fill for the wrong reasons. It came back with some hits but fell our of scope for that PR, so I'm adding these here with unit tests. This is also important for DevEx but it's mostly important as a security so we don't end up with footguns that can fill tests but end up with unintended JSON fixtures because of silent overrides of certain things the test itself didn't mean to create.

This shouldn't change anything related to the current state of the tests but I did run hasher compare on a fill of eip7928 module at the current forks/amsterdam and on this branch, just as a sanity check, and it's exact.

Checklist

  • Ran fast static checks to avoid CI fails, see Code Standards & Verifying Changes: just static
  • PR title has the form <type>(<area>): <title>, where <type> and <area> come from an appropriate C-<type>, respectively A-<area>, label. The title should match the target squash commit message.

Cute Animal Picture

Screenshot 2026-09-02 at 13 39 58

@fselmo fselmo added C-chore Category: chore A-test-specs Area: execution_testing.specs labels Sep 2, 2026
@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.01%. Comparing base (0d9d919) to head (5e8b801).
⚠️ Report is 1 commits behind head on forks/amsterdam.

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #3510   +/-   ##
================================================
  Coverage            94.01%   94.01%           
================================================
  Files                  624      624           
  Lines                36905    36905           
  Branches              3326     3326           
================================================
  Hits                 34695    34695           
  Misses                1533     1533           
  Partials               677      677           
Flag Coverage Δ
unittests 94.01% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@marioevz marioevz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome fixes! Just had one request. Approving for now so we can merge once implemented.

Comment thread packages/testing/src/execution_testing/test_types/block_access_list/t8n.py Outdated
@fselmo
fselmo merged commit fc701a4 into ethereum:forks/amsterdam Sep 2, 2026
19 checks passed
@fselmo
fselmo deleted the chore/avoid-silent-mistakes-test-writing branch September 2, 2026 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-test-specs Area: execution_testing.specs C-chore Category: chore

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants