Skip to content

fix(test-cli): ignore tests/json_loader and tests/spec_tools during collection - #3461

Merged
danceratopz merged 1 commit into
ethereum:forks/amsterdamfrom
mkzung:fix/check-eip-versions-ignores
Aug 28, 2026
Merged

fix(test-cli): ignore tests/json_loader and tests/spec_tools during collection#3461
danceratopz merged 1 commit into
ethereum:forks/amsterdamfrom
mkzung:fix/check-eip-versions-ignores

Conversation

@mkzung

@mkzung mkzung commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Description

check_eip_versions collects tests/json_loader and tests/spec_tools, whose items carry a fork parameter but no spec type format, and then crashes during collection. It is the same failure #3450 fixed for fill and execute:

INTERNALERROR> ValueError: No spec type format found in the test item.

pytest-fill.ini, pytest-execute.ini and pytest-execute-hive.ini all ignore both directories. pytest-check-eip-versions.ini has never carried an --ignore line and also sets testpaths = tests/, so it is the one config that still reaches them.

Toggling each ignore and reading the collected count puts 938 items in scope, 630 under tests/json_loader and 308 under tests/spec_tools. Nothing is lost by skipping them: is_test_for_an_eip matches eip\d{1,4} against the module path, so neither directory can produce an EIPSpecTestItem. With the two ignores, check_eip_versions --collect-only goes from an abort to 44384/189010 tests collected.

On master the same run then dies a second time in the tests/json_loader teardown with a KeyError, which #3460 guards. The two are independent; this one fixes the command on its own.

Related Issues or PRs

Follows #3450, which made the same fix for the other three configs.

Checklist

  • Ran fast static checks to avoid CI fails, see Code Standards & Verifying Changes: just static
  • PR title has the form <type>(<area>): <title>, where <type> and <area> come from an appropriate C-<type>, respectively A-<area>, label. The title should match the target squash commit message.

Cute Animal Picture

   ,__,
   (oo)
   (__)

@danceratopz danceratopz added C-bug Category: this is a bug, deviation, or other problem A-test-cli Area: execution_testing.cli labels Aug 28, 2026
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.77%. Comparing base (abbe057) to head (f124e47).
⚠️ Report is 1 commits behind head on forks/amsterdam.

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #3461   +/-   ##
================================================
  Coverage            93.77%   93.77%           
================================================
  Files                  624      624           
  Lines                36905    36905           
  Branches              3326     3326           
================================================
  Hits                 34606    34606           
  Misses                1597     1597           
  Partials               702      702           
Flag Coverage Δ
unittests 93.77% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@danceratopz danceratopz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, lgtm!

@danceratopz
danceratopz merged commit d8a126e into ethereum:forks/amsterdam Aug 28, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-test-cli Area: execution_testing.cli C-bug Category: this is a bug, deviation, or other problem

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants