feat(ci): test execute, consume and check_eip_versions via collect-only - #3451
Open
danceratopz wants to merge 4 commits into
Open
feat(ci): test execute, consume and check_eip_versions via collect-only#3451danceratopz wants to merge 4 commits into
execute, consume and check_eip_versions via collect-only#3451danceratopz wants to merge 4 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #3451 +/- ##
================================================
Coverage 94.01% 94.01%
================================================
Files 624 624
Lines 36905 36905
Branches 3326 3326
================================================
Hits 34695 34695
Misses 1533 1533
Partials 677 677
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
danceratopz
marked this pull request as draft
August 26, 2026 08:33
danceratopz
marked this pull request as ready for review
August 26, 2026 08:57
danceratopz
force-pushed
the
ci-execute-collect
branch
from
August 26, 2026 08:57
3ea282d to
8e9e140
Compare
danceratopz
commented
Aug 26, 2026
danceratopz
force-pushed
the
ci-execute-collect
branch
2 times, most recently
from
August 28, 2026 07:37
c3daad7 to
21b481a
Compare
danceratopz
marked this pull request as draft
August 31, 2026 08:10
danceratopz
force-pushed
the
ci-execute-collect
branch
from
September 3, 2026 09:19
21b481a to
e7300c0
Compare
execute, consume and check_eip_versions via collect-only
danceratopz
marked this pull request as ready for review
September 3, 2026 09:22
spencer-tb
approved these changes
Sep 3, 2026
spencer-tb
left a comment
Contributor
There was a problem hiding this comment.
LGTM! One nit/idea :)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add cheap collect-only CI guards for our pytest commands. This would have caught the collection crash behind #3438 (fixed in #3450).
pytest_hive: This also allows--collect-onlywith hive simulators w/o actually having to connect to a hive dev backend.just execute-collect: collect the fulltests/tree via both theexecute remoteandexecute hiveplugin stacks (this reproduces the eels/execute, hive mode internal error #3438 crash without the fix in fix(test-execute): ignore tests/spec_tools during collection #3450). Asserts a collected-count floor so a drastic silent shrink also fails.just consume-collect: fill a mini fixture set (test_dup.py, 16 cases, all formats), then collect it viaconsume engine/rlp/enginex/directwith exact-count assertions: an empty parametrization (e.g. a missing fixture format) collects a single placeholder item and exits 0, so counts must be checked. Notably,consume enginexcurrently has no CI coverage at all: itshive-consume.yamlmatrix rows are commented out.just check-eip-versions-collect: collect the EIP version checks over the fulltests/tree with the same collected-count floor. This is the guard for the failure mode fixed in fix(test-cli): ignore tests/json_loader and tests/spec_tools during collection #3461 (verified: it exits non-zero with fix(test-cli): ignore tests/json_loader and tests/spec_tools during collection #3461's parent commit, passes with the fix).test.yamlon every PR.hive-execute.yaml: also trigger on changes to the shared plugins andpytest-execute.ini, both of which the E2E consumes.Related Issues or PRs
Related to #3438; requires #3450 and #3461 (both merged).
Checklist
just static<type>(<area>): <title>, where<type>and<area>come from an appropriateC-<type>, respectivelyA-<area>, label. The title should match the target squash commit message.Cute Animal Picture