Repository navigation
Ability to only run failures (and/or to run them first) #172
Description
Activity
a problem with running only previously-failed tests, is that fixing these doesn't mean all tests now pass, since other tests may have begun to fail... so i'm tempted to think running previously-failed tests first is the more useful option, especially if also aborting the test run on the first failure (#132)
Reacted by Tomáš Drvoštěpnote: should actually distinguished between failed, passes, and not yet seen
pytest has these options:
--lf, --last-failed rerun only the tests that failed at the last run (or all if none failed) --ff, --failed-first run all tests, but run the last failures first. This may re-order tests and thus lead to repeated fixture setup/teardown. --nf, --new-first run tests from new files first, then the rest of the tests sorted by file mtimehttps://docs.pytest.org/en/7.1.x/reference/reference.html#id89
see also: https://docs.pytest.org/en/stable/how-to/cache.html
AFAICT go test doesn't have this option
To me it looks like the only "ordering of tests" option there is
-shuffle-shuffle off,on,N Randomize the execution order of tests and benchmarks. It is off by default. If -shuffle is set to on, then it will seed the randomizer using the system clock. If -shuffle is set to an integer N, then N will be used as the seed value. In both cases, the seed will be reported for reproducibility.Likewise Rust's cargo test doesn't have such an option AFAICT
https://doc.rust-lang.org/cargo/commands/cargo-test.htmlRuby's rspec has
--only-failures(and--next-failure)
https://rspec.info/features/3-13/rspec-core/command-line/only-failures/The `--only-failures` option filters what examples are run so that only those that failed the last time they ran are executed. There's also a `--next-failure` option, which is shorthand for `--only-failures --fail-fast --order defined`. It allows you to repeatedly focus on just one of the currently failing examples, then move on to the next failure, etc.Javascript's jest has this via
--onlyFailuresRun tests that failed in the previous execution.
name tbd (perhaps
fails_only,fails_first?)potentially would be nice to add a short-hand for combining this with
failfast#132