python/tskit/util.py: Add a file name to FileFormatError

[alxsimon]

Closes #2467
Uses the .add_note method of exception, introduced in Python 3.11. Also hardcodes the file name in
the raised from exceptions for HDF5 or zip.
Until 3.10 is EOF, needs the sys module.

Description

Would fix #2467

I'll tidy up (changelog and linting) if you think this change is ok.

PR Checklist:

• Tests that fully cover new/changed functionality.
• Documentation including tutorial content if appropriate.
• Changelogs, if there are API changes.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 74.71%. Comparing base (49eff38) to head (99921bf).

❗ There is a different number of reports uploaded between BASE (49eff38) and HEAD (99921bf). Click for more details.

HEAD has 15 uploads less than BASE

Flag	BASE (49eff38)	HEAD (99921bf)
python-tests-no-jit	2	1
c-python	2	1
C	2	1
python-tests	12	0

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #3366       +/-   ##
===========================================
- Coverage   91.68%   74.71%   -16.97%     
===========================================
  Files          38       38               
  Lines       32241    32242        +1     
  Branches     5155     5155               
===========================================
- Hits        29559    24090     -5469     
- Misses       2346     7800     +5454     
- Partials      336      352       +16     

Flag	Coverage Δ
C	82.26% <ø> (ø)
c-python	77.52% <ø> (ø)
python-tests	?
python-tests-no-jit	33.29% <0.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
Python API	31.39% <0.00%> (-67.34%)	⬇️
Python C interface	91.25% <ø> (ø)
C library	91.22% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jeromekelleher]

LGTM. Can we add a unit test please, checking that the filename goes into the exception as expected, and validating what happens when we use something that doesn't have a name? (Although, it's not obvious to me how you could have a seekable stream that doesn't have a name, but we need to do due diligence here.)

You can pytest.skip for Python < 310.

@benjeffery shall we just ditch Python 3.10 altogether?

[alxsimon]

LGTM. Can we add a unit test please, checking that the filename goes into the exception as expected, and validating what happens when we use something that doesn't have a name? (Although, it's not obvious to me how you could have a seekable stream that doesn't have a name, but we need to do due diligence here.)

You can pytest.skip for Python < 310.

@benjeffery shall we just ditch Python 3.10 altogether?

Thanks for taking a look, I've trying modifying the existing tests in test_utils.py.
I'm not sure if you'd prefer some dedicated tests that don't touch the existing ones? Especially if we consider keeping some skip for python 3.11 (which I guess would skip each test with the modification for this version of Python).

I could use some pointers on how to test the stream side of things, where should it go and how do you generally do this?

[jeromekelleher]

I think we're going to drop 3.10 ASAP, so let's not worry about the failures on 3.10 CI (and we can remove the conditional at the top). The tests you have look good, updating the existing ones here is fine (although may not need to check the filename on every downstream check for FileFormatError (as that's not what most of the tests are about).

Have a look at tests/test_fileobj.py for what happens in the stream case and see what kind of error messages we get.

[alxsimon]

Thanks, reduced the number of tests of FileFormatError.

In the TestBadStream, the error_text for matching does not seem to be used as modifying it to something that shouldn't be in the error does not fail the test.

For instance:

    def test_bad_stream(self, tmp_path):
        fifo_path = tmp_path / "fifo"
        bad_file_path = tmp_path / "bad_file"
        bad_file_path.write_bytes(b"bad data")
        write_and_read_from_fifo(
            fifo_path, bad_file_path, tskit.FileFormatError, "foo"
        )

Still passes

[jeromekelleher]

I've had a look at this and unfortunately it's really not worth bothering with until we drop Python 3.10 (which will be later this year when it's EOL'd by Python), as it's hard to emulate add_note and we'd have to add a whole bunch of conditionals in the testing code. I'll add a note to the issue for dropping 3.10 to pick this PR up again.

[alxsimon]

thanks for investigating

[hyanwong]

Needs the regex in the test suite to escape windows filenames that often contain backslashes, e.g.

with pytest.raises(
    ...,
    match=f"{filename}...",

should be

with pytest.raises(
    ...,
    match=f"{re.escape(filename)}...",

[alxsimon]

@hyanwong Thanks I've just fixed it for the windows tests.
But I don't know how to fix the codecov.