Every xarray open pays the backend auto-detection cost

[rhaegar325]

Summary

xr.open_dataset / open_mfdataset without an explicit engine= makes
xarray auto-detect the backend by importing every registered
xarray.backends entry point and asking each one guess_can_open(). The
analysis3 environment registers 37 of them, two of which — argopy's argo
and erddapy — reach out to raw.githubusercontent.com on import. A Gadi
compute node has no route there, so those imports block until they time
out.

The cost is paid once per process, at the first open that does not name
an engine. Measured at 38-46s per batch job, showing up in the phase
that happened to get there first — the Dask worker-sizing probe in
recommend_dask_config.

Change

engine="netcdf4" added at 11 call sites across base.py,
executors/dask_config.py, utilities.py (x4), qc/cmip7.py,
qc/plots.py (x2), ocean_supergrid.py and derivations/calc_utils.py.

This is not a new convention — CMORiser.load_dataset already pinned the
engine in its common_kwargs and in the fx branch; the rest had not been
brought in line.

Every file opened is netCDF, verified rather than assumed:

Source	Format
UM atmosphere output (3600 files sampled)	NETCDF4
MOM ocean output (13380 sampled)	NETCDF4_CLASSIC
CICE ice output (1440 sampled)	NETCDF4_CLASSIC
bundled resources/*.nc	NETCDF4_CLASSIC x3, NETCDF4 x3

Non-.nc files in the archive (namelists, logs, STASH tables, config.yaml)
are never opened — file discovery only matches .nc. All four raw-data
variants were read both ways and compared: identical variables, dimensions,
global attributes and data.

Tests

New tests/unit/test_xarray_engine_pinned.py, both mutation-checked:

• test_every_xarray_open_pins_the_engine walks the package AST and fails
  if any open_dataset/open_mfdataset/open_zarr call omits engine=.
  This is structural on purpose: "pinning some of them" is exactly the
  failure mode arm B measured, so a test of one function would not have
  caught it.
• test_load_dataset_common_kwargs_pins_the_engine closes the **kwargs
  hole the structural check has to allow, by asserting load_dataset's
  common_kwargs dict itself sets engine="netcdf4".

Full suite: 2119 passed (2117 before this change).

[codecov]

Codecov Report

❌ Patch coverage is 88.88889% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 79.0%. Comparing base (9ab0ceb) to head (1a3feee).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/access_moppy/qc/plots.py	0.0%	1 Missing ⚠️

❌ Your patch check has failed because the patch coverage (88.9%) is below the target coverage (90.0%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #695     +/-   ##
=======================================
+ Coverage   78.8%   79.0%   +0.2%     
=======================================
  Files         41      41             
  Lines       9070    9070             
  Branches    1692    1692             
=======================================
+ Hits        7143    7164     +21     
+ Misses      1592    1570     -22     
- Partials     335     336      +1     

Flag	Coverage Δ
unit	79.0% <88.9%> (+0.2%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.