What version of Parcels are you running?
v4
Is your feature request related to a problem?
The full set of unit tests in v4 runs considerably faster than in v3, but there are still some relatively slow unit tests. Much can be gained in terms of development workflow if these are sped up
Describe the solution you'd like
Speeding up of some unit tests, for example by running for shorter time (while still maintaining accuracy)
Describe alternatives you've considered
No response
Additional context
Below is the output of pytest tests/ --durations=0 on my machine (at ab0adbc). Note that the longest-duration unit tests (test_mitgcm()) takes more than 30% of the full 98s unit test!
====================================================================== slowest durations ======================================================================
32.29s call tests/test_advection.py::test_mitgcm
6.00s call tests/test_advection.py::test_nemo_curvilinear_fieldset
4.37s call tests/test_particleset_execute.py::test_particleset_run_RK_to_endtime_fwd_bwd[dt1-AdvectionRK45]
4.30s call tests/test_particleset_execute.py::test_particleset_run_RK_to_endtime_fwd_bwd[dt0-AdvectionRK45]
4.19s call tests/test_advection.py::test_nemo_3D_curvilinear_fieldset[AdvectionRK4_3D]
3.52s call tests/test_advection.py::test_nemo_3D_curvilinear_fieldset[AdvectionRK4]
3.26s call tests/sgrid/test_sgrid.py::test_parse_sgrid_3d
3.12s call tests/test_index_search.py::test_indexing_nemo_curvilinear
2.95s call tests/test_particleset_execute.py::test_particleset_run_RK_to_endtime_fwd_bwd[dt1-AdvectionRK4]
2.31s call tests/test_sigmagrids.py::test_advection_3DCROCO
1.57s call tests/test_particleset_execute.py::test_particleset_run_RK_to_endtime_fwd_bwd[dt1-AdvectionRK2]
1.28s call tests/test_uxarray_fieldset.py::test_fesom2_square_delaunay_antimeridian_eval
1.26s call tests/test_advection.py::test_peninsula_fieldset[A-AdvectionRK45-0.001]
1.13s call tests/sgrid/test_accessor.py::test_metadata
1.10s call tests/sgrid/test_accessor.py::test_isel_p3_specification_symmetry
1.01s call tests/sgrid/test_accessor.py::test_assert_metadata_ds_consistency_dropped_dim
0.91s call tests/test_particleset_execute.py::test_uxstommelgyre_multiparticle_pset_execute
0.87s call tests/sgrid/test_accessor.py::test_assert_metadata_ds_consistency_failures
0.81s call tests/test_particleset_execute.py::test_particleset_run_RK_to_endtime_fwd_bwd[dt1-AdvectionEE]
0.81s call tests/test_diffusion.py::test_randomvonmises[4-2.5132741228718345]
0.80s call tests/test_diffusion.py::test_randomvonmises[4-3.141592653589793]
0.80s call tests/test_particleset_execute.py::test_uxstommelgyre_pset_execute
0.71s call tests/sgrid/test_accessor.py::test_isel_p2_data_correctness
0.67s call tests/test_index_search.py::test_grid_indexing_fpoints
0.61s call tests/sgrid/test_accessor.py::test_isel_p1_consistency_invariant
0.51s call tests/sgrid/test_accessor.py::test_assert_metadata_ds_consistency
0.51s call tests/test_xgrid.py::test_xgrid_search_cpoints[2D lon/lat]
0.36s call tests/test_diffusion.py::test_randomvonmises[2-2.5132741228718345]
0.35s call tests/test_diffusion.py::test_randomvonmises[2-3.141592653589793]
0.30s call tests/test_particleset_execute.py::test_particleset_run_RK_to_endtime_fwd_bwd[dt0-AdvectionRK4]
0.30s call tests/sgrid/test_sgrid.py::test_parse_sgrid_2d
0.29s call tests/test_advection.py::test_decaying_moving_eddy[AdvectionRK45-0.0001]
0.27s call tests/test_advection.py::test_stommelgyre_fieldset[C-AdvectionRK45-0.1]
0.26s call tests/test_particleset_execute.py::test_particleset_run_to_endtime
0.24s call tests/sgrid/test_sgrid.py::test_Grid3DMetadata_roundtrip
0.23s call tests/test_advection.py::test_stommelgyre_fieldset[A-AdvectionRK4-0.1]
0.23s call tests/sgrid/test_sgrid.py::test_parse_grid_attrs
0.21s call tests/test_uxadvection.py::test_ux_constant_flow_face_centered_2D[AdvectionRK4]
0.21s call tests/test_particleset_execute.py::test_uxgrid_particle_leaving_domain_raises
0.21s call tests/test_uxadvection.py::test_ux_constant_flow_face_centered_2D[AdvectionRK2]
0.20s call tests/test_uxadvection.py::test_ux_constant_flow_face_centered_2D[AdvectionEE]
0.20s call tests/test_advection.py::test_stommelgyre_fieldset[A-AdvectionRK45-0.1]
0.20s call tests/test_field.py::test_field_unstructured_z_linear
0.20s call tests/test_uxarray_fieldset.py::test_icon_evals
0.19s call tests/test_advection.py::test_stommelgyre_fieldset[C-AdvectionRK4-0.1]
0.19s call tests/test_sigmagrids.py::test_conversion_3DCROCO
0.19s call tests/test_uxarray_fieldset.py::test_fesom2_square_delaunay_uniform_z_coordinate_eval
0.18s call tests/test_advection.py::test_radialrotation
0.18s call tests/sgrid/test_sgrid.py::test_Grid2DMetadata_roundtrip
0.18s call tests/test_field.py::test_field_constant_in_time
0.17s call tests/utils/test_time.py::test_time_interval_contains
0.17s call tests/datasets/test_strategies.py::test_sgrid_dataset_field_dims_are_valid
0.17s call tests/test_xgrid.py::test_xgrid_search_cpoints[1D lon/lat]
0.16s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros_none
0.16s call tests/test_particleset_execute.py::test_particleset_run_RK_to_endtime_fwd_bwd[dt0-AdvectionRK2]
0.16s call tests/test_advection.py::test_stommelgyre_fieldset[A-AdvectionRK2-0.1]
0.16s call tests/sgrid/test_sgrid.py::test_edge_node_mapping_metadata_roundtrip
0.15s call tests/test_advection.py::test_advection_zonal_with_particlefile
0.15s call tests/datasets/test_strategies.py::test_sgrid_dataset_node_coordinates_present
0.15s call tests/test_convert.py::test_convert_fesom_to_ugrid
0.14s call tests/test_diffusion.py::test_fieldKh_SpatiallyVaryingDiffusion[AdvectionDiffusionEM-spherical]
0.14s call tests/test_diffusion.py::test_fieldKh_SpatiallyVaryingDiffusion[AdvectionDiffusionEM-flat]
0.14s call tests/test_advection.py::test_peninsula_fieldset[A-AdvectionRK4-0.005]
0.14s call tests/test_diffusion.py::test_fieldKh_SpatiallyVaryingDiffusion[AdvectionDiffusionM1-spherical]
0.13s call tests/test_diffusion.py::test_fieldKh_SpatiallyVaryingDiffusion[AdvectionDiffusionM1-flat]
0.13s call tests/test_advection.py::test_stommelgyre_fieldset[C-AdvectionRK2-0.1]
0.12s call tests/datasets/test_strategies.py::test_sgrid_dataset_coordinate_shapes
0.12s call tests/datasets/test_strategies.py::test_sgrid_dataset_has_grid_topology
0.11s call tests/datasets/test_strategies.py::test_sgrid_dataset_no_nan_in_fields
0.11s call tests/datasets/test_strategies.py::test_sgrid_dataset_has_at_least_one_field
0.10s call tests/test_advection.py::test_decaying_moving_eddy[AdvectionRK4-1e-05]
0.10s call tests/test_spatialhash.py::test_spatialhash_init
0.09s call tests/test_particlefile.py::test_pset_execute_outputdt_backwards_fieldset_timevarying
0.09s call tests/test_spatialhash.py::test_invalid_positions
0.09s call tests/datasets/test_strategies.py::test_sgrid_dataset_returns_dataset
0.09s call tests/test_basegrid.py::test_basegrid_ravel_unravel_index[grid0]
0.09s call tests/test_particleset_execute.py::test_particleset_run_RK_to_endtime_fwd_bwd[dt0-AdvectionEE]
0.09s call tests/test_spatialhash.py::test_mixed_positions
0.08s call tests/test_advection.py::test_peninsula_fieldset[A-AdvectionRK2-0.02]
0.08s call tests/utils/test_time.py::test_time_interval_intersection_commutative
0.07s call tests/test_advection.py::test_advection_3D_outofbounds[True-up]
0.07s call tests/test_advection.py::test_advection_zonal[spherical]
0.07s call tests/test_particlefile.py::test_pset_execute_outputdt_backwards
0.06s call tests/utils/test_time.py::test_time_interval_intersection_with_self
0.06s call tests/utils/test_time.py::test_time_interval_equality
0.06s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[None-ds_2d_left]
0.06s call tests/test_advection.py::test_decaying_moving_eddy[AdvectionRK2-0.003]
0.06s call tests/test_particlefile.py::test_time_is_age[outputdt0]
0.06s call tests/utils/test_time.py::test_datetime_get_cf_attrs
0.05s call tests/test_advection.py::test_advection_zonal_periodic
0.05s setup tests/test_uxarray_fieldset.py::test_fesom_fieldset
0.05s call tests/test_advection.py::test_advection_meridional[spherical]
0.05s call tests/sgrid/test_sgrid.py::test_grid2Dmetadata_standard_names
0.05s call tests/test_advection.py::test_moving_eddy[AdvectionRK45-0.0001]
0.05s setup tests/test_uxarray_fieldset.py::test_set_interp_methods
0.05s call tests/sgrid/test_sgrid.py::test_grid3Dmetadata_standard_names
0.04s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[None-ds_2d_right]
0.04s call tests/test_particleset_execute.py::test_execution_recover_out_of_bounds
0.04s setup tests/test_uxarray_fieldset.py::test_fesom_in_particleset
0.04s call tests/sgrid/test_sgrid.py::test_dim_sizes_roundtrip
0.04s call tests/test_advection.py::test_horizontal_advection_in_3D_flow[spherical]
0.04s call tests/test_advection.py::test_horizontal_advection_in_3D_flow[flat]
0.04s call tests/test_advection.py::test_advection_zonal[flat]
0.04s call tests/test_advection.py::test_advection_meridional[flat]
0.04s call tests/test_advection.py::test_length1dimensions[single_w_layer-single_v_layer-single_u_layer]
0.04s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[None-2d_left_rotated]
0.03s call tests/test_advection.py::test_length1dimensions[no_vertical-full_v-single_u_layer]
0.03s call tests/test_advection.py::test_length1dimensions[full_w-single_v_layer-full_u]
0.03s call tests/test_advection.py::test_decaying_moving_eddy[AdvectionEE-0.1]
0.03s call tests/test_advection.py::test_length1dimensions[full_w-single_v_layer-single_u_layer]
0.03s call tests/test_advection.py::test_length1dimensions[single_w_layer-single_v_layer-full_u]
0.03s call tests/test_advection.py::test_length1dimensions[single_w_layer-full_v-single_u_layer]
0.03s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[None-ds_2d_inner]
0.03s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[None-ds_2d_outer]
0.03s call tests/test_advection.py::test_length1dimensions[single_w_layer-full_v-full_u]
0.03s call tests/test_convert.py::test_convert_nemo_offsets
0.03s call tests/test_advection.py::test_length1dimensions[full_w-full_v-full_u]
0.03s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[None-2d_left_unrolled_cone]
0.03s call tests/test_advection.py::test_length1dimensions[full_w-full_v-single_u_layer]
0.02s call tests/test_advection.py::test_length1dimensions[no_vertical-single_v_layer-single_u_layer]
0.02s call tests/test_advection.py::test_length1dimensions[no_vertical-full_v-full_u]
0.02s call tests/test_advection.py::test_length1dimensions[no_vertical-single_v_layer-full_u]
0.02s call tests/test_uxarray_fieldset.py::test_set_interp_methods
0.02s call tests/test_uxarray_fieldset.py::test_fesom_in_particleset
0.02s call tests/test_convert.py::test_convert_mitgcm_offsets
0.02s call tests/test_uxarray_fieldset.py::test_fesom_fieldset
0.02s call tests/test_particlefile.py::test_pset_execute_outputdt_forwards
0.02s call tests/test_advection.py::test_moving_eddy[AdvectionRK4_3D-1e-05]
0.02s call tests/test_particlefile.py::test_file_warnings
0.02s call tests/test_advection.py::test_moving_eddy[AdvectionDiffusionM1-0.01]
0.02s call tests/test_advection.py::test_moving_eddy[AdvectionDiffusionEM-0.01]
0.02s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[coords-2d_left_rotated]
0.01s call tests/test_particleset_execute.py::test_some_particles_throw_outofbounds
0.01s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[coords-ds_2d_inner]
0.01s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[coords-ds_2d_outer]
0.01s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[coords-2d_left_unrolled_cone]
0.01s call tests/test_advection.py::test_advection_3D_outofbounds[False-up]
0.01s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[coords-ds_2d_left]
0.01s call tests/datasets/test_utils.py::test_replace_arrays_with_zeros[coords-ds_2d_right]
0.01s call tests/test_advection.py::test_advection_3D_outofbounds[False-down]
0.01s call tests/test_advection.py::test_moving_eddy[AdvectionRK4-1e-05]
0.01s call tests/test_particlefile.py::test_particlefile_init_existing_path_modes
0.01s call tests/test_particleset_execute.py::test_particleset_endtime_type[endtime0-expectation0]
0.01s call tests/test_particlefile.py::test_pset_execute_output_time_forwards
0.01s call tests/test_advection.py::test_moving_eddy[AdvectionRK2_3D-0.0001]
0.01s call tests/test_advection.py::test_advection_3D_outofbounds[True-down]
0.01s call tests/test_particlefile.py::test_time_is_age[outputdt1]
0.01s call tests/test_convert.py::test_nemo_to_sgrid_2d[NemoCurvilinear_data_zonal]
0.01s call tests/test_interpolation.py::test_interp_regression_v3[linear]
0.01s call tests/test_advection.py::test_moving_eddy[AdvectionRK2-0.0001]
0.01s call tests/test_particlefile.py::test_time_is_age[outputdt2]
0.01s call tests/test_particlefile.py::test_write_timebackward
0.01s call tests/test_convert.py::test_nemo_to_sgrid_with_depth[U0-V0-depth0-coords0]
0.01s call tests/test_advection.py::test_moving_eddy[AdvectionEE-0.01]
0.01s call tests/test_convert.py::test_convert_copernicusmarine_no_logs[ds1]
0.01s call tests/test_convert.py::test_convert_copernicusmarine_no_logs[ds0]
0.01s setup tests/test_interpolation.py::test_raw_2d_interpolation[Zero]
0.01s call tests/test_particleset_execute.py::test_pset_multi_execute[False]
0.01s call tests/test_fieldset.py::test_fieldset_no_UV
0.01s call tests/datasets/test_strategies.py::test_sgrid_dataset_raises_when_no_node_coordinates
0.01s call tests/test_particlefile.py::test_pfile_array_remove_all_particles
0.01s call tests/test_interpolation.py::test_interpolation_mesh_type[spherical]
0.01s call tests/test_diffusion.py::test_fieldKh_Brownian[spherical]
0.01s call tests/test_diffusion.py::test_randomexponential[1]
0.01s call tests/test_diffusion.py::test_fieldKh_Brownian[flat]
0.01s call tests/test_particlefile.py::test_write_fieldset_without_time
0.01s call tests/test_particlefile.py::test_compression[brotli]
0.01s call tests/test_interpolation.py::test_interpolation_mesh_type[flat]
0.01s call tests/test_particleset_execute.py::test_delete_on_all_errors
0.01s call tests/test_fieldset.py::test_fieldset_from_icon
(2417 durations < 0.005s hidden. Use -vv to show these durations.)
============================================ 512 passed, 512 skipped, 9 xfailed, 1034 warnings in 98.08s (0:01:38) ============================================
What version of Parcels are you running?
v4
Is your feature request related to a problem?
The full set of unit tests in v4 runs considerably faster than in v3, but there are still some relatively slow unit tests. Much can be gained in terms of development workflow if these are sped up
Describe the solution you'd like
Speeding up of some unit tests, for example by running for shorter time (while still maintaining accuracy)
Describe alternatives you've considered
No response
Additional context
Below is the output of
pytest tests/ --durations=0on my machine (at ab0adbc). Note that the longest-duration unit tests (test_mitgcm()) takes more than 30% of the full 98s unit test!