Changing Argo float kernels to use boolean masking#2632
Open
erikvansebille wants to merge 2 commits into
Open
Conversation
Which is easier to read than the np.where() clauses used before Performance can be tested in the benchmarking suite
12 tasks
Member
Author
|
To inform the difference between Short summary: they are virtually the same speed. I therefore propose to change the tutorial to use this simpler/more-intuitive kernel Below is the output for the (old) kernel using np.where(). cumtime is 0.229 seconds (boolean_masking_profiling) erik ~/Codes/elphe-hackathon/boolean_masking_profiling % python argo_kernel_profiling.py --argo-kernel npwhere
/Users/erik/Codes/elphe-hackathon/boolean_masking_profiling/argo_kernel_profiling.py:9: UserWarning: This is an alpha version of Parcels v4. The API is not stable and may change without deprecation warnings.
import parcels
INFO: Output files are stored in output_argo.parquet
Integration time: 2024-07-06T22:00:00 100%|█████████████████████████████████████████████████| [00:12<00:00, 40740.27it/s]
8916903 function calls (8782721 primitive calls) in 16.386 seconds
Random listing order was used
List reduced from 4024 to 3 due to restriction <'run_simulation|particleset\\.py:\\d+\\(execute\\)|ArgoVerticalMovement'>
ncalls tottime percall cumtime percall filename:lineno(function)
576 0.043 0.000 0.229 0.000 /Users/erik/Codes/elphe-hackathon/boolean_masking_profiling/argo_kernel_profiling.py:34(ArgoVerticalMovement_npwhere)
2/1 0.000 0.000 12.726 12.726 /Users/erik/Codes/elphe-hackathon/boolean_masking_profiling/argo_kernel_profiling.py:127(run_simulation)And below is the output for the (new) kernel using boolean masking. cumtime is 0.227 seconds (boolean_masking_profiling) erik ~/Codes/elphe-hackathon/boolean_masking_profiling % python argo_kernel_profiling.py --argo-kernel boolean
/Users/erik/Codes/elphe-hackathon/boolean_masking_profiling/argo_kernel_profiling.py:9: UserWarning: This is an alpha version of Parcels v4. The API is not stable and may change without deprecation warnings.
import parcels
INFO: Output files are stored in output_argo.parquet
Integration time: 2024-07-06T22:00:00 100%|█████████████████████████████████████████████████| [00:12<00:00, 42156.98it/s]
8964778 function calls (8830591 primitive calls) in 16.681 seconds
Random listing order was used
List reduced from 4026 to 3 due to restriction <'run_simulation|particleset\\.py:\\d+\\(execute\\)|ArgoVerticalMovement'>
ncalls tottime percall cumtime percall filename:lineno(function)
576 0.021 0.000 0.227 0.000 /Users/erik/Codes/elphe-hackathon/boolean_masking_profiling/argo_kernel_profiling.py:85(ArgoVerticalMovement_boolean)
2/1 0.000 0.000 12.299 12.299 /Users/erik/Codes/elphe-hackathon/boolean_masking_profiling/argo_kernel_profiling.py:127(run_simulation) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR changes the Kernels in the Argo tutorial to use boolean masking, which is easier to read than the
np.where()clauses used before.I'll create a script in the benchmarking repo so that we can test performance between the two types of vectorised
if-statements (see also #2197)Description
Checklist
np.where()versus boolean masking in Parcels kernels and interpolators #2197mainfor normal development,v3-supportfor v3 support)AI Disclosure