Hi L1B: filter gain tests using the pointing's own reference voltages - #3397
Merged
Conversation
Retire the ancillary-file-based gain-configuration classification (config_id, GainConfigLookupTable, load_gain_configuration) added in IMAP-Science-Operations-Center#3358. Gain test intervals are now detected by comparing every HVSCI segment's median detector voltages against the pointing's own first HVSCI segment (first ~3 packets), using hardcoded per-field tolerances (HiConstants.GAIN_TEST_HV_DELTA_V) rather than an ancillary lookup table, per Paul Janzen's feedback (IMAP-Science-Operations-Center#3391). - hi/utils.py: remove load_gain_configuration()/GainConfigLookupTable; add GoodMetRangeLookupTable and HiConstants.GAIN_TEST_HV_DELTA_V. - hi/hi_l1b.py: replace de_gain_configuration()/ classify_gain_configuration()/get_gain_configuration_lut() with de_gain_test_filter(), compute_reference_hv_values(), and compute_gain_match_values(). The latter's output (gain_match_{field} global attrs) is the new L1B->L1C interface for a future gain-state geometric factor lookup (IMAP-Science-Operations-Center#3395). - hi/hi_l1c.py: drop the retired gain_config_anc dependency; the geometric_factor lookup is stubbed to FILLVAL until IMAP-Science-Operations-Center#3395 implements the real gain-state-aware lookup. - cli.py: drop the gain-configuration ancillary dependency from both the L1B DE and L1C PSET branches. - Delete the gain-configuration ancillary test fixture (no production file exists yet). Part of IMAP-Science-Operations-Center#3391. Closes IMAP-Science-Operations-Center#3394. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ation de_gain_test_filter() was classifying direct events as good/bad using "ccsds_met" (the CCSDS packet creation time, logged by flight software only after the data was collected). Verified against a real gain-test pointing (repoint 152, 2026-02-09): ccsds_met lags esa_step_met (the MET when the ESA was actually stepped, i.e. when data collection for that 8-spin group began) by ~30-120 seconds. Out of 2713 direct event packets, 51 were classified differently depending on which time was used -- including cases where good data would be wrongly excluded and cases where gain-test data would be wrongly included as good, right at segment transitions. Switch to "esa_step_met" for the good_met_ranges query, and reorder annotate_direct_events() to compute it (de_esa_step_met()) before de_gain_test_filter() runs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
tmplummer
requested review from
leowerneck and
tech3371
and
a balanced review from Copilot
August 18, 2026 17:27
Contributor
There was a problem hiding this comment.
Pull request overview
Replaces ancillary-based Hi gain classification with per-pointing reference-voltage filtering and prepares the L1B-to-L1C gain-state interface.
Changes:
- Filters gain-test HVSCI intervals using first-segment reference voltages.
- Adds
gain_match_*L1B attributes and hardcoded tolerances. - Removes gain-configuration ancillary dependencies and updates tests/docs.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
imap_processing/hi/hi_l1b.py |
Implements reference-voltage gain-test filtering. |
imap_processing/hi/utils.py |
Adds tolerances and boolean MET-range lookup. |
imap_processing/hi/hi_l1c.py |
Removes legacy gain lookup and uses placeholder factors. |
imap_processing/hi/hi_goodtimes.py |
Updates voltage-filter documentation. |
imap_processing/cli.py |
Removes gain ancillary dependencies. |
imap_processing/quality_flags.py |
Clarifies detector-voltage flag meaning. |
imap_processing/cdf/config/imap_hi_variable_attrs.yaml |
Updates quality-flag metadata. |
imap_processing/tests/hi/test_hi_l1b.py |
Tests gain filtering and derived attributes. |
imap_processing/tests/hi/test_utils.py |
Tests good-MET range lookup. |
imap_processing/tests/hi/test_hi_l1c.py |
Tests placeholder geometric factors. |
imap_processing/tests/test_cli.py |
Updates CLI dependency cases. |
imap_processing/tests/hi/conftest.py |
Removes obsolete fixture. |
imap_processing/tests/hi/data/l1/imap_hi_90sensor-gain-configuration_20240101_v001.csv |
Deletes obsolete ancillary data. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
leowerneck
approved these changes
Aug 18, 2026
leowerneck
left a comment
leowerneck
left a comment
Contributor
There was a problem hiding this comment.
LGTM. See some suggestions for minor docstrings/comments fixes.
tmplummer
merged commit Aug 18, 2026
a33e631
into
IMAP-Science-Operations-Center:dev
16 checks passed
tmplummer
deleted the
3394-hi-l1b-filter-gain-tests-using-the-pointings-own-reference-voltages-no-ancillary-lookup
branch
August 18, 2026 21:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sorry for the large PR. I couldn't find a good way to break this up more than this.
Change Summary
Reworks IMAP-Hi gain-state handling in three steps:
Follow on work: