Cast hit-operation float parameters to float32 at load time

[ggmarshall]

Stacked on #41 (input-validation) — only the last commit is new here; retarget/rebase to main once #41 merges.

Motivation

Profiling LEGEND production files showed the pht (hit) tier's float64 columns — 10 calibrated energies, 3 PSD classifiers, 8 QC cut classifiers — account for 83% of the tier's raw bytes and barely compress (gzip ratios 0.84–0.95, they are noise-like mantissas). They are float64 only because the calibration parameters load from the pars YAML as python floats, and Table.eval/numexpr promotes float32-array × float64-scalar to float64. YAML cannot express float32, so the pars files cannot carry the dtype — the cast has to happen at load time.

Change

build-tier-hit casts python-float entries of each operation's parameters to np.float32 after loading/merging the parameter dictionary (_cast_op_params_f32), in both the multi-channel and single-channel entry points. Since build-tier-hit --tier pht is the same script, this covers the hit and pht tiers. Ints, bools and strings are untouched; floats inside lists are cast.

Operations whose expressions reference genuine float64 columns (e.g. timestamp) still promote to float64, which is correct and safe.

Validation (production p16 pht pars for V05267A over real raw data)

• 48 of 49 float operations become float32; all 23 boolean cut columns bit-identical; NaN patterns identical.
• Every value difference ≤ 5×10⁻⁵ of the column's spread (energies at ~1e-7 relative — ~0.3 eV at 2.6 MeV).
• The single holdout is AoE_Classifier: its generated expression embeds a 10**-99 division guard (a float64 literal, which also underflows to 0 in float32) — that needs a one-line fix in the A/E calibration generator (pygama), out of scope here.
• Expected effect: ~40% pht file-size reduction (in combination with compression), since the float64 columns dominate the tier.

Tests

tests/test_build_tier_hit.py: unit test of the cast helper's type handling, plus an end-to-end run of the build-tier-hit console script over a synthetic float32 DSP file with a minimal rule-config tree, asserting the derived column comes out float32 with values matching a float64 reference at 5e-6.

Disclosure

Per AI_POLICY.md: developed with AI assistance (Claude) and reviewed by the submitter.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR reduces HIT/PHT output size by preventing Table.eval/numexpr from promoting derived columns to float64 solely due to Python-float (float64) calibration parameters loaded from YAML. It introduces a load-time cast of per-operation float parameters to np.float32, and adds tests that exercise both the casting helper and the end-to-end CLI behavior.

Changes:

• Add _cast_op_params_f32 to cast operation parameters floats (including floats inside lists) to np.float32 after config/pars merge.
• Apply the cast in both build-tier-hit entry points (multi-channel and single-channel).
• Add unit + end-to-end tests asserting derived HIT outputs remain float32 when inputs are float32.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
src/legenddataflowscripts/tier/hit.py	Adds and applies _cast_op_params_f32 to keep derived HIT/PHT columns float32 when possible.
tests/test_build_tier_hit.py	Adds targeted tests for type handling and an end-to-end run validating float32 output dtype and numerical agreement.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.