python/ pulse library: development history and validation

Note 4 · Author(s): Anthony Villano · 22 Sept 2026 (revised 22 Oct 2026) · Status: In progress

Caution (2026-10-22): the sample time has changed. The phonon sample period is 0.8 µs (1.25 MHz), not the 1.6 µs (625 kHz) this note was written with (see The sample period in python/README.md). What changes in this note:

How the library came about, what changed when, and how it was checked. What the library is — the modules, the function tables with their status, and the done / under-development convention — is in Note 1a.

Code Repository

Why it was written

Three notebooks (07221203_2025_dump1_noise.ipynb, 07221203_2025_dump2_pulse.ipynb and good_pulses_overlay.ipynb) had each grown their own copy of the same handful of computations, with the same constants (PRETRIGGER_SAMPLES, GLITCH_SAMPLES) hardcoded separately in each. The request was for a reusable library of pulse operations, cuts and derived quantities with a uniform input for every category.

Timeline

DateCommitChangeFunctionsTests
2026-09-22b23b9edLibrary created: config, quantities, cuts, operations, I/O and the status-tagging mechanism; the three notebooks refactored to use it. Two functions (rise_sample, excursion_below_percentile) tagged under development.2029
2026-09-23c3d6754Named log-ratio cuts excursion_band and excursion_band_loose added (from Note 2), with log_excursion_ratio. Pulse veto added: excursion_duration, looks_like_pulse, excursion_band_AI (Note 6). Old generic excursion_band renamed ratio_band.2635
2026-09-23894ef9dHDF5 good-event archive added (pulse_archive.py: save, load, list).2940
2026-09-238b0ab40Branch 2 cuts renamed Loose (excursion_band_loose, [0.5, 1.0)) and Tight (excursion_band, [0.5, 0.7)) in the notebook, plots and docstrings, replacing “Best” and “Tightened”. No behavior change.2940
2026-09-25f0196bdFixed-window baseline quantities added: bstd (std of the first 500 samples by default, glitch samples skipped) and bstd_1000 (first 1000). pretrigger_std and the PulseConfig default are unchanged; both reproduce the notebook’s bstd arrays exactly.3143
2026-09-253e4efc6Baseline-level companions added: bline (mean of the first 500 samples by default, glitch samples skipped) and bline_1000, built on pretrigger_mean. pretrigger_mean and the PulseConfig default are unchanged; both reproduce the notebook’s bmean arrays exactly.3346
2026-09-2561c374fTime-of-rise quantity added: half_max_time_us, the time in microseconds (0.8 µs per sample; the library then used 1.6 µs, which was wrong) at which a pulse first reaches 50% of its maximum deviation, built on rise_sample and tagged under development. It needs a window that ends before the pulse (500 samples for the dump 1 triggers). Used in Note 2a.3449
2026-09-25e6033f0Trigger-type cuts added: randoms (recorded Physics) and real_triggers (recorded Unknown) in pulse_cuts, plus pulse_io.trigger_types to read the label from the dump’s .csv. The two cuts take the trigger-type array rather than pulses, and are tagged under development because the label mapping (assumed swapped) is unconfirmed. Used in Note 2a.3752

Function counts are status-tagged functions (PulseConfig and the status machinery are not counted). The current inventory, 29 done and 8 under development, is in Note 1a.

Validation

Refactoring reproduced the notebooks’ numbers

After each notebook was switched to the library it was re-run end to end and its printed results compared with the earlier ad-hoc version:

Unit tests

There are 52 synthetic-data unit tests (no dependence on real detector data or nsdf_dark_matter), all passing. They run under the base conda environment, since pytest is not installed in darkmatter_cli_env: cd python && python -m pytest tests/. h5py was installed into darkmatter_cli_env on 2026-09-23 for the archive module.

One test began life with a wrong expected value and, once corrected, became a regression test for the pretrigger-contamination effect of Note 2: a spike confined to the pretrigger window drags the baseline mean far enough that flat post-trigger samples end up looking like the largest excursion.

Checks on real data

The new cuts were checked against real Detector 0 / Channel 0 data, not just synthetic tests. excursion_band and excursion_band_loose reproduced the notebook’s inline masks exactly (192 and 744 of 1517). The first version of excursion_duration passed its unit test but failed on real data, flagging every trace of the Tight cut as a pulse, and was replaced by the consecutive-run definition (details in Note 6).

Naming change on 2026-09-23

The original generic excursion_band(low, high) thresholded the linear ratio. To free the name for the specific log10(ratio) cut, the generic function was renamed ratio_band. Its one caller, the ratio > 9 cut in 07221203_2025_dump2_pulse.ipynb (Note 5), was updated and re-run to confirm it still selects the same 10 traces. An early attempt to edit that notebook programmatically silently did nothing (the cell source was a single string, not a list of lines) and was only caught because the re-run failed.

Status / Next Steps