excursion_band_AI: removing pulse-like traces from the noise selection, and the good_noise archive

Note 6 · Author(s): Anthony Villano · 23 Sept 2026 (revised 22 Oct 2026) · Status: In progress

Caution (2026-10-22): the sample time has changed. The phonon sample period is 0.8 µs (1.25 MHz), not the 1.6 µs (625 kHz) this note was written with (see The sample period in python/README.md for the evidence). What changed in this note:

The frequency axis of the PSD figure is twice what the first version showed (bins of 305.2 Hz up to 625 kHz); the figure of the three average PSDs was replaced.
The table of ratios at fixed frequencies was recomputed: the same bins now sit at twice the frequency (the first bin is 305 Hz, not 150 Hz, and so on), so the rows are at 305 Hz, 2.1, 4.0, 10, 20 and 60 kHz instead of 150 Hz, 1, 2, 5, 10 and 30 kHz, and the sentence “below ~2 kHz” now reads ~4 kHz. Reading the table at the old frequencies would give different numbers (at 1 kHz the ratios are now 122 and 136, at 5 kHz 16 and 16, at 30 kHz 1.3 and 0.9).
archives/good_noise.h5 was regenerated; it now records sample_period_s = 0.8×10⁻⁶. The selection in it (179 events) is the same.
Unchanged: every count (192, 744, 179, 13 rejected), the run lengths in samples, min_duration = 50 samples, the traces in the two trace figures (axes in samples) and all |FFT|² values.

Code Repository

07221203_2025_dump1_noise.ipynb — the “Tight, AI-filtered”, “What the AI filter removes” and “Archive” sections at the end. Pinned to eda12dc.
good_pulses_overlay.ipynb — the three-cut PSD comparison. Pinned to eda12dc.
python/pulse_cuts.py and pulse_quantities.py — excursion_band_AI, looks_like_pulse, excursion_duration. Pinned to ac55c3c.
python/pulse_archive.py and archives/ — the HDF5 archive. Pinned to eda12dc.

Motivation

The Tight noise cut of Note 2 (log10(ratio) in [0.5, 0.7), 500-sample window) keeps 192 of 1517 traces and is the cleanest slice found, but a few real pulses still get through. excursion_band_AI is the same cut with a second criterion added: remove whatever looks like a real pulse. The name is literal: the pulse judgment is a heuristic worked out with an AI assistant, not a validated physical criterion, and the cut is tagged under_development.

The pulse-shape criterion

A real pulse rises and then decays slowly, so it stays near its own peak for a long time. Noise does not. The quantity excursion_duration is the longest run of consecutive post-pretrigger samples whose deviation is at least half of the trace’s own dev. looks_like_pulse flags a trace when that run is at least min_duration samples.

A first version that failed

The first attempt counted the total number of samples above half of dev. On real data it flagged all 192 traces of the Tight cut as pulses. With ~2500 post-pretrigger samples and dev only 3–5 times the noise standard deviation, half of dev is just 1.5–2.5 sigma, which ordinary Gaussian noise exceeds in a few percent of samples. Requiring the samples to be consecutive removes that effect, since the chance of a long unbroken run in noise falls off geometrically.

Choosing `min_duration`

With the consecutive-run definition, the 192 traces of the Tight cut split cleanly (Detector 0 / Channel 0, dump 1, 500-sample window):

179 traces have a longest run of 2–25 samples (median 7);
13 traces have runs of 168 to 3564 samples (168, 225, 919, 1699, 1940, 2055, 2079, 2768, 2875, 3300, 3521, 3538, 3564);
nothing lies between 25 and 168.

The default min_duration = 50 sits inside that gap. It was picked by eye from this one dataset, so it may need a different value elsewhere.

What it removes

excursion_band_AI keeps 179 of the 192 traces. All 13 rejected traces are visually unambiguous real pulses or slow drifts:

The 13 traces rejected by excursion_band_AI: real pulses and slow baseline drifts — The 13 traces kept by `excursion_band` but rejected by `excursion_band_AI`: pulses (rising near samples 450–600 and 2400) and slow monotonic drifts.

The 179 kept traces are flat noise (30 shown; the spike at sample 0 is the known leading glitch):

30 of the 179 traces kept by excursion_band_AI, flat noise — `excursion_band_AI`: 179 of 1517 traces.

Because slow drifts also stay near their peak for a long time, the same criterion rejects drifting baselines, not only pulses. That is desirable for a noise selection, and relevant to Note 5, where drift is the problem.

Effect on the noise PSD

The clearest practical check is the average PSD of each selection (Detector 0 / Channel 0, dump 1). Below ~4 kHz, excursion_band and excursion_band_loose lie well above excursion_band_AI, because a pulse’s rise and decay is low-frequency dominated and a few of them are enough to raise a mean spectrum:

Frequency	tight / AI	loose / AI
305 Hz (first bin)	164	165
2.1 kHz	64	61
4.0 kHz	23	20
10 kHz	4.3	4.0
20 kHz	1.8	1.4
60 kHz	1.1	0.7

Ratio of the mean |FFT|² of each selection to that of excursion_band_AI at that frequency (nearest bin; bins are 305 Hz apart).

Average PSD for excursion_band, excursion_band_loose and excursion_band_AI on the same vertical scale as the 14-trace PSD — Average PSD for the three selections (N = 192, 744, 179), on the same vertical scale as the 14-trace PSD of Note 3; the first two run off the top at low frequency.

The PSD is therefore an independent check of the pulse criterion, on top of the visual one. It is not a proof: the same data were used to choose min_duration.

The good_noise archive

Since the AI selection is the one to build on, its detector_ids are stored in an HDF5 file, archives/good_noise.h5, written and read through python/pulse_archive.py. Layout: one group per (series, detector, channel) at /<series>/detector<N>/channel<C>, containing a detector_ids dataset plus attributes recording how the selection was made: cut name and parameters, the cut’s status (under_development here), the PulseConfig values, dump, source notebook, counts kept and total, and a UTC timestamp.

First entry: 07221203_2025, detector 0, channel 0: 179 events out of 1517, from excursion_band_AI.
Saving a group overwrites it: the file holds the current selection per key, not a history.
The functions (save_good_events, load_good_events, list_archive) are cut-agnostic; a future non-noise archive would get its own file rather than a group in this one.

Status / Next Steps

Check looks_like_pulse and its threshold on other channels, detectors and dumps before relying on it; it was tuned on one dataset.
Decide whether small or unusual pulses that never reach 50 consecutive samples above half-maximum could slip through.
Fill the archive for the remaining channels and detectors as they are processed.
The archived selection is marked under_development and will change if the cut does.