python/The shared pulse-analysis library. Background and history: Note 1a and Note 4. The modules are flat files, imported by putting python/ on sys.path (see python/README.md). Samples are on the last axis, so every function takes one pulse or a whole batch. A function marked under development still runs but emits a UserWarning on every call.
| Module | What it is | Functions | Done | Under development |
|---|---|---|---|---|
pulse_io.py | Bridge from nsdf_dark_matter's cdms objects into the plain-ndarray convention ((n_pulses, n_samples), samples on the last axis) the rest of this library expects | 6 | 6 | 0 |
pulse_config.py | Shared configuration for the pulse operations/cuts/quantities library | 0 | 0 | 0 |
pulse_operations.py | Operations: pulses in, new (modified) pulses out -- same pulse-shaped family | 3 | 3 | 0 |
pulse_quantities.py | Derived quantities: pulses in, one real number (or one array) per pulse out | 14 | 11 | 3 |
pulse_cuts.py | Cuts: pulses in, one boolean per pulse out (True = keep) | 11 | 6 | 5 |
pulse_archive.py | Persistent archive of "good event" selections, keyed by series/detector/channel | 3 | 3 | 0 |
pulse_channels.py | Which physical detector channel is a dump's (detector, channel) pair? | 6 | 0 | 6 |
pulse_status.py | Development-status tagging for functions in this library | 2 | 0 | 0 |
Bridge from nsdf_dark_matter's cdms objects into the plain-ndarray convention ((n_pulses, n_samples), samples on the last axis) the rest of this library expects. Only this module knows about cdms; everything else in python/ only ever sees plain arrays.
| Function | Summary | Status |
|---|---|---|
detector_index(detector_id: str) -> int | The <detector> component of a "<event>_<detector>_Phonon_<n>" id. | done |
event_id(detector_id: str) -> str | The <event> component of a "<event>_<detector>_Phonon_<n>" id. | done |
filter_by_detector(detector_ids, detector) | Keep only ids whose <detector> component matches detector. | done |
load_channel_batch(cdms, detector_ids, channel: int) | Stack one channel from a list of detector_ids into a batch array.detailsReturns | done |
trigger_types(cdms, detector_ids) | The trigger_type string the dump's .csv records for each id's event, as an array aligned with detector_ids (e.g. "Physics", "Unknown"). Metadata is per event, so the three detectors of one event share a value. Feed it to pulse_cuts.randoms / real_triggers. | done |
load_channel_batch_multi(cdms_by_dump, channel: int, detector=None) | Like load_channel_batch, pooled across multiple dumps.details
Returns | done |
Shared configuration for the pulse operations/cuts/quantities library.
One PulseConfig instance is passed explicitly to every function in pulse_operations.py, pulse_cuts.py, and pulse_quantities.py, rather than having each function hardcode PRETRIGGER_SAMPLES/GLITCH_SAMPLES constants the way the exploratory notebooks did.
The sample period (sample_period_s) is 0.8 us (1.25 MHz) for the phonon channels of the DCRC readout. Until 2026-10-22 the library used 1.6 us, which was wrong: see "The sample period" in python/README.md for the evidence. PulseConfig warns when 1.6e-6 is passed explicitly.
PulseConfig(pretrigger_samples: int = 1000, glitch_samples: int = 10, sample_period_s: Optional[float] = 8e-07) -> None| Field | Meaning |
|---|---|
pretrigger_samples | Number of leading samples that make up the pretrigger (baseline) window. |
glitch_samples | Leading samples within the pretrigger window to skip before computing baseline mean/std, to avoid a brief electronic-glitch step artifact seen at the very start of some traces (see 07221203_2025_dump1_noise.ipynb). |
sample_period_s | Seconds per sample (0.8 us -> 1.25 MHz for the phonon channels of this readout). None if unknown for a given dataset -- functions that need it (e.g. frequencies_hz) will raise rather than silently assume a value. |
Operations: pulses in, new (modified) pulses out -- same pulse-shaped family.
Every function takes an array shaped (..., n_samples) and returns an array of the same leading shape (trailing/sample axis may shrink, e.g. trim_glitch), never a scalar or boolean -- that's what distinguishes an operation from a quantity or a cut. None of these mutate their input in place.
| Function | Summary | Status |
|---|---|---|
baseline_subtract(pulses, config: PulseConfig = DEFAULT_CONFIG) | Subtract each pulse's own pretrigger mean (see pulse_quantities.pretrigger_mean). | done |
trim_glitch(pulses, config: PulseConfig = DEFAULT_CONFIG) | Drop the leading config.glitch_samples samples from every pulse.detailsShortens the sample axis by | done |
clip(pulses, low=None, high=None) | Clip every sample to [low, high]. Plain np.clip wrapper for pipeline symmetry with the other operations (e.g. clipping a display copy without touching source data). | done |
Derived quantities: pulses in, one real number (or one array) per pulse out.
Every function here takes an array shaped (..., n_samples) -- a single pulse (n_samples,) or a batch (n_pulses, n_samples) both work unchanged, since all reductions run along axis=-1. Output shape is one dimension smaller ((...,)) for scalar-per-pulse quantities, or (..., n_freq) for spectra.
These are the primitives: cuts in pulse_cuts.py are built by thresholding the quantities defined here.
| Function | Summary | Status |
|---|---|---|
pretrigger_mean(pulses, config: PulseConfig = DEFAULT_CONFIG) | Mean of the pretrigger window, skipping config.glitch_samples leading samples. | done |
pretrigger_std(pulses, config: PulseConfig = DEFAULT_CONFIG) | Std of the pretrigger window, skipping config.glitch_samples leading samples. | done |
bstd(pulses, config: PulseConfig = DEFAULT_CONFIG, n_samples: int = 500) | Baseline scatter ("bstd"): std of the first n_samples samples (default 500), skipping config.glitch_samples leading samples.detailsUnlike | done |
excursion_duration(pulses, config: PulseConfig = DEFAULT_CONFIG, fraction: float = 0.5) | Longest run of consecutive post-pretrigger samples with |sample - pretrigger_mean| >= fraction * max_deviation -- how long the trace stays near its own peak without dropping back down. | under development devised specifically to tell a sustained real pulse (rises and stays elevated for many consecutive samples while it decays) apart from noise, for use by pulse_cuts.looks_like_pulse / excursion_band_AI. Deliberately uses the longest contiguous run, not a total count -- with ~2500 post-pretrigger samples, a plain count of samples above half-max is dominated by ordinary Gaussian tail hits (checked against real 07221203_2025_F0001 data: an early total-count version flagged every trace in the tight excursion_band as 'pulse-like'). Not independently validated beyond that one check. |
half_max_time_us(pulses, config: PulseConfig = DEFAULT_CONFIG) | Time (microseconds from the first sample of the trace) at which the pulse first reaches 50% of its own max_deviation: rise_sample(fraction=0.5) * sample_period_s.detailsOnly samples after | under development built on rise_sample, which is still under development; meaningless for a trace with no pulse (it just finds the largest noise excursion) |
rise_sample(pulses, config: PulseConfig = DEFAULT_CONFIG, fraction: float = 0.5) | First post-pretrigger sample index (absolute, not relative to the window) where the pulse crosses fraction of its own max_deviation.detailsGuaranteed to find a crossing (the sample achieving the max deviation itself always satisfies | under development fraction=0.5 half-max crossing heuristic; only validated on one exploratory analysis (07221203_2025_dump1_noise.ipynb's Cut A rise-time check) -- not yet checked against noisy/multi-peak traces |
bstd_1000(pulses, config: PulseConfig = DEFAULT_CONFIG) | bstd over the first 1000 samples (skipping config.glitch_samples): the baseline scatter of the 1000-sample window used by Branch 1 of 07221203_2025_dump1_noise.ipynb. | done |
bline(pulses, config: PulseConfig = DEFAULT_CONFIG, n_samples: int = 500) | Baseline level ("bline"): mean of the first n_samples samples (default 500), skipping config.glitch_samples leading samples. The companion of bstd.detailsComputed with | done |
bline_1000(pulses, config: PulseConfig = DEFAULT_CONFIG) | bline over the first 1000 samples (skipping config.glitch_samples). | done |
max_deviation(pulses, config: PulseConfig = DEFAULT_CONFIG) | Max |post-pretrigger sample - pretrigger mean| per pulse ("dev" in the notebooks).detailsMeasured only in the post-pretrigger region (samples | done |
excursion_ratio(pulses, config: PulseConfig = DEFAULT_CONFIG) | max_deviation / pretrigger_std ("ratio" in the notebooks).detailsFor pure noise, extreme-value statistics put this around 3-8 (the max of ~3000 samples is expected to be several sigma out), not near 1 -- traces that land near 1 usually have a contaminated (inflated) pretrigger std instead of being unusually quiet. See the "inverts the naive expectation" finding in 07221203_2025_dump1_noise.ipynb before using this as a "smaller = quieter" cut. | done |
log_excursion_ratio(pulses, config: PulseConfig = DEFAULT_CONFIG) | log10(excursion_ratio) -- the quantity actually thresholded by the notebooks' Branch 2 exploration (07221203_2025_dump1_noise.ipynb), since the ratio's dynamic range spans orders of magnitude between the contaminated and quiet populations. | done |
power_spectrum(pulses, config: PulseConfig = DEFAULT_CONFIG) | Real part of rfft(pulse) * conj(rfft(pulse)) ("complex square"/|FFT|^2) per pulse.detailsDoes not baseline-subtract for you -- pass already-baseline-subtracted pulses (see | done |
frequencies_hz(n_samples: int, config: PulseConfig = DEFAULT_CONFIG) | Frequency bins (Hz) matching power_spectrum's / np.fft.rfft's output for a pulse of length n_samples. Raises if config.sample_period_s is unset. | done |
Cuts: pulses in, one boolean per pulse out (True = keep).
Every function takes an array shaped (..., n_samples) and returns a boolean array of shape (...,). Cuts are plain functions of the quantities in pulse_quantities.py -- combine them with ordinary &/|/~ rather than a special combinator API.
randoms and real_triggers are the exception: they take an array of trigger-type labels (see pulse_io.trigger_types), not pulses, since that is per-event metadata.
Percentile-based cuts (quiet_baseline, excursion_below_percentile) compute the percentile from the batch passed in, i.e. "quiet relative to this population" -- pass the same batch you're about to select from, not some other reference set, unless that's actually what you want.
| Function | Summary | Status |
|---|---|---|
dead_channel(pulses, config: PulseConfig = DEFAULT_CONFIG) | True where the pretrigger std is exactly zero -- channel not read out for this pulse, not merely a quiet channel. See the Detector 0/Channel 2 finding in 07221203_2025_dump1_noise.ipynb (100% dead in that dump). | done |
active(pulses, config: PulseConfig = DEFAULT_CONFIG) | ~dead_channel, as its own name for readability at call sites. | done |
quiet_baseline(pulses, config: PulseConfig = DEFAULT_CONFIG, percentile: float = 75, bstd=None) | True where pretrigger_std is below its own percentile-th value across the active pulses in this batch. Dead channels are always excluded (never True).detailsPass a precomputed | done |
excursion_band_AI(pulses, config: PulseConfig = DEFAULT_CONFIG, low=0.5, high=0.7, log_ratio=None, min_duration: int = 50, duration=None) | excursion_band(low, high) with looks_like_pulse traces removed. | under development excursion_band ([0.5, 0.7) by default) with looks_like_pulse traces removed -- i.e. 'whatever Claude judged to be a real pulse, taken out.' Both the band and the pulse-shape veto are judgment calls; treat this as an exploratory variant of excursion_band, not an independently validated cut, and check what it removes before trusting it on a new dataset. |
excursion_below_percentile(pulses, config: PulseConfig = DEFAULT_CONFIG, percentile: float = 25, ratio=None) | True where excursion_ratio is below its own percentile-th value across the active pulses in this batch. Convenience wrapper matching the original dump1_noise notebook's "good noise" ratio cut -- see its module docstring for why a low ratio is not automatically "quieter. | under development matches the original dump1_noise 'good noise' cut, which the same notebook's widened-Cut-A investigation found is biased toward contaminated-baseline traces rather than quiet ones when used alone -- combine with quiet_baseline (already the notebook's practice) rather than relying on this in isolation |
looks_like_pulse(pulses, config: PulseConfig = DEFAULT_CONFIG, min_duration: int = 50, duration=None) | True where excursion_duration is at or above min_duration samples. | under development 'looks like a pulse' is a judgment call, not a validated physical criterion. Flags traces whose excursion stays near its own peak for a long consecutive run (excursion_duration >= min_duration), which a real detector pulse's rise-then-decay does and ordinary noise usually doesn't. default min_duration=50 was picked by eye from a real gap in the data (07221203_2025_F0001, Detector0/Channel0, 500-sample window): within the tight excursion_band, 179/192 traces had duration <= 25 and the other 13 had duration >= 168 -- nothing in between -- and those 13 are visually unambiguous real pulses/drifts, the other 179 visually flat noise. Devised for excursion_band_AI -- treat as a first guess to be checked against the traces it flags on any new dataset, not ground truth. |
randoms(trigger_types, label: str = 'Physics') | True where the recorded trigger type equals label (default "Physics"), taken to be random triggers. Takes the trigger-type array from pulse_io.trigger_types rather than pulses. | under development assumes the dump's trigger_type labels are swapped relative to their names: events recorded as 'Physics' are taken to be randoms (fixed-count random triggers) and 'Unknown' to be real triggers. That mapping is a working assumption from 07221203_2025_F0001 (see Note 2a), not confirmed against the DAQ configuration or the nsdf library |
real_triggers(trigger_types, label: str = 'Unknown') | True where the recorded trigger type equals label (default "Unknown"), taken to be real triggers. Takes the trigger-type array from pulse_io.trigger_types rather than pulses. | under development the other half of the randoms assumption: events recorded as 'Unknown' are taken to be real (pulse-triggered) events. Not confirmed; see randoms |
ratio_band(pulses, config: PulseConfig = DEFAULT_CONFIG, low=None, high=None, ratio=None) | True where excursion_ratio (linear, not log) is in [low, high) (either bound optional). The generic band-threshold utility -- see excursion_band for the specific log10(ratio) cut used by the Branch 2 exploration.detailsPass a precomputed | done |
excursion_band(pulses, config: PulseConfig = DEFAULT_CONFIG, low=0.5, high=0.7, log_ratio=None) | True where log_excursion_ratio is in [low, high) -- default [0.5, 0.7), the "tight" Branch 2 cut from 07221203_2025_dump1_noise.ipynb (cleanest slice found there, though even it keeps 13 of its 192 traces that look like real pulses -- see excursion_band_AI). Pass a config with the pretrigger window you want (Branch 2 used 500 samples, not the library default of 1000).detailsPass a precomputed | done |
excursion_band_loose(pulses, config: PulseConfig = DEFAULT_CONFIG, log_ratio=None) | excursion_band widened to [0.5, 1.0) -- the "loose" Branch 2 cut from 07221203_2025_dump1_noise.ipynb (more traces retained, somewhat more contamination than the tight [0.5, 0.7) default). | done |
Persistent archive of "good event" selections, keyed by series/detector/channel.
Every cut in pulse_cuts.py is cheap to recompute from raw data, so this isn't a cache -- it's a durable record of which detector_ids a particular cut (run with particular parameters, on a particular dump) actually selected, so a later notebook or a re-run with different library code can compare against what was kept before, without re-deriving it from scratch or re-reading notebook output by hand.
One HDF5 file, one group per (series, detector, channel):
/<series>/detector<N>/channel<C>
dataset "detector_ids" -- variable-length UTF-8 strings
attrs: cut_name, cut_status, cut_params (JSON string), n_total,
n_kept, dump, pretrigger_samples, glitch_samples,
sample_period_s, source_notebook, created (ISO 8601, UTC)
Saving a group overwrites it -- the archive holds the current good-event list per series/detector/channel, not a history of every past run. Only this module knows about h5py; everything else in python/ stays free of that dependency.
| Function | Summary | Status |
|---|---|---|
save_good_events(path, series: str, detector: int, channel: int, detector_ids, cut_name: str, *, cut_status: str = None, cut_params: dict = None, n_total: int = None, dump: str = None, config=None, source_notebook: str = None) | Write (or overwrite) the good-event list for one (series, detector, channel) group. detector_ids is any iterable of id strings (e.g. from pulse_io.load_channel_batch, indexed by a cut's boolean mask).detailsPass | done |
load_good_events(path, series: str, detector: int, channel: int) -> dict | Read back one (series, detector, channel) group.detailsReturns a dict with | done |
list_archive(path) -> dict | Return {series: {detector: {channel: n_kept}}} for every group in the archive -- a quick overview of what's been saved so far. Returns {} if path doesn't exist yet. | done |
Which physical detector channel is a dump's (detector, channel) pair?
A dump labels its traces <event>_<detector>_Phonon_<n> and holds 4 channels per detector, so a trace is addressed by two small integers: detector (0, 1, 2) and channel (0 to 3). In Run 76 those are not three detectors but three 4-channel readout boards (DCRC1, DCRC3, DCRC4) of one two-sided detector, and the real channel names are Side1pA ... Side1pF, Side2pA ... Side2pF. This module turns the two integers into those names:
import pulse_channels as pch
pch.channel_name(0, 0) # 'Side2pC'
pch.channel_label(0, 0) # 'Side2pC (DCRC1 pA, HV2 pA)'
pch.channel_info(0, 0) # Channel(detector=0, channel=0, name='Side2pC', ...)
pch.find_channel('Side2pC') # (0, 0)
pch.all_channels() # the 12 Channel records, in (detector, channel) order
pch.sensor_channels() # the 6 that are QET channels of the detector
Which table to use. The Setup table of the Run 76 summary (https://villano-lab.github.io/R76Analysis/cdms/k100/run_summary/run_76.html; Matt Fritts's explanation of it is on page 28 of notebook N-MISC-23-001) gives, for every sensor Label, the HV connector pin it is wired to in Side-1 readout and in Side-2 readout. HV1 and HV2 are the two D-sub connectors the DCRCs plug into (HV1 is DCRC3, HV2 is DCRC1; the 2022 run diaries' header says "HV 1(L)/2(R) -> DCRC50(3)/DCRC55(1), ... NaI trigger signal -> DCRCX(4)"), and pin pA ... pD of a connector is input pA ... pD of its DCRC. The table's DCRC (2-sided) column is for the two-sided readout and is not used. The run was in Side-2 readout from late in the Run 76 diaries, so the December 2022 series (07221203_2025) is Side-2 readout, the default here (readout=2). In Side-2 readout only the pins A, B and D carry sensors (pC is for Side-1 readout) and DCRC4 pA carries the trigger signal of the NaI array, so six channels are sensors, one is the NaI trigger input and five are not connected. (For DCRC1 the table's A/B order is the one used; the explanatory text next to the table has A and B the other way round, which is taken to be a typo.)
Only this module knows the table; everything else in python/ works on plain arrays and indices.
The mapping is under development: that the dump's detectors 0, 1, 2 are DCRC1, DCRC3, DCRC4 (ascending board number) is inferred, and that channels 0 to 3 are the board inputs pA to pD in order is assumed. If they turn out wrong, change DCRC_OF_DETECTOR or the order of the rows in PINS and nothing else.
History. Until 2026-10-22 this module used the table's DCRC (2-sided) column and named twelve sensors on both sides, which was wrong for Side-2 readout (six of its "Side 1" channels do not exist in the readout of this series).
Channel(detector: int, channel: int, name: str, role: str, side: Optional[int], letter: Optional[str], dcrc: str, board_input: str, hv_output: Optional[str])One channel of the readout: where it is in the dump and what it is on the detector.
| Function | Summary | Status |
|---|---|---|
all_channels(readout: int = 2) -> list | All 12 Channel records (sensors, the NaI input and the free inputs), in (detector, channel) order. | under development the dump's detector index -> DCRC board order (0,1,2 = DCRC1,DCRC3,DCRC4) is inferred, and channels 0 to 3 = board inputs pA to pD is assumed; not yet confirmed against the run's ODB |
channel_info(detector: int, channel: int, readout: int = 2) -> Channel | The Channel record for a dump (detector, channel) pair (readout 2 = Side-2 readout, the default). | under development the dump's detector index -> DCRC board order (0,1,2 = DCRC1,DCRC3,DCRC4) is inferred, and channels 0 to 3 = board inputs pA to pD is assumed; not yet confirmed against the run's ODB |
channel_label(detector: int, channel: int, readout: int = 2) -> str | Name with the board input and the HV pin, for plot labels: 'Side2pC (DCRC1 pA, HV2 pA)'. | under development the dump's detector index -> DCRC board order (0,1,2 = DCRC1,DCRC3,DCRC4) is inferred, and channels 0 to 3 = board inputs pA to pD is assumed; not yet confirmed against the run's ODB |
channel_name(detector: int, channel: int, readout: int = 2) -> str | Channel name, e.g. channel_name(0, 0) == 'Side2pC'; 'NaI' for the NaI trigger input. | under development the dump's detector index -> DCRC board order (0,1,2 = DCRC1,DCRC3,DCRC4) is inferred, and channels 0 to 3 = board inputs pA to pD is assumed; not yet confirmed against the run's ODB |
find_channel(name: str, readout: int = 2) -> tuple | The dump (detector, channel) of a channel name, e.g. find_channel('Side2pC') == (0, 0). | under development the dump's detector index -> DCRC board order (0,1,2 = DCRC1,DCRC3,DCRC4) is inferred, and channels 0 to 3 = board inputs pA to pD is assumed; not yet confirmed against the run's ODB |
sensor_channels(readout: int = 2) -> list | The Channel records of the sensors (6 in Side-2 readout), in (detector, channel) order. | under development the dump's detector index -> DCRC board order (0,1,2 = DCRC1,DCRC3,DCRC4) is inferred, and channels 0 to 3 = board inputs pA to pD is assumed; not yet confirmed against the run's ODB |
Development-status tagging for functions in this library.
Usage:
from pulse_status import status, DONE, UNDER_DEVELOPMENT
@status(DONE)
def pretrigger_mean(pulses, config=DEFAULT_CONFIG):
...
@status(UNDER_DEVELOPMENT, note="fraction=0.5 crossing heuristic, lightly validated")
def rise_sample(pulses, config=DEFAULT_CONFIG, fraction=0.5):
...
DONE functions are called normally. UNDER_DEVELOPMENT functions still run, but emit a UserWarning on every call (with the note, if given) so it's obvious in notebook output when a result depends on something not yet considered settled. Use list_statuses(module) to get an overview of a whole module at a glance.
| Function | Summary | Status |
|---|---|---|
status(level, note=None) | Decorator: tag a function with a development status.detailsAttaches | — |
list_statuses(module) | Return {function_name: (status, note)} for every status-tagged function defined in module (skips imports from other modules). | — |