goldpan.adapters.midas

The MIDAS adapter: raw SuperCDMS SNOLAB DAQ files (.mid, .mid.gz, .mid.lz4) into an Events. Read only.

A file is a sequence of MIDAS events. The first (id 0x8000, begin of run) carries the ODB, the experiment’s settings, as XML; the last (id 0x8001, end of run) carries it again; each event in between (id 1) holds a bank SCD0 with one trigger in the SNOLAB data format (the CDMS I/O library, CDMSIOLIB::pack_event; this reader follows unpack_event, format versions 4 to 8) and sometimes a veto bank LDV0. Polling-cycle events (trigger type 8) and events of other detectors carry no traces and are skipped. The format is described at https://gitlab.com/supercdms/DAQ/IOLibrary (README, “Description of Data Structures”).

Like the idx adapter, this one does not know your detector: you say which detector number and which channel numbers are the named channels, and you name the readout, which is checked against the sample rate in the file:

from goldpan.adapters import midas
from goldpan.config import Z7
ev = midas.read("24260424_145101_F0001.mid.lz4", detector_name="Z7", detector_id=7, config=Z7, max_events=100,
                channels={"PAS1": 0, "PAS2": 1, "PBS1": 2})              # channel name -> channel number

  • channels are phonon channels (channel_type=1; pass channel_type=0 for charge). The traces keep the file’s uint16.

  • Only the channels you name are copied; the others are read past, so a 15 GB file costs a pass over it. Use max_events= to stop early, or iter_chunks to process a file in pieces.

  • Event quantities: series and dump (from a file name like <series>_F0003.mid.lz4, or series=/dump=), event (the event number), trigger_id, trigger_type (as text: “Physics”, “BeginOfRunRandom”, …), readout_type and global_timestamp.

  • meta: run (run number, state, start and stop time from the ODB), complete (the end-of-run event was seen: a file cut short is not complete), skipped_events, format_version, source.

  • Hybrid sampling (pre- or post-pulse samples at a coarser rate) is not read yet: such a file raises NotImplementedError.

  • detectors(path) lists what the ODB says about each detector (number, name, type, tower, plane, the readout lengths), and registry_entry writes a starting entry for goldpan.registry from it. The ODB is read as a source of information about the detector, never required: read takes the detector number and the channels from you.

.mid.lz4 files need the lz4 package (pip install lz4); .mid.gz and plain .mid need nothing.

Module Attributes

REVISION

Raised by one when a change of this reader changes what it returns for the same file (a fix), so that goldsluice run redoes the outputs made with the old one (goldpan.adapters.revision).

Functions

detectors(path)

What the begin-of-run ODB of a MIDAS file says about each detector: the OdbDetector list of parse_odb_detectors.

iter_chunks(path, *[, chunk, roles, ...])

Yield the events of read in pieces of at most chunk events, reading the file once from the start.

iter_events(path)

The MIDAS events of a file one by one, as RawEvent(event_id, mask, serial, timestamp, data) with data the bytes after the 16-byte header.

parse_odb(xml)

The run information of an ODB (XML bytes of a begin- or end-of-run event), from its /Runinfo directory: run_number, state (as stored; the real begin- and end-of-run dumps show 1 and 3), start_time and stop_time (unix seconds; 0 while running).

parse_odb_detectors(xml)

The OdbDetector of every /Detectors/Det<NN> block of an ODB (XML bytes of a begin-of-run event), in order.

read(path, *, detector_name, detector_id, ...)

The events of a MIDAS file in which detector_id has the channels (name -> channel number), as Events.

registry_entry(detector, *[, origin])

A starting entry for a goldpan.registry file from what the ODB says about detector, as TOML text.

unpack_scd0(data[, want, channel_type])

The trigger in the SCD0 bank of a MIDAS data event.

Classes

OdbDetector(number, name, type_code, tower, ...)

What the ODB of a file says about one detector (/Detectors/Det<NN>).

RawEvent(event_id, mask, serial, timestamp, data)

Exceptions

MidasFormatError

The file does not follow the SNOLAB data format (the message says where).
goldpan.adapters.midas.REVISION = 1

Raised by one when a change of this reader changes what it returns for the same file (a fix), so that goldsluice run redoes the outputs made with the old one (goldpan.adapters.revision). Not for a change that leaves the results alone.

class goldpan.adapters.midas.RawEvent(event_id, mask, serial, timestamp, data)

Bases: tuple

data

Alias for field number 4

event_id

Alias for field number 0

mask

Alias for field number 1

serial

Alias for field number 2

timestamp

Alias for field number 3

exception goldpan.adapters.midas.MidasFormatError[source]

Bases: ValueError

The file does not follow the SNOLAB data format (the message says where).

goldpan.adapters.midas.iter_events(path)[source]

The MIDAS events of a file one by one, as RawEvent(event_id, mask, serial, timestamp, data) with data the bytes after the 16-byte header. A file that stops in the middle of an event ends there without an error.

Return type:

Iterator[RawEvent]

goldpan.adapters.midas.parse_odb(xml)[source]

The run information of an ODB (XML bytes of a begin- or end-of-run event), from its /Runinfo directory: run_number, state (as stored; the real begin- and end-of-run dumps show 1 and 3), start_time and stop_time (unix seconds; 0 while running). Keys that are not found are left out; the rest of the ODB (tens of megabytes) is not parsed.

Return type:

dict

class goldpan.adapters.midas.OdbDetector(number, name, type_code, tower, plane, readout)[source]

Bases: NamedTuple

What the ODB of a file says about one detector (/Detectors/Det<NN>).

Parameters:
number: int

The NN of Det<NN>; the N of zip<N> in RQ files.

name: str | None

DCRCInfo/DetectorName, for example G149.

type_code: int | None

710 and 711 are Ge and Si HV, 700 and 701 iZIP.

Type:

DCRCInfo/DetectorType

tower: int | None

Alias for field number 3

plane: int | None

Alias for field number 4

readout: Mapping[str, float]

phonon_on_pulse_ms, phonon_on_pulse_samples, phonon_pre_trigger_offset_ms, phonon_pre_trigger_offset_samples, phonon_pre_pulse_ms, phonon_pre_pulse_samples, phonon_post_pulse_ms, phonon_post_pulse_samples (pre- and post-pulse are the down-sampled regions of hybrid readout).

Type:

The phonon lengths of ReadoutControl that the block has, by name

goldpan.adapters.midas.parse_odb_detectors(xml)[source]

The OdbDetector of every /Detectors/Det<NN> block of an ODB (XML bytes of a begin-of-run event), in order. Only the DCRCInfo and ReadoutControl keys are read, by search, not by parsing the tens of megabytes; a key a block lacks is None (or missing from readout). An ODB without a /Detectors directory gives an empty list.

Return type:

list

goldpan.adapters.midas.detectors(path)[source]

What the begin-of-run ODB of a MIDAS file says about each detector: the OdbDetector list of parse_odb_detectors. Reads only up to the first event (the ODB); ValueError if the file does not begin with one.

Return type:

list

goldpan.adapters.midas.registry_entry(detector, *, origin='the begin-of-run ODB of a MIDAS file')[source]

A starting entry for a goldpan.registry file from what the ODB says about detector, as TOML text. It states what the ODB gives (number, name, type, tower, plane) as confirmed; the readout, the channels and the series are for you to add.

Parameters:
Return type:

str

goldpan.adapters.midas.unpack_scd0(data, want=None, channel_type=1)[source]

The trigger in the SCD0 bank of a MIDAS data event.

Returns dict(format_version, trigger_id, event_number, trigger_type, readout_type, global_timestamp, detectors) where detectors maps a detector number to {channel_number: dict(rate_khz, downsampling, pretrigger, pre, on, post, data)}. want ({detector: {channel numbers}}) keeps the waveforms of those channels of type channel_type only (the others are read past); None keeps every channel. Returns None for an event without an SCD0 bank.

Parameters:
Return type:

dict

goldpan.adapters.midas.read(path, *, detector_name, detector_id, channels, config, roles=None, channel_type=1, series=None, dump=None, max_events=None)[source]

The events of a MIDAS file in which detector_id has the channels (name -> channel number), as Events.

config is the readout of the data; its sample period is checked against the file. max_events stops after that many events (a whole file can be 15 GB). meta["complete"] is True only if the end-of-run event was reached.

Parameters:
Return type:

Events

goldpan.adapters.midas.iter_chunks(path, *, chunk=256, detector_name, detector_id, channels, config, roles=None, channel_type=1, series=None, dump=None, max_events=None)[source]

Yield the events of read in pieces of at most chunk events, reading the file once from the start. Every chunk’s meta["run"] has the begin-of-run information; whether the file is complete is only known to read.

Parameters:
Return type:

Iterator[Events]