goldpan.rqdata

The RQs of a whole list of series at once, as numpy arrays, with an on-disk cache: the second read is fast.

The call is that of the collaboration’s rqdata, so that an analysis can move over with its lines unchanged:

from goldpan.rqdata import load

RQS = ["EventType", "PTOFamps", "P*max", "P*std"]            # P* = all 12 phonon channels
data = load("24260424_145101", RQS)                         # a series name, a list of them, or a CSV with a `series` column
data["PTOFamps"]                                            # one numpy array over every event of every series
data["series"]                                              # which series each row came from (also "dump" and "event")
data.files                                                  # series -> the file that was read

Each branch comes back as one array with the rows of the files one after the other. A branch name is looked up in eventTree first, then in zip<N>; a branch a file does not have is filled with NaN for that file; a name with *, ? or [ is a pattern (see goldpan.adapters.rq_root.read, which does the reading), except P*, which is expanded first into the phonon channels (P*max is PAS1max, PAS2max, …). The channels are those of the Z7 detector unless you pass phonon_channels=.

Where the files are is not in this library. Say it once, in the environment, and the call above needs nothing more:

  • GOLDPAN_RQ_FILES (or files= in the call): a pattern with {series} in it, for example "/data/R*/Processed/Prompt/*/Submerged/*_{series}.root"; of the files that match, the last in name order is read (a series with one file per dump, like Run 76, is files="/data/{series}_F*.root" with all_files=True: every match is read, in name order);

  • GOLDPAN_RQ_DETECTOR (or detector=): the number N of the zip<N> tree.

The cache. load saves what it read in $GOLDPAN_CACHE/rqdata (default ~/.cache/goldpan/rqdata, a folder of your own, never inside a repository). A later load of the same files reuses any cached read that contains all the branches asked for, so asking for fewer RQs than before costs nothing; refresh=True reads again and cache=False neither reads nor writes the cache. Unlike a cache keyed on file names, this one is keyed on the state of the files (size and modification time) and on the version of goldpan: a file that was rewritten is read again, never served from an old entry. Entries are written to a temporary file and renamed, so a reader never sees half of one. cache_info() lists them and cache_clear() removes them.

Module Attributes

PHONON_CHANNELS

what P* stands for unless phonon_channels= is given.

Functions

cache_clear()

Delete every entry of the RQ cache; returns how many there were.

cache_dir()

The folder of the RQ cache: $GOLDPAN_CACHE/rqdata, by default ~/.cache/goldpan/rqdata.

cache_info([verbose])

The entries of the RQ cache as a list of {"file", "bytes", "modified"} (newest first); prints a summary unless verbose=False.

expand_branches(rqs[, phonon_channels])

rqs with every P* replaced by the phonon channels (P*max -> PAS1max, ...), each name once, in order.

find_file(series[, files])

The file of series: files is a pattern with {series} in it (default: $GOLDPAN_RQ_FILES), or a function of the series name that returns a path or None.

find_files(series[, files])

Every file of series, in name order: the matches of the pattern files (default: $GOLDPAN_RQ_FILES), or what the function files(series) returns (a path, a list of paths or None).

load(dataset, rqs[, detector, cache, ...])

All the requested RQs of every series of dataset, concatenated (see the module documentation).

series_list(dataset)

The series names of dataset: a series name ("24260424_145101"), a list of them, or the path of a CSV with a series column (lines starting with # are skipped).

Classes

RQData

dict of branch name -> numpy array (plus series, dump and event per row), with .skipped (series with no file), .files (series -> the file that was read) and .dataset (what was asked for).
goldpan.rqdata.PHONON_CHANNELS = ['PAS1', 'PAS2', 'PBS1', 'PBS2', 'PCS1', 'PCS2', 'PDS1', 'PDS2', 'PES1', 'PES2', 'PFS1', 'PFS2']

what P* stands for unless phonon_channels= is given.

Type:

The twelve phonon channels of Z7, in RQ naming

class goldpan.rqdata.RQData[source]

Bases: dict

dict of branch name -> numpy array (plus series, dump and event per row), with .skipped (series with no file), .files (series -> the file that was read) and .dataset (what was asked for).

skipped: tuple = ()
files: dict = {}
dataset = None
goldpan.rqdata.cache_dir()[source]

The folder of the RQ cache: $GOLDPAN_CACHE/rqdata, by default ~/.cache/goldpan/rqdata.

Return type:

str

goldpan.rqdata.series_list(dataset)[source]

The series names of dataset: a series name ("24260424_145101"), a list of them, or the path of a CSV with a series column (lines starting with # are skipped).

Return type:

list

goldpan.rqdata.find_file(series, files=None)[source]

The file of series: files is a pattern with {series} in it (default: $GOLDPAN_RQ_FILES), or a function of the series name that returns a path or None. Of several matches the last in name order is taken. None if there is no file.

Parameters:
Return type:

str | None

goldpan.rqdata.find_files(series, files=None)[source]

Every file of series, in name order: the matches of the pattern files (default: $GOLDPAN_RQ_FILES), or what the function files(series) returns (a path, a list of paths or None). The empty list if there is none. For a series that is stored as many files, such as the dumps of Run 76.

Parameters:
Return type:

list

goldpan.rqdata.expand_branches(rqs, phonon_channels=None)[source]

rqs with every P* replaced by the phonon channels (P*max -> PAS1max, …), each name once, in order.

Parameters:
Return type:

list

goldpan.rqdata.load(dataset, rqs, detector=None, cache=True, refresh=False, verbose=True, files=None, phonon_channels=None, all_files=False)[source]

All the requested RQs of every series of dataset, concatenated (see the module documentation).

dataset: a series name, a list of series names, or the path of a CSV with a series column. rqs: branch names, patterns and P* names. detector: the number N of the zip<N> tree (default $GOLDPAN_RQ_DETECTOR). files: where the files are (default $GOLDPAN_RQ_FILES), see find_file. cache=False bypasses the cache and refresh=True reads again and replaces the entry. all_files=True reads every file that matches a series (its dumps, in name order) instead of the last one; data.files then maps each series to a list of files. Series with no file are skipped and listed in data.skipped.

Parameters:
Return type:

RQData

goldpan.rqdata.cache_info(verbose=True)[source]

The entries of the RQ cache as a list of {"file", "bytes", "modified"} (newest first); prints a summary unless verbose=False.

Parameters:

verbose (bool)

Return type:

list

goldpan.rqdata.cache_clear()[source]

Delete every entry of the RQ cache; returns how many there were.

Return type:

int