Data quality — check before you analyse

Nothing in this module contributes to a physics result. These are the checks you run first, on a fresh acquisition, to answer one question: did the TDC actually record timing?

The failure mode is silent

When the ring-oscillator TDC never stops — or the readout is misconfigured — unpack still returns a full (32, 32, nframes) array of plausible-looking integers. Every downstream analysis still runs. The delta-t histogram still has a shape. It is simply meaningless.

There is no exception, no warning, and no obviously wrong number. The only way to catch it is to look at the raw code distribution.

What healthy data looks like

A working pixel's TDC codes spread over the full range for the reference clock in use — ~279 codes at 50 MHz, ~1397 at 10 MHz, ~2794 at 5 MHz, ~6985 at 2 MHz, since the ring sets the ~71.6 ps code width and the clock sets the full scale. A broken run collapses onto a handful of values — often a single code repeated across every frame.

from dapkel.functions import data_quality

fig, codes = data_quality.plot_time_code_histogram(
    folder, pixel=(16, 16), nframes=10_000
)
print(codes.size, "valid codes,", len(set(codes)), "unique")

A few unique values out of thousands of frames means the run is unusable. This mirrors the timestamp inspection in Kelpie_run.m.

Run it on a pixel or two before spending time on coincidences.

API

data_quality

Data-quality checks to run on a fresh Kelpie acquisition.

Nothing here contributes to a physics result - these answer one question before you spend time on analysis: did the TDC actually record timing?

The failure mode is silent. When the TDC never stops, 'unpack' still returns a full array of plausible integers and every downstream analysis still runs on meaningless data. Healthy codes spread over the full code range of the clock in use (~279 codes at 50 MHz, ~1397 at 10 MHz, ~6985 at 2 MHz); a broken run collapses onto a handful of values.

See docs/guide/data_quality.md.

collect_time_mat

collect_time_mat(folder: str, pixels: Pixel | Sequence[Pixel], *, nframes: int, tag: str = 'ORT', max_files: int | None = None) -> dict[Pixel, ndarray]

Collect the per-frame TDC codes (time_mat) for given pixels.

Returns, per requested pixel, the stack of its per-frame TDC codes over every file, keeping only frames with a valid timestamp - 'unpack' leaves non-fired slots at <= 0, so that is time_series > 0.

Parameters:

Name	Type	Description	Default
folder	str	Path to the folder with the '.bin' data files.	required
pixels	tuple[int, int] | Sequence[tuple[int, int]]	A single (row, col) pixel or a sequence of them.	required
nframes	int	Number of frames stored in each '.bin' file.	required
tag	str	Filename fragment selecting the files. The default is 'ORT'.	'ORT'
max_files	int | None	Process at most this many files. The default is None.	None

Returns:

Type	Description
dict[tuple[int, int], ndarray]	Mapping (row, col) -> 1D array of valid TDC codes.

Source code in src/dapkel/functions/data_quality.py

def collect_time_mat(
    folder: str,
    pixels: Pixel | Sequence[Pixel],
    *,
    nframes: int,
    tag: str = "ORT",
    max_files: int | None = None,
) -> dict[Pixel, np.ndarray]:
    """Collect the per-frame TDC codes (``time_mat``) for given pixels.

    Returns, per requested pixel, the stack of its per-frame TDC codes over
    every file, keeping only frames with a valid timestamp - 'unpack' leaves
    non-fired slots at ``<= 0``, so that is ``time_series > 0``.

    Parameters
    ----------
    folder : str
        Path to the folder with the '.bin' data files.
    pixels : tuple[int, int] | Sequence[tuple[int, int]]
        A single ``(row, col)`` pixel or a sequence of them.
    nframes : int
        Number of frames stored in each '.bin' file.
    tag : str, optional
        Filename fragment selecting the files. The default is ``'ORT'``.
    max_files : int | None, optional
        Process at most this many files. The default is None.

    Returns
    -------
    dict[tuple[int, int], np.ndarray]
        Mapping ``(row, col) -> 1D array`` of valid TDC codes.
    """
    pix = cd.as_pixel_list(pixels)
    files = io.find_bin_files(folder, tag)
    if max_files is not None:
        files = files[:max_files]

    chunks: dict[Pixel, list[np.ndarray]] = {p: [] for p in pix}
    for fp in tqdm(files, desc=tag or "time_mat"):
        ts, _ = unpack(fp, nframes, compute_time_series=True)
        for r, c in pix:
            code = ts[r, c]
            chunks[(r, c)].append(code[code > 0])

    return {
        p: (np.concatenate(v) if v else np.empty(0, dtype=np.float64))
        for p, v in chunks.items()
    }

plot_time_code_histogram

plot_time_code_histogram(folder: str, pixel: Pixel, *, nframes: int, tag: str = 'ORT', bins: int | ndarray = 256, time_unit_s: float | None = None, max_files: int | None = None) -> tuple[Figure, ndarray]

Histogram one pixel's TDC codes over all frames (QA / jitter view).

Mirrors the Kelpie_run.m timestamp inspection. Useful to confirm, on a fresh acquisition, that the timing is populated (codes should span the full full code range for the reference clock in use - ~279 codes at 50 MHz, ~1397 at 10 MHz, ~6985 at 2 MHz - not collapse onto a few values).

Parameters:

Name	Type	Description	Default
folder	str	Path to the folder with the '.bin' data files.	required
pixel	tuple[int, int]	The (row, col) pixel to inspect.	required
nframes	int	Number of frames stored in each '.bin' file.	required
tag	str	Filename fragment selecting the files. The default is 'ORT'.	'ORT'
bins	int | ndarray	Bins for np.histogram. The default is 256.	256
time_unit_s	float | None	Seconds per code for the x-axis (the axis itself is drawn in ps); None (default) keeps raw codes.	None
max_files	int | None	Process at most this many files. The default is None.	None

Returns:

Name	Type	Description
fig	Figure	The generated figure.
codes	ndarray	The pixel's valid TDC codes over all frames.

Source code in src/dapkel/functions/data_quality.py

def plot_time_code_histogram(
    folder: str,
    pixel: Pixel,
    *,
    nframes: int,
    tag: str = "ORT",
    bins: int | np.ndarray = 256,
    time_unit_s: float | None = None,
    max_files: int | None = None,
) -> tuple[plt.Figure, np.ndarray]:
    """Histogram one pixel's TDC codes over all frames (QA / jitter view).

    Mirrors the ``Kelpie_run.m`` timestamp inspection. Useful to confirm, on a
    fresh acquisition, that the timing is populated (codes should span the full
    full code range for the reference clock in use - ~279 codes at 50 MHz,
    ~1397 at 10 MHz, ~6985 at 2 MHz - not collapse onto a few values).

    Parameters
    ----------
    folder : str
        Path to the folder with the '.bin' data files.
    pixel : tuple[int, int]
        The ``(row, col)`` pixel to inspect.
    nframes : int
        Number of frames stored in each '.bin' file.
    tag : str, optional
        Filename fragment selecting the files. The default is ``'ORT'``.
    bins : int | np.ndarray, optional
        Bins for ``np.histogram``. The default is 256.
    time_unit_s : float | None, optional
        Seconds per code for the x-axis (the axis itself is drawn in ps); None
        (default) keeps raw codes.
    max_files : int | None, optional
        Process at most this many files. The default is None.

    Returns
    -------
    fig : plt.Figure
        The generated figure.
    codes : np.ndarray
        The pixel's valid TDC codes over all frames.
    """
    r, c = int(pixel[0]), int(pixel[1])
    codes = collect_time_mat(
        folder, (r, c), nframes=nframes, tag=tag, max_files=max_files
    )[(r, c)]

    scale = time_unit_s * 1e12 if time_unit_s else 1.0
    unit = "ps" if time_unit_s else "TDC code"

    fig, ax = plt.subplots()
    if codes.size:
        ax.hist(codes * scale, bins=bins, alpha=0.75)
    ax.set_xlabel(f"Timestamp  ({unit})")
    ax.set_ylabel("Counts")
    ax.set_title(
        f"Pixel ({r},{c})  time codes  (n={codes.size}, "
        f"{np.unique(codes).size} unique)"
    )
    ax.grid(True, linewidth=0.5, alpha=0.6)
    fig.tight_layout()
    return fig, codes