Time-walk Correction: Apply

Luna outputs timewalk correction matrix and lookup tables for correcting the ToA based on the magnitude of the ToT and a relationship inferred from your data. That relationship can be examined to refine your curve by adjusting the ctot-cut parameter.

However once you are happy with your lookup table application of the correction is a simple matter of lookup and subtraction.

Luna Command

tpx3dump process -i /Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/example_data.tpx3 -o /Users/Ciaran/atlassian-bitbucket-pipelines-runner/temp/e71169e4-520a-5b30-a5ab-ee8a44eb5fac/build/docs/source/_static/example_data.hdf5  --eps-t 150ns --eps-s 1 --ctot-cut 500

Python Script

Apply the timewalk correction

import os, sys
from typing import *
import lmfit
import pandas as pd  # ensure you have `pip install pandas`
import warnings
import seaborn as sns
import matplotlib.pyplot as plt
from matplotlib import cm
from matplotlib.colors import Normalize

warnings.filterwarnings("ignore")  # suppress warnings from plotting libraries.

sns.set_context(context="talk")

# add some paths to PYTHONPATH
for directory in ["..", "."]:
    sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), directory)))

# on our system "EXAMPLE_DATA_HDF5" refers to the absolute path
# to a hdf5 file generated by luna. Replace with your own!
from env_vars_for_docs_examples import EXAMPLE_DATA_HDF5, PLOTS_DIRECTORY

# re-use functions from previous example
from ex2_read_data_time_units import load_pixel_hits, load_timewalk_lookup_table, TimeUnit


def apply_timewalk_correction(pixel_data: pd.DataFrame, timewalk_lut: pd.DataFrame):
    """Takes the timewalk look up table (LUT) that was calculated by luna (using some
    ctot-cut parameter x) and uses it to correct ToA for timewalk effects
    """
    pixel_data["corrected_toa"] = pixel_data["toa"] - pixel_data["tot"].map(timewalk_lut["AverageDToA"])
    return pixel_data


def plot_lookup_table(timewalk_lut: pd.DataFrame, toa_unit: TimeUnit):
    """Plot a scatter graph with error bars (Std) of the timewakl lookup table"""
    fig = plt.figure()
    print(timewalk_lut)
    timewalk_lut = timewalk_lut.dropna(subset=["AverageDToA", "Std"])
    plt.errorbar(timewalk_lut.index, timewalk_lut["AverageDToA"], yerr=timewalk_lut["Std"],
                 marker="x", color="black", linewidth=1, linestyle="None" )
    plt.xlabel("ToT (ns)")
    plt.ylabel(f"Mean dToA \n({toa_unit})\n (Err=STD))")
    plt.title("Effect of ToT magnitude on ToA")

    sns.despine(fig=fig)

    fname = os.path.join(PLOTS_DIRECTORY, f"ex7_correct_toa_for_timewalk.png")
    plt.savefig(fname, bbox_inches="tight", dpi=300)


if __name__ == "__main__":
    toa_unit = TimeUnit.Nanoseconds
    pixel_data: pd.DataFrame = load_pixel_hits(EXAMPLE_DATA_HDF5, toa_unit)
    timewalk_lut: Optional[pd.DataFrame] = load_timewalk_lookup_table(EXAMPLE_DATA_HDF5, toa_unit)

    if timewalk_lut is not None:
        pixel_data = apply_timewalk_correction(pixel_data=pixel_data, timewalk_lut=timewalk_lut)

        plot_lookup_table(timewalk_lut=timewalk_lut, toa_unit=toa_unit)

        print("Timewalk corrected ToA's")
        print(pixel_data.head(15).to_string())

Script Output

Example Output

hdf5 datasets: ['Clusters', 'ExposureTimeBoundaries', 'PixelHits', 'TimewalkLookupTable', 'TimewalkMatrix']
      AverageDToA  SumSquareDiff        Std  Count
ToT                                               
25      76.245934    803962240.0  23.546906    146
50      70.257935    642109440.0  20.419456    155
75      60.629959    549626880.0  18.476557    162
100     50.204540    444413952.0  16.562899    163
125     43.402039    267351552.0  12.729150    166
...           ...            ...        ...    ...
6200     0.000000            0.0   0.000000      1
6400    -1.562500            0.0   0.000000      1
6675     0.000000            0.0   0.000000      1
6825     0.000000            0.0   0.000000      1
6850     0.000000            0.0   0.000000      1

[212 rows x 4 columns]
Timewalk corrected ToA's
             toa  corrected_toa  cid    dtoa   tot    x    y
0   1.103455e+10   1.103455e+10    0       0   200  140  193
1   1.103455e+10   1.103455e+10    1       0   150  143  193
2   1.103459e+10   1.103459e+10    2       0  1475   68   92
3   1.103459e+10   1.103459e+10    2   15625   850   67   92
4   1.103459e+10   1.103459e+10    2  218750   300   68   91
5   1.103459e+10   1.103459e+10    2  453125   150   67   91
6   1.103464e+10   1.103464e+10    3       0   800   80   80
7   1.103464e+10   1.103464e+10    3  218750   250   80   81
8   1.103464e+10   1.103464e+10    3  296875   250   79   80
9   1.103470e+10   1.103470e+10    4       0   375   14   18
10  1.103470e+10   1.103470e+10    4   31250   375   14   19
11  1.103493e+10   1.103493e+10    5       0   375   49    5
12  1.103506e+10   1.103506e+10    6       0  1675  146  106
13  1.103506e+10   1.103506e+10    6   78125   425  146  105
14  1.103506e+10   1.103506e+10    6  171875   350  145  106

Note

Since we are (necessarily) using a small tpx3 file for the examples there is some instability between the ToT=2000ns - 4000ns. This is because the time walk lookup table is inferred directly from your data and since this dataset is minimal there may not be enough data to fully inform the timewalk matrix.