Eye Tracking Test-Retest Reliability As an Effect of Diagnosis and Delta Time: Results from the ABC-CT Interim Analysis

Background: A core aim of the Autism Biomarkers Consortium for Clinical Trials (ABC-CT) is to identify biomarkers which can reliably measure treatment effects in autism spectrum disorder (ASD). There are many qualifying characteristics to the ideal biomarker, including robustness across a heterogeneous population and reliability across time. To address reliability, the project collected mirrored data at three different timepoints. This study analyzes the first two. ABC-CT protocols targeted Timepoint 2 (T2) to be scheduled 28-56 days after T1. However, 12.4% of participants had their T2 outside the standard range (Table 1).

Objectives: To evaluate the effect of the time between visits (Δdays) on T1 to T2 relationships on eye-tracking (ET) biomarker variables for typically-developing (TD) participants and participants with ASD.

Methods: Participants were 225 6- to 12-years-old children (TD: n=64; ASD: n=161). There was no average between-group difference in Δdays (p>.05). However, more scheduling deviations occurred in the ASD group (p=.03).

Participants viewed a battery of ET paradigms including activity monitoring (AM), biomotion (BM), pupillary light reflex (PLR), social interaction (SI), static scenes (SS), and visual search (VS). Outcome variables included total valid looking percent per paradigm (ValidLooking%) and the ratios of valid looking time spent looking at social information versus non-social or background information.

Reliability was tested by calculating Pearson’s correlations between T1 and T2 for each variable. Effects of Δdays was examined in a linear model: T2 ~ T1 * group * Δdays. We also examined effects of time as a categorical variable by separating the extreme 10^th percentiles as Earlier (< 30 days) and Later (>55 days) scheduling for Δdays.

Results: (Table 2) Correlations from T1 to T2 performance were highly significant (p < .001) for all variables across the entire sample and within ASD; most variables (86%) were correlated within TD.

Similarly, linear models showed all T2 outcomes were strongly associated with initial T1 performance even when controlling for the other variables. Six variables showed significant main effects of diagnosis beyond the influence of T1 performance. Among these, five were ValidLooking% variables, generally consistent with diminished looking by children with ASD in T2.

Most variables showed non-significant main effects and higher order interactions for either time variable except for AM_head% (T1*dx*Δdays) and BM_affective% (dx*Δdays_categorical) suggesting these two variables demonstrate more complex relationships between group and time between testing.

Conclusions: Strong correlations combined with the highly significant effect of T1 results on T2 outcome demonstrate the variables’ reliability as well as robustness across heterogeneous populations.

Significant diagnosis effects suggested that individual paradigms affected attentional patterns during second viewings differently per diagnostic group. The fact that the majority of diagnostic effects are seen in ValidLooking% suggests variables normalized by total looking duration may be more stable in ASD.

Overall, these analyses demonstrate that the vast majority of ABC-CT’s variables have strong test-retest characteristics which remain reliable across a-priori time ranges and are additionally unaffected by linear effects of scheduling time variation as measurable by the current sample’s level of variability.