A Novel Hybrid Telehealth Approach for Assessing Social Communication in at-Risk Toddlers in Rural Communities from a Distance

Background: Despite evidence for the efficacy of ASD-specific early behavioral intervention in improving children’s outcomes, there remain significant barriers to disseminating empirically-validated treatments to families and community settings (Rogers & Talbott, 2016; Stahmer, Aranbarri, Drahota, & Rieth, 2017). Effectiveness trials demonstrating positive effects of a treatment in the final community setting are needed before public early intervention programs will cover such treatments. One central challenge to the conduct of such randomized controlled effectiveness studies is the measurement of proximal social communication behavioral treatment targets, which require live, reciprocal interactions, for toddlers living in rural and geographically distributed community settings.

Objectives: Our goal is to assess the feasibility, fidelity, validity, and reliability of one approach to meeting this need: a hybrid live-distance assessment, in which professionals working within local EI systems administer a play-based interaction, which is later scored from video by highly trained expert coders blind to study group at the University site.

Methods: Data were collected as part of an ongoing community-based pilot randomized controlled trial (RCT) testing an adapted Community-Early Start Denver Model (C-ESDM) across 4 states: Alabama, California, Colorado, and Pennsylvania. University partners in each state recruited 7 assessors (all female), who were graduate students (n=1) and early intervention professionals (n = 6) working in their local communities. Assessors received training on an adapted Community-ESDM Curriculum Checklist (CC) via telehealth and received feedback on submitted training tapes before conducting independent assessments. Assessments were scheduled in families’ homes, lasted approximately 1.5 hours, and were videotaped for later coding by raters trained to reliability and blind to families’ study group. Analyses examined the following 4 measures: Feasibility (proportion of scheduled visits completed); Fidelity (number of administrations meeting published CC administration and scoring rules); Inter-rater reliability (agreement between assessor and coding scores); and Validity (relations between assessor-derived CC scores and standardized and normed measures of social communication).

Results: 31 children (21 male) receiving low-intensity early intervention (EI) services by participating providers, and their caregivers, enrolled and had an intake assessment scheduled. An additional 3 families enrolled but withdrew before scheduling. Families completed 97% (30 of 31) of scheduled visits, indicating high feasibility. Preliminary analyses indicate acceptable fidelity levels for community assessors. Intra-class correlation coefficients for the Curriculum Checklist Proportional Score Totals between 21 coders and assessors was .87, indicating excellent interrater reliability. There were significant positive correlations between the MacArthur Bates total vocabulary inventory for both assessor (r = .57, p = .03, n = 15) and coder (r = .56, p = .03, n = 18) total scores.

Conclusions: While this method requires substantial efforts and staff time, it provides a potential route for increasing the participation of at-risk toddlers living in rural and low-resources areas into high-quality University-based treatment studies. This increases the acceptability and validity of the interventions and procedures.