Development of a Video-Based Coding Tool: Autism Behavioural Change Observational Measure (ABCOM) to Monitor Response to Intervention in School-Age Children with Autism

Background:

Evaluating outcomes effectively is a significant challenge for intervention research in Autism Spectrum Disorder (ASD). Although many tools exist for monitoring response to intervention in toddlers, few tools exist for the school-age population (Cunningham, 2012). Commonly used measures such as the ADOS were designed for diagnostic purposes, are expensive and require highly-specialized training. The Brief Observation of Social Communication Change (BOSCC; Grzadzinski et al, 2016) was specifically developed to measure changes in social-communication in ASD but is limited to early intervention and depends on caregiver-child interactions during the ADOS. To date, there is no treatment-response tool for school-age children that has applicability across multiple intervention settings, allows for reliable description of functional skills across several clinically-relevant domains, and has the potential to be sensitive to changes in social-communication.

Objectives: The purpose of this study was to develop a flexible video observation-based behaviour coding instrument that may be used in multiple settings (e.g., therapist-child or educator-child play or therapy sessions) for school-age children with ASD to measure response to treatment. Crucially, the tool is designed to evaluate behaviors within a therapy session itself, and therefore does not require the administration of a separate assessment.

Methods: We developed a 27-item questionnaire called the Autism Behavioural Change Observational Measure (ABCOM) to monitor autism-relevant outcomes in school-age children with ASD. The ABCOM (Figure1) is designed to be used in naturalistic settings involving one-on-one interactions with an interventionist/educator. The items are categorized under 6 domains (Table1) which are rated on a likert-scale of 0(never/rarely applies) to 3(almost always applies) based on frequency of behaviours. We used 45-minute video-recordings from a completed randomized control trial (RCT: ISRCTN26821793) comparing music and non-music intervention in 6-12 year-old children with ASD, to develop and test the tool. Two experienced raters were trained using a standardized procedure (lasting 15 hours) using 3 example videos and a written manual to achieve acceptable levels of inter-rater reliability (measured using Intraclass Correlation Coefficient, ICC). 9 videos were coded by both raters till they met intended coding targets. 10 test videos were coded by each rater to measure inter-rater reliability for subsequent independent coding of videos to evaluate treatment-related change in each intervention group.

Results: After initial training and coding of 9 videos, both raters reached an ICC=0.92, p=.001. On the subsequent 10 test videos, raters had an ICC=0.95 (p<.001). Domain-wise ICC and internal consistency values (measured using Cronbach’s alpha) are presented in Table1, suggesting high reliability and internal consistency for initiation of communication, shared social interaction and challenging behaviour domains, and acceptable levels of reliability for other domains.

Conclusions: The ABCOM has moderately high inter-rater reliability and consistency and maybe applied in multiple intervention settings to measure autism-related outcomes such as shared social interaction, initiation of communication and occurrence of challenging behaviours in school-age children. Though the ABCOM requires some training of coders, it removes the burden of administering a separate outcome assessment. In ongoing work we are applying this tool to measure the outcomes of the full music vs. non-music RCT.