Treatment Fidelity and the Mainstream Environment: Predictors of Successful Outcomes for ASD Youth

Background: Thirty-seven percent of students with an autism spectrum disorder (ASD) spend the majority of the school day in the mainstream environment, making inclusion classrooms the optimal setting for the delivery of ASD interventions (U.S. Department of Education, 2015). However, the increased executive and social demands present within this environment pose major challenges for the dissemination of evidence-based treatments (Humphrey & Lewis, 2008). Therefore, it is critical for researchers to measure the extent to which interventions are being delivered as intended within the school setting, and utilize this information to make treatment manuals more accessible, identify optimal levels of treatment fidelity, and enhance student outcomes.

Objectives: The purpose of this study is to 1) evaluate the role of treatment fidelity and interventionist characteristics on student behavioral outcomes; 2) examine the relationship between fidelity and the experience/training of the interventionist; and 3) determine group differences in fidelity, interventionist characteristics, and student outcomes. 

Methods: All children had IQ>70 (M=108.45, SD=18.01), met ASD criteria on the ADOS or ADI, and were in the 3^rd–5^th grade (M=9.52, SD=1.02). Participating schools (N=14) were randomly assigned to either the UOT intervention (10 schools; 16 interventionists; 47 students) or a social skills training program (4 schools; 7 interventionists; 20 students).

Fidelity observations were used to evaluate the quality of treatment implementation across the 23 intervention groups (e.g., intervention adherence, group/time management, student engagement, etc.). Classroom observations were conducted to measure pre-post changes in real-world functioning (i.e., cognitive flexibility, planning, and social appropriateness). All observations were conducted by a blind evaluator 2-3 times throughout the school year (~15 minutes each). The Challenge Task (CT), an ADOS-like interview designed to measure executive functioning, was also used as a measure of pre-post change in flexibility, planning, and social appropriateness. Independent samples t-tests were used to evaluate group differences. Correlational analyses were used to measure the relationship between fidelity and interventionist experience/training. Longitudinal linear models with random effects for school and student were used to measure the effect of fidelity on student outcomes.

Results: There were significant group differences in treatment fidelity (t=3.45, p<.01), acquired knowledge (t=-2.24, p<.05), years of experience (t=3.2, p<.01), with UOT interventionists demonstrating higher levels of acquired knowledge (M=70.31, SD=18.42) and SST interventionists adhering to higher levels of fidelity (M=86.92, SD=5.14) and having more years of experience (M=16.20, SD=12.04). In comparison to SST, UOT groups also demonstrated greater improvements in CO Plan (t=-6.97, p<.001), CO Flex (t=-11.02, p<.001), and CO Social (t=-6.44, p<.001). Treatment fidelity was correlated with higher levels of acquired knowledge, r=.40, p<.05. There was no significant effect of treatment fidelity, years of experience, or acquired knowledge on student outcomes.

Conclusions: Results suggest that high levels of treatment fidelity and professional experience are not required in order to achieve successful treatment outcomes in the mainstream educational setting. While additional research is needed in this area, these data provide support for the training of school personnel to implement evidence-based practices for students with ASD.