Development and Preliminary Validation of a New Scale to Measure the Social Validity of Skill Building Interventions for Autism Spectrum Disorder

Background:  There is growing evidence that evidence-based interventions for ASD are often not adopted or successfully implemented in community settings.  Research suggests that social validity (i.e., the social importance of treatment procedures/outcomes) may strongly influence both the adoption and implementation of treatments. Thus, it is imperative to develop a better understanding of community stakeholders’ views of different evidence-based interventions for children with ASD. While there are established scales for examining the social validity of interventions designed to decrease unwanted behaviors, there are currently no tools for assessing the social validity of skill-building interventions.

Objectives:  The primary purpose of this study is to refine a newly developed scale meant to assess the social validity of skill-building interventions for individuals with ASD, and to examine the scale’s psychometric properties. The secondary purpose of the study is to use the scale to examine how consumers view different evidence-based, comprehensive interventions for ASD.

Methods:  A 35-item measure (Scale of Treatment Perceptions; STP) was constructed to assess the social validity of skill-building interventions for individuals with ASD. In Phase 1, 600 undergraduate students read a vignette describing a young child with ASD, and one of four different comprehensive treatment descriptions (LEAP, EIBI, TEACCH, Floortime). Responses were submitted to item analysis and exploratory factor analysis. The scale was refined to improve the structure and reduce number of items. In Phase 2, 500 parents of young children unfamiliar with ASD interventions were recruited from Amazon mTurk and subjected to the Phase 1 procedure (using the refined version of the STP). This allowed us to investigate the robustness of the psychometric structure of the STP when including individuals who are potential consumers of early intervention. In each phase of the study we also examined the relative social validity of the four interventions as measured by the STP.

Results:  Results from both Phase 1 and 2 provided support for a four-factor structure of the scale (acceptability, effectiveness, family impact, and potential risks). Internal reliability was excellent. In both college students and parents, the scale successfully discriminated between treatments both in terms of overall score and across individual factors.

Conclusions:  The STP appears to have adequate psychometric properties for examining the social validity of skill-building treatments for ASD. The ability of the scale to evaluate social validity across four dimensions is a considerable strength, as it allows researchers to investigate social validity at a more nuanced level. Findings indicating that well-established treatments targeting skill development in individuals with ASD differ in their overall level of social validity and across these four factors suggest that the STP can be effectively utilized to compare the social validity of existing, evidence-based, interventions. Differences in the social validity of ASD treatments has significant implications for dissemination practices.