Measures of Treatment Fidelity and Social Validity within a Parent-Mediated Behavior Intervention

Background: An effective intervention not only results in good outcomes, but has quality features of treatment fidelity (the accuracy and consistency of implementation of an intervention) and social validity (participants’ impressions of the importance and acceptability of the intervention, including satisfaction and therapeutic alliance, cf. Ardito & Rabellino, 2011. We examined these constructs within a quasi-experimental study of an 8-week parent training and therapeutic support program for children with ASD and problem behavior called COMPASS for Hope (C-HOPE). Despite the fact that individuals with ASD manifest higher levels of challenging behaviors and that parents are the first-line interventionists (Matson, Wilkins, & Macken, 2009), little parent training group-design research is available targeting these problem behaviors. Moreover, of the extant studies, few measured parent outcomes, none measured multiple parent outcome indicators – parent stress and parent competency – and few measured fidelity and social validity. Given the potential for parent training and support to decrease child problem behavior, increase parent competency, and decrease parent stress, an understanding of factors such as fidelity and social validity is critical for future dissemination.

Objectives: The purpose of this study was to examine (a) therapist treatment fidelity; (b) parent satisfaction; (c) parent outcomes; and (d) therapeutic alliance within C-HOPE. The larger study included both face-to-face (FF) and telehealth (TH) delivery formats, thus, a secondary objective was to determine whether differences exist between delivery methods.

Methods: The larger study employed a pre-post waitlist control design in a sample (N=33) of parents of children with ASD (M_age=8.1, SD_age=2.5). Participants received C-HOPE delivered via TH (N=20) or FF (N=13). Fidelity was measured using a checklist of essential session components. Satisfaction was measured using a 4-point Likert scale of session characteristics. Parent outcomes were measured using the Outcome Rating Scale (ORS; Miller & Duncan, 2000), which is divided into subscales of Individual, Interpersonal, Social, and Overall outcomes. Therapeutic alliance was measured using the Session Rating Scale (SRS; Johnson, Miller, & Duncan, 2000) and the Group Session Rating Scale (GSRS; Duncan & Miller, 2007), which are divided into subscales of Relationship, Goals/Topics, Approach/Method, and Overall. All ORS, SRS, and GSRS ratings are made using a 10-cm line visual analog scale.

Results: Therapist treatment adherence ranged from 76.2% to 100.0% (M=94.2, SD=7.1), and parent-reported satisfaction with sessions was high (M=3.7, SD=0.3). Parent outcome scores improved significantly from the first session (M=24.8, SD=8.8) to the eighth session (M=30.8, SD=8.5), t(17)=-3.71, p=0.002. Alliance was high with regard to parent-therapist relationship (M=9.3, SD=0.7), goals/topics (M=9.3, SD=0.8), approach/method (M=9.4, SD=0.6), and overall (M=9.2, SD=0.8). One-way analysis of variance revealed no differences between FF and TH modalities in the areas measured.

Conclusions: Social validity for C-HOPE is high, and can be implemented with fidelity across cohorts and modalities. Additionally, C-HOPE favorably impacted parents, who were both satisfied and demonstrated improvements individually, relationally, and socially. Future research is needed to assess how these factors impact final outcomes of child problem behavior, parent stress, and parent competency.