Concurrent Validity and Stability of Diagnosis Using Three Measures of ASD Symptom Severity

Background:  Proposed changes to the forthcoming Diagnostic and Statistical Manual of Mental Disorders would include severity criteria for the Autism Spectrum Disorders (ASD) category. Currently, the Childhood Autism Rating Scale (CARS) and Social Responsiveness Scale (SRS) are two readily used measures of symptom severity. In addition, the Autism Diagnostic Observation Schedule (ADOS) was recently standardized to yield a severity score. Evaluating symptom severity could promote further understanding of the clinical phenotype, response to intervention, and underlying neurobiology of the disorder. The primary purpose of this study was to examine if children’s symptom severity was similarly categorized across the three measures.

Objectives:  To examine (1) concurrent validity of the ADOS, CARS, and SRS; (2) categorization of children’s diagnostic status or symptom severity across measures; and (3) stability of their diagnostic status.

Methods:  Data for this study were collected on 140 children (ages 3 – 5) as part of a larger study comparing the efficacy of school-based, comprehensive treatment models for preschoolers with ASD. The reported data were collected across four states (CO, NC, FL, and MN), and at the beginning and end of one school year (range of 6-9 months between assessment time points).Trained and reliable research staff administered the ADOS and CARS. Parents of children enrolled in the study completed the SRS. Correlational analysis was used to determine concurrent validity, cross-tabulations were performed to examine children’s categorizations across measures, and weighted kappas were used to examine diagnostic stability.

Results:  The ADOS severity score was significantly correlated with the CARS total score (r = .47) but not the SRS total score (r = .09). The CARS and SRS were significantly correlated with each other (r = .26). Cross-tabulations were performed to examine children’s diagnostic and severity classification across the three measures. Forty-five percent of the sample was similarly classified across all three measures, 24% on the ADOS and SRS but not CARS, 16% on the ADOS and CARS but not SRS, 13% on the ADOS but not CARS or SRS, and 2% on the SRS but not ADOS or CARS. In examining longitudinal data, children’s scores on each measure at the beginning of the school year were significantly correlated with their end of the year scores; the weighted kappas values were (ADOS, k = .35; CARS, k = .61; SRS, k = .61). (Note: Per Chlebowski et al. 2010, the CARS cut-off score of 25 was used).

Conclusions:  Diagnostic status and symptom severity is a mostly stable construct across measures. Children’s scores on the measures were correlated with the exception of the SRS and ADOS. This may reflect method variance in that parents completed the SRS; however, research staff completed the ADOS and CARS. However, for the near majority of the sample, children were similarly classified across the three measures. Further, children’s diagnostic status remained largely unchanged from the beginning to the end of the school year; although kappa values were lower for the ADOS this was primarily an artifact of using weighted kappas.