28129
Testing the Limits of the SRS-2: Understanding Psychometric Limitations and How the Revised Short Form May Help in Some Ways
Because of its perceived dimensional, quantitative structure, the Social Responsiveness Scale (SRS; SRS-2) is widely employed in behavioral and biological studies of individuals with and without autism spectrum disorders (ASD). However, numerous studies have raised concerns that the degree of influence exerted by ASD-nonspecific factors like age, IQ, and behavior problems may affect the construct validity of the SRS. Recently, a short form of the measure was proposed (Sturm et al., 2017). Using an item response theory (IRT) approach, the authors determined that a shorter form may be a more valid unidimensional quantification of social-communication impairments. However, because this work combined samples of affected individuals and controls without testing key assumptions, it is possible that the item parameters were substantially biased. Further, additional evaluation in clinical samples is needed to determine whether short form scores successfully differentiate individuals with ASD from those with non-ASD diagnoses.
Objectives:
We sought to test the psychometric properties of the short form of the SRS proposed by Sturm et al. (2017).
Methods:
We used the IRT procedures employed by Sturm et al. (2017) to evaluate differential item functioning (DIF)-by-diagnosis in a subset of their data [affected individuals from the Simons Simplex Collection (n=2,630) and their unaffected siblings (n=2,273)]. We then evaluated the psychometric performance of the SRS short form in a clinically ascertained sample of children with ASD (n=670) or other neurodevelopmental disorders or psychiatric diagnoses (n=247), drawn from an existing databank (CADB/UMACC).
Results:
Initial examination of the expected a posteriori distribution revealed a multimodal distribution of the latent trait measured by the SRS short-form. The component distributions were not themselves normally distributed; although the proband distribution was relatively normal, the siblings comprised a zero-class and a right-skewed distribution. These results suggest that the short form is not valid for use as a quantitative measure in individuals without ASD. Further, we were unable to quantify DIF-by-diagnosis without eliminating further items from the SRS short form. In the clinical sample, children with (M±SD=28.58±8.57) and without (M±SD=18.23±9.88) ASD had significantly different SRS short form scores [F(1,890)=237.99, p<.0001]. Scores were moderately related to CBCL Internalizing (r=0.49, p<.0001) and Externalizing (r=0.41, p<.0001), and were not meaningfully related to nonverbal IQ (r=-0.15) or age (r=0.11). As previously documented with the SRS long form (Havdahl et al., 2016), children with significant emotional/behavioral problems who did not have ASD were indistinguishable from children with ASD but no other emotional/behavioral problems (t(471)=1.52, p=.13). Children with similar levels of emotional and behavioral problems were differentiated by the SRS short form (ASD versus non-ASD, t(471)=9.59, p<.0001). Further psychometric evaluation of the SRS short form in the clinical sample will be presented.
Conclusions:
As ASD symptoms are increasingly assessed in large-scale studies (e.g., ECHO), and as clinical psychology and psychiatry continue to ease reliance on categorical diagnoses, the appeal of quantitative instruments like the SRS becomes more apparent. However, to prevent misinterpretation, there is a clear need to understand more about the appropriate uses and limitations of this widely used tool.