Evaluating the Pragmatic Rating Scale-School Age: Robust Psychometrics and Clinical Implications

Background: Pragmatic, or social communication, deficits are key characteristics of autism spectrum disorder (ASD); subclinical differences are detectable in first degree relatives of individuals with ASD (Landa, 1992; Greenslade et al., in press). Two main types of pragmatic language measures exist: parent report and standardized structured clinician-administered assessments. Both have limitations. Parent–rated may be less sensitive than clinician-rated measures (Hess & Landa, 2012) and structured measures lack ecological validity. To provide a clinician-rated measure based on ecologically valid communication samples, the Pragmatic Rating Scale-School Age (PRS-SA; Landa, unpublished) was developed. The PRS-SA enables clinicians to rate level of social difficulty (0=none, 3= severe) across 21 components of communication.

Objectives: To evaluate the internal consistency and concurrent validity of the PRS-SA in 4- to 8-year-olds with a continuum of ASD symptomatology, and preliminarily examine whether the PRS-SA is useful for detecting ASD symptoms and risk. Hypothesis: The PRS-SA will demonstrate: (1) moderate internal consistency due to the diversity of social language and communication features measured; (2) strong correlation (concurrent validity) with standard ASD measures, as pragmatic impairment is central to ASD; and (3) good sensitivity and specificity for ASD classification (positive/negative).

Methods: 136 4- to 8-year-olds (M=6.94 years, SD=1.85) (83 males; n=49 with ASD) at high and low risk for ASD participated in a prospective, longitudinal study of ASD (additional data will be added by the INSAR presentation). All had complete data on the Abbreviated Stanford Binet Intelligence Scale, 5^th Edition, ADOS-2, SRS-2, and PRS-SA. Cronbach’s alpha examined internal consistency. Pearson’s correlation coefficients assessed PRS-SA criterion validity (concurrent) with the SRS-2 and ADOS-2 (standardized ASD assessments). A receiver operating curve (ROC) analysis was conducted to examine PRS-SA sensitivity and specificity for identifying ASD+ or ASD- status (Figure 1).

Results: Cronbach’s alpha (α=.635) indicated a moderate level of internal consistency, and no improvements were noted with the removal of any individual item. PRS-SA total score was significantly correlated with SRS-2 total score (r=.453, p=.002) and ADOS-2 Communication domain algorithm score (r=.424, p<.001).

ROC analysis generated an AUC of .890, indicating a good level of sensitivity and specificity for identifying ASD status. A PRS-SA cut off score of 21 is optimal for identifying ASD+/ASD- status: sensitivity=.837, specificity=.790.

Conclusions: As hypothesized, the PRS-SA demonstrated moderate internal consistency, likely reflecting the diversity of communication behavior encompasses in pragmatics and measured by the PRS-SA. Factor analysis will explore potential subscales when sample size is increased, and will be reported in this presentation. Strong correlations between PRS-SA total score and two standardized mainstream ASD measures (one a parent report and one a clinician-rated, direct observation), provide strong concurrent validity evidence, affirming the PRS-SA’s use to detect ASD-related pragmatic communication behavior. Our findings suggest that PRS-SA scores of ≥21 in 4- to 8-year-olds indicate the need for ASD evaluation. Additional recruitment will allow a validation of this cut off score in an independent confirmatory sample; sensitivity and specificity results from this analysis will be reported.