Assessment of Change in Autism: Inter-Rater Reliability of Developmental Disabilities CGAS and the OSU Autism CGI

Background: Research on outcome and treatment response (change) in autism spectrum disorder (ASD) is of paramount importance, but the literature on these issues is only slowly improving. One reason is the lack evaluated economic tools to assess change in autism. Despite the general widespread use the Children’s Global Assessment Scale (CGAS) and Clinical Global Impression Scale (CGI) in psychiatry, these tools cannot be easily applied to children with autism spectrum disorder (ASD).

Objectives: The aim of this study is to examine the feasibility and inter-rater reliability of the Swedish versions of Developmental Disabilities Children Global Assessment (DD-CGAS), a version of the CGAS modified for developmental disorders like ASD, and the OSU Autism Clinical Global Impression (OSU Autism CGI-S).

Methods: Sixteen clinicians (13 female, 3 male ) with varying professions and experience  (> 2 yrs. and <2 yrs.) were recruited from 9 different child and adolescent psychiatry units in Stockholm County independently and spontaneously rate eight clinical vignettes of ASD for clinical referral and at discharge.

The vignettes consist of cases of ASD including clinical descriptions of everyday functioning and symptomatology. According to the clinical consensus ratings for the vignettes on the DD CGAS are 30-70, and 2-4 on the OSU Autism CGI Severity (S) Scale. The vignettes described cases of clinical heterogeneity with varying symptomatology and adaptive skills. 

Statistical analyses, two-way random interclass correlation coefficients (ICC) in SPSS 20 were used to calculate interrater reliability for all vignettes and all possible pairs of raters for both the DD-CGAS and the OSU Autism CGI-S, as well as separately for experienced and less experienced clinicians. In addition to ICC, Pearson correlations between the DD CGAS and the OSU Autism CGI -S results were calculated.

Results: ICC for all raters (experienced and inexperienced) and points in time was .63 (95% CI= .34-.94) on the DD-CGAS and .60 (95% CI= .31-.93) on the OSU Autism CGI-S. On the DD-CGAS, ICC was 0.75 (95% CI = .45-.96, [ICC.78 referral-.79 discharge]) for experienced, and .58 (95% CI =.39-.96, [ICC.54 referral-.53 discharge]) for inexperienced clinicians. On the OSU Autism CGI-S, ICC was .72 (95% CI =.39-.96, [ICC.73 referral-.72 discharge]) for experienced and .59 (95% CI= .40-.79, [ICC.48 referral-.46 discharge]) for inexperienced clinicians. The correlation between DD CGAS and OSU CGI for referral was r= -0.86, SD=0.69 and the correlation between those instruments for discharge was r= -0.82, SD =0.28.

Conclusions: In a naturalistic clinical settings for agreement between experienced and inexperienced clinicians on the DD-CGAS and OSU Autism CGI-S is good to substantial. Correlation between DD-CGAS and OSU Autism CGI-S was high, both at referral and at discharge, which indicates a substantial association between severity of symptoms and functional impairment. DD-CGAS and OSU Autism CGI-S are promising, economic, intuitive tools to assess change for various clinical and research purposes in ASD.