Evaluating the Psychometric Properties of Social Cognitive Tasks for Adults with Autism

Background: Although social cognition offers a promising treatment target for adults with autism, this effort is hindered by a lack of well-validated measures of social cognition for use in clinical trials. Many tasks measure the same constructs (e.g., emotion recognition), and little work has been done to assess which measures are the most reliable and valid for identifying social cognitive impairments in adults with autism.

Objectives: Similar challenges in the schizophrenia literature prompted the Social Cognition Psychometric Evaluation (SCOPE; Pinkham et al., 2015), which evaluated the psychometric properties of common social cognitive tasks and provided recommendations regarding their future use. Because related deficits in social cognition characterize autism (Sasson et al., 2011), we tested the reliability and discriminant validity of measures examined in SCOPE for use with adults with autism.

Methods: Adults with autism (n=103; Age: M=24.28; IQ: M=105.64) and typically-developing (TD) controls (n=94; Age: M=24.04; IQ: M=107.80) completed eight tasks from the SCOPE study spanning four domains of social cognition: 1) the Ambiguous Intentions and Hostility Questionnaire (AIHQ) measured hostile attribution style; 2) social appraisal was measured by a) The Awareness of Social Inference Task (TASIT), b) Hinting Task, and c) Reading the Mind in the Eyes; 3) emotion recognition was measured by a) Bell Lysaker Emotion Recognition Test (BLERT), and b) The Penn Emotion Recognition Test (ER-40); 4) social perception was measured with the Relationships Across Domains (RAD). Additionally, the Trustworthiness task was added as it overlaps with each domain. The Cartoon Theory of Mind Task (CToM), which measures non-verbal inferences of intentionality, and the Benton Facial Recognition Test were also added for their relevance to autism.

Results: Reliability was assessed using Cronbach’s alpha. Social appraisal, social perception, and emotion recognition tasks showed moderate to strong reliability: TASIT (α=.86), Hinting (α=.74), CToM Attribution of Intention (α=.72), and Eyes (α=.73); Benton (α=.72) and RAD (α=.81); and BLERT (α=.72), ER-40 (α=.67). The Trustworthiness task also showed strong reliability (α=.92). The AIHQ Blame score showed high reliability (α=.87), but the Hostility (α=.48) and Aggression subscales (α=.43) did not.

We also tested the discriminant validity of the tasks for autism and TD groups. A MANOVA with Group (autism, TD) as a between subjects factor found significant differences across all measures (λ=.636, F(12, 173)=8.27, p< .001). TD adults preformed significantly better than adults with autism on all tasks (p’s< .02) except for the AIHQ Blame (p=.86), Hostility (p=.48), and Aggression (p=.85) subscales, as well as the CToM (p=.58) and Trustworthiness task (p=.10).

Conclusions: Most of the social cognitive tasks assessed demonstrated strong internal consistency and were sensitive at revealing impairments in the autism group relative to the TD group. Specifically, the BLERT, Hinting, ER-40, Eyes, TASIT, RAD, and Benton performed well and are recommended for use with adults with autism. Although further psychometric evaluation is warranted, these findings are largely consistent with SCOPE (Pinkham et al., 2015) and indicate many tasks developed for evaluating social cognitive impairments in schizophrenia are equally applicable to autistic adult populations.