Testing the Ecological Validity of Executive Function Assessment in Autism with a Novel ‘Tea-Making’ Task
Objectives: The current study sought to test the ecological validity of current neuropsychological EF assessment in autism in two ways. First, we tested the verisimilitude of EF by comparing the performance of autistic and non-autistic adolescents on two measures of EF: a battery of common EF tasks (National Institutes of Health (NIH) Examiner battery) and a novel, ecologically-valid ‘tea-making’ task. Second, we assessed the veridicality of EF by testing whether performance on the NIH battery was related to autistic participants’ quality of life and level of disability.
Methods: Data collection is ongoing. Participants assessed so far include autistic (n=20) and non-autistic (n=19) adolescents, between 12 and 19 years, group-matched by age (p=.15) and IQ (p=.30). A composite EF variable was generated by the NIH battery. The tea-making task involved preparing food and materials for a group study session while a researcher rated the number of executive errors committed (e.g. boiling the kettle extra times, failing to adapt to a change in task instructions). Performance on both tasks was scaled and standardized such that higher scores were indicative of better EF ability. Parents also completed The Pediatric Quality of Life Inventory (PedsQL) and The World Health Organization Disability Assessment Schedule (WHODAS-2).
Results: A 2x2 repeated-measures ANOVA comparing group (autistic and non-autistic) and EF assessment type (NIH and ‘tea-making’ task) yielded a significant main effect of group, F(1,36)=8.00, p=.008, ηp2=.18. Autistic participants showed, on average, more EF difficulties compared to non-autistic participants. There was, however, no main effect of EF assessment type, F(1,37)=.001, p=.99, ηp2<.001 and no significant group x EF assessment type interaction, F(1,37)=.29, p=.59, ηp2=.008. There were also no significant correlations between performance on the NIH battery and PedsQL, r(16)=-.28, p=.26 or WHODAS-2, r(16)=.23, p=.36, scores when age and IQ were controlled for among autistic participants.
Conclusions: Both the neuropsychological and ecological assessments of EF employed here were sensitive to group differences. This study demonstrated that, for autistic people, the verisimilitude of EF assessment is comparable with typical development. Yet, the absence of any relationship with parent reported quality of life or level of disability draws into question the veridicality of EF assessment in autism.