The Impact of the Audience Effect and Inattention on Online Vs in-Person IQ Assessment

Background: In order to facilitate collection of large samples of individuals with Autism Spectrum Disorder (ASD) and typically developing controls (TDCs), we developed an online, computer-adaptive IQ test based on item response theory - the Center for Autism Research Adaptive Test (CARAT; Clements et al., 2015). The online CARAT is self-administered; however, the absence of an examiner may affect performance in at least two meaningful ways for individuals with ASD and/or ADHD. First, individuals with ADHD may show periods of inattention without an examiner to help guide focus, missing some questions they ordinarily would have gotten correct. Second, the presence of another individual can affect one’s behavior via the “audience effect,” which is believed to be tied to a desire to positively manage one’s reputation in front of others. Chevallier et al. (2014) found that examiner administration of a task benefited typically developing controls (TDCs) more than individuals with ASD compared to a computerized version of the test taken alone.

Objectives:  Examine the effects of administration mode (in-person or computerized) on IQ test performance for individuals with ASD and/or ADHD.

Methods:  Individuals ages 6-90 will complete both the online CARAT and the in-person WASI-II (n=50 ASD, 50 TDC). We expect a subset of both groups to have ADHD. We will detect “aberrant” responses due to inattention, meaning responses that are unexpectedly incorrect given the examinee’s response pattern, by comparing the fit of two ability models that allow different levels of variability (Zoltowski et al., 2016). We previously used this method to successfully detect other types of aberrant responses (e.g., seeking help). We will compare the number of aberrant responses detected in individuals with and without ADHD. To test the impact of the audience effect, we will analyze the interaction between diagnostic group (ASD and TDC) and type of test (computerized CARAT or in-person WASI-II).

Results:  To date, 24 TDC individuals between ages 7 and 53 (mean(sd) age = 24.7(12.6) years) have completed this testing, and WASI-II Full Scale IQ scores ranged from 86 to 136 (mean(sd) IQ score =114.1(15.6)). The remaining sample will be recruited in time for analysis and presentation by May 2017, and results in either direction will provide information about the appropriateness of online testing for different diagnostic groups.

Conclusions:  As online tasks become increasingly popular in the field of ASD research, it is important to understand how different testing formats affect performance for people with autism and inattention. These analyses may help researchers determine how to interpret results from self-administered tasks and how to identify aberrant responses to items. The variability analysis may be able to detect individuals who have greater difficulty sustaining attention during online assessments so that researchers can implement supports for these situations, or rescore the measure removing aberrant responses. Furthermore, computerized testing may either provide a more equal testing environment, if we find that the audience effect boosts performance of TDC individuals, or may not affect group differences reported in examiner-administered tasks. Extensions of these approaches could be applied to other online tasks.