Comparing Patterns of Errors on the Raven's Progressive Matrices Test: Strategy Differences Among Typically Developing Individuals, Individuals with Autism, and Computational Models

Background: Recent studies of the Raven’s Progressive Matrices (RPM) test have shown differences in both behavior (Dawson et al. 2007) and neural activity (Soulières et al. 2009) between individuals with autism and typically developing (TD) individuals.  We hypothesize that these effects stem from differences in the cognitive strategies used by each group.  In particular, fMRI data suggests that individuals with autism might use more visually oriented strategies, whereas TD controls may use more verbally oriented strategies.  These data are consistent with computational accounts of the behavior of TD individuals on the RPM, which suggest that they convert perceptual inputs of test images into propositional representations (e.g. Carpenter, Just, & Shell 1990).  However, our recent computational models (Kunda, McGreggor, & Goel, IMFAR 2010 oral presentation) have proposed that purely visual strategies that reason directly on perceptual inputs can also be successful on the RPM.

Objectives: To investigate differences in patterns of incorrect answers on the RPM chosen by TD individuals versus individuals with autism, and to compare these human data with data from Kunda, McGreggor, and Goel’s (2010) visual affine algorithm.

Methods: Participants included 84 typically developing individuals and 111 individuals with autism, ranging from children to adults.  Data on answer choices were obtained from previous RPM and IQ evaluations of these participants.  Answer rankings were calculated according to the percentage of each group choosing each answer choice.  For the visual algorithm, answer rankings were obtained directly from the algorithm’s output.  Differences between answer rankings were calculated using a normalized Kendall’s Tau measure, which counts the number of pairwise transpositions between two given rankings.

Results: Initial results show that, using a paired t-test, differences in answer rankings between the two human participant groups were found to be significantly smaller than differences in rankings between each human group and the visual algorithm (p < 0.001 for each).  However, differences in rankings between each human group and the visual algorithm were not significantly different (p = 0.9).  Preliminary analyses were also conducted according to participant variables (e.g. total RPM score, age, cognitive profile) as well as according to RPM variables (e.g. RPM sets, problem types).  Differences in rankings between TD individuals and individuals with autism, as well as differences between each group and the algorithm, seem to increase with the progression of RPM problem types from gestalt to visuospatial to verbal.

Conclusions: These preliminary results appear to indicate some relationship between the cognitive strategies used on RPM problems and the patterns of incorrect answer choices that emerge.  In particular, for problems classified as gestalt or visuospatial, there is less disagreement on incorrect answer choices among individuals with autism, TD individuals, and the visual affine algorithm than on problems classified as verbal, although problem difficulty might be a confounding factor.  Ongoing analyses include the use of alternative and finer-grained measures for comparing answer rankings, as well as the development of additional methods for analyzing this type of error pattern data from human participants and computational models.