Exploring Responses of Children with ASD to a Virtual Character In the ECHOES Technology Enhanced Learning Environment

Background:  Virtual environments and characters allow individuals with an autism spectrum disorder (ASD) to practice skills repeatedly, in a way that may be less threatening, less socially demanding, and more controllable than is face-to-face interaction with a human partner. To date, their use as an intervention tool has focused on teaching situation-specific social skills to older children or adolescents (e.g. navigating through a bus or restaurant). Little is known about how young children with ASD interact with virtual characters and whether or not they perceive them as being intentional. Even less is known about which specific agent behaviours or combinations of behaviours are particularly effective for capturing, directing, and eliciting responses to attention within a virtual environment.

Objectives:  The ECHOES technology enhanced learning project has developed a multi-modal virtual learning environment to support joint attention initiation and response in young children (aged 5-8) with and without an ASD. Gathering qualitative data while children interact with ECHOES illuminates general research questions about whether children with ASD treat virtual characters as intentional social beings with whom they are willing to engage. Reaction time and accuracy data were gathered to assess the effectiveness of specific agent cueing behaviours in attracting and directing a child’s attention during a task in the virtual environment.

Methods:  30 children with ASD (aged 5-14) individually completed a flower-selection task in the ECHOES environment in cooperation with the virtual agent, Paul. To complete each trial successfully, participants had to respond to Paul’s joint attentional initiations via a touch screen. Initiations varied on the mutuality of Paul’s gaze and the presence or absence of a distal pointing cue.

Results:  Participants had a mean accuracy of 88.12% (SD=20.22%, Median accuracy 95.14%). Accuracy did not vary significantly across the four trial types, nor was it correlated with age (r=0.23, p=0.21).  Examining patterns of errors provided a useful basis for classifying participants into different categories of learners.  A 2 (mutual gaze) x 2 (gesture) repeated measures ANOVA of reaction time data showed a interaction of mutual gaze and pointing cues, p=<.01 (F=1, 30), with a strong effect size (Cohen’s f= 0.48).  The lack of a strong main effect driving this ANOVA suggests that it is the conjunction of gaze and pointing cues which results in the fastest reaction times.

Conclusions:  Qualitative data and high accuracy indicate that children with ASD successfully engaged with agent Paul and followed his joint attentional bid to complete the selection task. Almost all children treated Paul as intentional, attributing his behaviours to a variety of mental states and greeting or addressing him directly. These preliminary results are encouraging for further use of virtual agents as an intervention tool. Results of the reaction time analyses suggest that ASD children may be able to more rapidly process and respond to combined gaze and gesture cues in a virtual environment than to single cues, emphasising the importance of including both in future design.