28014
Design and Evaluation of an Artificial Intelligent Agent to Measure Communication Skills of Children with ASD
Researchers are increasingly exploring virtual reality environments as potential intervention platforms for children with Autism Spectrum Disorder (ASD). A Collaborative Virtual Environment (CVE) is a virtual reality environment that could facilitate real-time interactions with peers across distance. However, interactions within CVEs change based on specific partner input and as such, fundamentally limit consistent, controlled, and replicable interactions within the CVEs. In addition, manual coding of interactions is necessary to understand patterns of communication for meaningful measurement, creating a resource burden that fundamentally limits realistic paradigm scale-up. An Artificial Intelligence (AI) agent may address these problems by interacting with children as a consistent partner and automatically measuring their verbal communication patterns. Therefore, a virtual intelligent system (VIS) that combines CVE and AI technologies may facilitate complex, dynamic real-world interactions with social partners, as well as automatically measure these interactions to enable scale-up of CVEs to improve social communication skills.
Objectives:
We present a novel VIS that could not just promote peer-based interaction in real time, but also yield quantitative metrics of social communication that can be used within system to facilitate salient aspects of social collaborative learning. The objective of this work is to evaluate whether the VIS could be used as a consistent partner to measure social communication skills of children with ASD in peer-based interactions.
Methods:
Our VIS was composed of a CVE and an AI agent. The CVE was developed with Unity3D game engine (http://unity3d.com/). A series of 9 puzzle games were designed in a shared virtual environment. The interaction in the CVE was governed by implicit rules that required cooperation and communication in order to achieve success. An AI agent was designed to i) monitor the peer-based interactions in the CVE, and ii) to be a consistent partner that could talk and play these games with each child. The AI agent was composed of five modules, i.e., a speech recognition module, a natural language understanding module, a dialogue manager module, a natural language generation module, and a text-to-speech module. In the natural language understanding module, the AI agent could generate communication related features for meaningful measurements using natural language processing and machine learning technologies.
Results:
The Unity-based CVE system was developed with 9 different collaborative tasks. The AI agent was developed with the capability to achieve real-time conversation and interactions. We conducted an initial pilot study with 20 age-matched and gender-matched pairs. All participants enjoyed communicating and playing games with both human-partners and AI-partners. The AI agent was able to capture some communication features, such as number of words, frequency of questions, and frequency of response, which could be used to measure the communication skills of children with ASD.
Conclusions:
The initial pilot study indicated the potential value of VIS in automatically measuring social communication skills of children with ASD in peer-based interactions. This automatic measurement capability will enable a future adaptive system that will modify tasks based on the communication skills of each individual to enhance social communication with peers.