31575
An Open-Source, Computing Platform-Agnostic, Calibration-Free Preferential Gaze Detection Approach for Social Preference Assessment

Poster Presentation
Thursday, May 2, 2019: 5:30 PM-7:00 PM
Room: 710 (Palais des congres de Montreal)
R. Bishain1, S. Chandran1, I. Dubey2, J. Dasgupta3, M. Belmonte4, T. Gliga5, D. Mukherjee6, S. Bhavnani6, G. Estrin5, M. H. Johnson7, V. Patel8, S. Gulati9, G. Divan10 and B. Chakrabarti2, (1)Computer Science and Engineering, Indian Institute of Technology, Bombay, India, (2)Centre for Autism, School of Psychology & Clinical Language Sciences, University of Reading, Reading, United Kingdom, (3)Sangath, Delhi, India, (4)Com DEALL Trust, Bangalore, India, (5)Centre for Brain and Cognitive Development, Birkbeck University of London, London, United Kingdom, (6)Centre for Chronic Conditions and Injuries, Public Health Foundation of India, Gurgaon, India, (7)Department of Psychology, University of Cambridge, Cambridge, United Kingdom, (8)London School Of Hygiene and Tropical Medicine, London, United Kingdom, (9)Child Neurology Division Department of Pediatrics, All India Institute of Medical Sciences, Delhi, India, (10)Sangath, Bardez, Goa, India
Background: Social preference assessment, using contrasting video or image pairs, has been commonly used in autism screening. Such assessments tend to be conducted in the presence of experts in a controlled laboratory environment, primarily because most gaze-tracking apparatus are not portable. Further, the solutions employed are proprietary, and platform-dependent, inhibiting the wide acceptance of this important method across researchers. Recent advancements in machine-learning based gaze-tracking techniques, combined with the proliferation of low-cost hand-held devices, have challenged this paradigm. We present an open-source eye-tracking pipeline which enables casual, on-field preferential gaze understanding that can be carried out cost-effectively under minimal supervision in a typical home environment.

Objectives: To develop a portable, fast, open-source preferential gaze tracking pipeline for visual-stimuli guided social preference assessments.

Methods: The proposed gaze-tracking pipeline was deployed on data sourced from the ‘preferential-looking’ task of START (“Screening Tool for Autism Risk using Technology)”, a tablet-based app usable by non-specialists in home settings. The task was administered to a group of 127 children aged 2-7 years at their homes in India. The preferential looking task presents the child with simultaneous social and nonsocial videos on the tablet screen. The tablet camera records the child’s face during the task. This recording is analyzed asynchronously by our gaze tracking algorithm to predict the child’s visual preference in each frame. The algorithm implemented in python is deployed offline on a desktop system, and processes the data for multiple children in a batch mode. The average processing time for the algorithm is 90ms per frame. The algorithm pipeline, consisting of existing open-source modules,
(a) extracts (using OpenCV[1] library calls) regions corresponding to the face and the eyes,
(b) obtains gaze coordinates[2] in pixels based on the device used (no reference needed for the subject or the camera parameters), and
(c) transforms, with temporal smoothing by a median filter, raw pixel coordinates to higher level preference prediction, enabling social preference assessment.

Results: The algorithm achieved 91% accuracy (percentage of frames with correct classification of gaze preference) on a manually annotated dataset with 5000 frames. The frame-drop rate (failure to detect faces due to poor lighting, occlusions, multiple faces, etc.) is observed to be no more than 5% for more than 73.2% of the on-field dataset, and it exhibits a decreasing trend in this range (Figure 1).

Conclusions: Our computer program (available freely[3]) works on video frames, and can, therefore, be used in any system as long as video frames are provided. The video can be captured by non-specialists in casual environments regardless of the hardware employed, making the system cost-effective for a screening measure. The approach is calibration-free, and can be executed asynchronously on video inputs.

References:

[1] Bradski, G. The OpenCV Library, Dr. Dobb's Journal of Software Tools, 2000
[2] Kyle Krafka, Aditya Khosla, Petr Kellnhofer, Harini Kannan, Suchi Bhandarkar, Wojciech Matusik and Antonio Torralba. “Eye Tracking for Everyone”. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[3] The Apache Software Foundation License: https://www.apache.org/licenses (Last Accessed: Nov 10, 2018)

See more of: Technological approaches
See more of: Technological Approaches