Tracking Toddlers' Head Movement with Facial Landmarks during the Viewing of a Social Scene

Background: Infants at high risk for developing autism spectrum disorder (ASD) exhibit atypical head movements as early as 1-2 months after birth (Denisova & Zhao, 2017). However, it is challenging to track or quantify head movement with subjective observation. With improvement of computer vision technologies, automated tracking of head movement in toddlers can be a helpful quantification tool for this behavior.

Objectives: (1)To apply 3D head movement tracking with facial landmarks in behavioral videos; (2)To compare spontaneous head movement in toddlers with and without ASD engaged in a free-viewing eye-tracking task involving social scenes; and (3)To examine the extent to which head movement is correlated with social affect (ADOS-SA) and restricted and repetitive behaviors (ADOS-RRB). We hypothesized that ASD toddlers would exhibit increased head movement, and that this would be related to RRB symptoms.

Methods: Openface implementation (Amos et al., 2014) with dlib was applied for face detection and facial landmark extraction from video frames with classic Histogram of Oriented Gradients (HOG) feature combined with a linear classifier (Kazemi & Sullivan, 2014). The positional and rotational position and their changes through time were identified by head pose estimation in dlib (Kazemi & Sullivan, 2014). Participants included 26 toddlers (ASD, n=13, M_age=41.70±3.03; typically developing (TD), n=13; M_age=37.87±2.69) who completed a 3-minute head- and eye-tracking task consisting of an actress engaged in dyadic bids for engagement. Linear mixed model (LMM) analyses were used to examine fixed effects of actress’ speech (present/not present), actress’ gaze direction (mutual/averted), clinical diagnosis (ASD/TD), and their interactions on the maximum amplitude of toddlers’ head positions (pxyz) and rotation angles (rx,ry,rz; see Figure 1).

Results: LMM analyses of pxyz revealed a main effect of speech (F(1,799)=9.08, p=0.003); less head movement was observed when speech was present. There was no main effect of gaze direction, clinical diagnosis, or their interactions, and no correlation with ADOS-SA or RRB (p’s>.1). Analyses of rx, ry, and rz also revealed a main effect of speech (p’s<.05) with less rotation at speech presence, but no effect of gaze direction (p’s>.1). While ry analysis revealed a main effect of diagnosis (F(1,24)=5.01, p=0.035; ASD>TD), rx revealed no effect (F(1,24)=2.44, p=0.13; ASD=TD) and rz revealed a marginal effect (F(1,24)=3.70, p=0.066; ASD>TD). In toddlers with ASD, rx correlated with ADOS-SA (r(13)=0.73, p=0.005); rz marginally correlated with SA (r(13)=0.54, p=0.055); ry did not correlate with SA (p>.1). None of the rotational angles correlated significantly with ADOS-RRB (r’s=.113-.266; p’s>.1).

Conclusions: Both groups displayed less head movement in the presence of speech, suggesting that speech may elicit greater attentional engagement with the stimuli and associated suppression of extraneous motor activity. Gaze direction, however, had no effect on toddlers’ head positions or rotation angles. ASD toddlers displayed more rotational head movement then TD controls in both y and z directions. Further, rotational head movement was related to social affect, but unrelated to repetitive behavior. This finding suggests that during a dyadic condition, head rotations may be a result of diminished interest in or uncomfortableness with social situations rather than unsuppressed motor behavior.