Quantifying the Social Gaze in Ongoing Triadic Interaction

Background: We use our eyes to obtain social information about others, but they also convey information about our inner mental state. Gaze encounters can therefore be considered as a social reciprocal process. Referring to our recently proposed theoretical framework of a "social gaze (state) space" that differentiates five different gaze states of an agent in triadic interactions: ‘partner-oriented’, ‘object-oriented’, ‘introspective’, ‘initiating joint attention’, and ‘responding joint attention’, we aim to empirically and systematically investigate gaze behavior in typically developing and clinical populations across the life span.

Objectives: Social gaze behavior and its deviations in psychiatric conditions has typically been investigated using static, non-interactive settings (e.g. image or video stimuli) neglecting the highly dynamic character of real life gaze encounters. Increasingly, gaze contingent settings are being employed, but these are often restricted to atomic units of interactions (e.g. single trials of joint attention). We propose to employ an empirically informed agent that oscillates between different gaze states, thus mimicking typical human gaze behavior. This approach will allow to fully explore the dynamics of ongoing triadic encounters in a much more ecologically valid research setting.

Methods: We experimentally induced social gaze states in 37 adult volunteers (19 identifying themselves as female, 17 as male, 1 as queer) with no record of neurological or psychiatric conditions who were interacting with an algorithm-controlled virtual agent in ongoing triadic interactions. Behavioral parameters (e.g. temporal lag of gaze following after agent initiates attention, duration of object fixation after agent has responded to initiating joint attention,…) in different combinations of the five different gaze states were obtained to describe typical human gaze behavior.

Results: We extracted behavioral parameters including their dynamics and distribution within the investigated sample and used these to create an empirically informed, algorithm-controlled virtual Markovian agent that allows to study the full range of gaze-contingent triadic interactions.

Conclusions: We provide a reference space for investigating gaze behavior in clinical populations with impairments in social interactions, such as autism.