Systematic Review of Evidence-Based Practice and User-Centered Design for Technology-Based Interventions for the Autistic Community

Background:

The past decade has seen an exponential increase in the number of technology-based interventions for the autistic community. Interventions having robust evidence of effectiveness are rarely commercially available, whilst commercially available therapeutic technologies are infrequently evaluated in research (Constantin et al., 2017). Evidence-Based Practice (EBP) dictates that practitioners should ensure interventions are informed by good evidence (Reichow, 2008). While an EBP framework has been applied to traditional interventions for autism (Mesibov & Shea, 2011), this is not the case for technology-based supports. We argue that technology-based support requires not only a robust evidence base, but also the participation of the autistic community in the development process (Brosnan et al., 2016) to ensure the needs of the autistic community are satisfactorily met. According to ISO 9241-210:2010, User-Centred Design (UCD) is an iterative design approach based on explicit understanding of users, tasks and environments, driven by user evaluation and including multidisciplinary perspectives.

Objectives:

We aimed to systematically review the current state of EBP and UCD for technology-based interventions for the autism community. Additionally, we examined the potential link between EBP and UCD.

Methods:

We followed the PRISMA guidelines for systematic reviews and we used keywords pertaining to technology-based interventions combined with autism conditions. We searched in scientific databases of different foci to accommodate the multidisciplinary nature of the field. From 792 eligible articles we took a random sample of 30% (n= 215) to evaluate two dimensions:

1. EBP: We implemented Reichow’s (2008) evaluative method that identifies evidence-based practices for autistic children and we gave group or single-subject designs a score according to a set of quality indicators.
2. UCD: We developed a scale evaluating the extent to which the end-users and their characteristics were taken into account in the development process. The scale was based on Druin's (1999) taxonomy of user participation and ISO norms for human-computer interaction.

Results:

The mean score on Reichow’s scale was 1.055, 95% CI [0.915, 1.19] (with 1=weak) and on the UCD scale 3.555, 95% CI [3.36, 3.75] (in a range between 0 and 8). Reichow’s scale was not applicable for 81 (38%) of the papers. 71 (33%) papers were rated as weak, 33 (15%) as adequate and 30 (14%) as strong in terms of research rigour. In terms of user-centeredness of the design, 59 (28%) papers received low scores, 135 (62%) medium and 21 (9%) high.

A marginally significant correlation was found between the ratings on the two scales (p=0.05153).

Conclusions:

The resulting ratings indicate a low level of evidence base and a medium to low user consideration and inclusion in the existing literature on technology for autism. Many of the included studies did not meet standard criteria of quality group or single-subject designs. In addition, many of the included studies did not involve end-users or proxies at all. EBP and UCD seem to have a marginally significant positive correlation. Neither framework is currently being applied within the majority of technology-based interventions for autism.