Systems Science Approach to Conceptualize and Analyze the Role of Screening

Background:

System science offers a range of models and methodologies that have the potential to improve public health programs, yet they are rarely applied to the problem of improving the detection and diagnosis of autism spectrum disorder (ASD) among young children. From a systems perspective, screening is but one (albeit critical) element in a system of care, and its effectiveness for improving children’s health is dependent on other resources for further assessment, diagnosis, and linkage to effective service. Systems methods have the potential to offer insight at both the theoretic and methodological level that can be used to help improve systems of care for young children with ASD.

Objectives:

The objective of this talk is to introduce two systems science methods and how they can be applied to research on screening for ASD to address (1) questions regarding missing data in diagnostic accuracy studies, and (2) the potential for policy resistance resulting from clinical uncertainty during dissemination of evidence-based screening policies.

Methods:

To address the problem of missing data in studies of the diagnostic accuracy of ASD-specific screening instruments, we developed a Monte Carlo simulation model of the multi-stage screening process described in the published literature. The model was then tested under a range of assumptions regarding overall prevalence and reasons for dropout. To address the potential for policy resistance in the dissemination of evidence-based screening instruments, we created and validated a system dynamics model of the clinical decision making process and applied it to published implementation trials of behavioral screening instruments for children.

Results:

Results of the Monte Carlo model suggest that despite design challenges in the published study, evidence supports the hypothesis that screening demonstrates adequate sensitivity and specificity. Specifically, a range of prevalence estimates between 1 in 131 (as assumed in the original study) and 1 in 39 (a maximum value that approximates reported prevalence in New Jersey) are all consistent with findings that stage 1 screening displays sensitivity over 70% and specificity over 90%. Results also suggest alternative strategies for improving “process” accuracy, including improving follow-through at each stage of the process.

Results of the system dynamics model highlight how clinicians can use accurate screening instruments to reduce either false positive or false negative errors or both. Specifically, the model helps to explain heterogeneous findings from published implementation trials by demonstrating how observed changes in referral rates may be attributable not only to the use of screening instruments, but also to decision thresholds and the provision of screening support services. The model also suggests strategies for improving detection, including using systematic feedback and co-located care models to shift focus to reducing false negative errors.

Conclusions:

Systems science offers methods to conceptualize and analyze the role of screening as a single element in a larger system of care. Evidence from systems models offers insights into published research on the accuracy and implementation of screening instruments, as well as possible strategies for further improving detection of ASD in community settings.