Identifying Therapeutic Mechanisms with Intervention Studies

Background: While the current focus is on identifying effective treatments the understanding of treatment mechanisms is crucial to development and improvement of those treatments. The traditional methods of Baron, Judd and Kenny (BJK) for assessing treatment mediation are helpful but require assumptions to be made about the absence of confounding and reverse causation between mediator and outcome that are hard to justify.  Where confounding and reverse causation exist, for example where within a caregiver training program changes in a child’s symptoms and behaviour influence a caregivers behaviour, mistaken conclusions may be being drawn.

Objectives: We describe how randomisation can be exploited to construct “instruments” that moderate the effect on the mediator which in turn allow us to estimate the extent of treatment mediation in a way that does not suffer from such bias. We illustrate the application of these methods using the PACT trial, the largest formal RCT of a parent based intervention for autism (Green et al, 2010, The Lancet).

 Methods: We consider a range of candidate instruments that exploit the range of baseline variables available in PACT that might influence the extent of response to treatment of the first target mediator, parental synchrony.  We examine the strength of the available instruments and compare the estimates of mediation obtained using this new methodology to those from the traditional BJK approach. We illustrate the use of model averaging.

Results: We examine a range of instruments based around potential variations in quality of therapy, parent compliance and engagement, and child characteristics. The range of estimates of mediation that we obtain is considerable and the relatively wide confidence intervals we obtain reflect our difficulty in finding strong instruments.  We suggest using a model averaging method as a means of synthesizing the estimates to obtain a single overall conclusion.

Conclusions: The need to exploit intervention studies to the full as a means of achieving incremental progress in the development of treatments is clearly essential. Understanding treatment mechanisms is a key part of that process. While the limitations of the BJK approach to the analysis of mediation is becoming increasingly well known, and more sophisticated methods for their analysis are being developed, we will need to enhance the design and measurement protocols of studies in order to increase the power of these methods to deliver clear answers.  In the meantime the new methods provide valuable guidance for a sensitivity analysis, may give some indication as to the extent of bias when using the BJK method, and can be blended with BJK estimates using model averaging.

*PACT Consortium members: Sam Barron, Barbara Barrett, Karen Beggs, Katy Bourne, Sarah Byford, Julia Collino, Ruth Colmer, Anna Cutress, Tori Houghton, Patricia Howlin, Kristelle Hudry, Ann Le Couteur, Sue Leach, Dharmi Kapadia, Kathy Leadbitter, Wendy MacDonald, Sarah Randles, Vicky Slonims, Carol Taylor, Kathryn Temple, Lydia White