Validation of Parent Collected Observational Data In the Natural Environment

Background: Some individuals with autism engage in problem behaviors such as aggression or self-injury which may seriously impact their quality of life and that of their caregivers. Aside from threatening physical health, problem behaviors can restrict the range of choices available to individuals with respect to residence, employment, education, or social activities. As such, it is important to develop effective interventions that produce lasting decreases in problem behavior. Studies have demonstrated that treatments are most effective at reducing problem behavior when they are based on the results of a functional behavioral assessment (Horner & Carr, 1997; Vollmer & Smith, 1996).  Directly observing the target behavior allows a clinician to indentify subtle relationships between environmental events and problem behavior.  When in vivo observation or problem behavior is not feasible, some clinicians may ask caregivers to collect video footage of the problem behavior. Recent innovations in technology have lead to selective archiving video systems.  Selective archiving continuously captures video using a temporal buffer that erases captured data as more is recorded.  When a caregiver triggers the system, the relevant event is captured, as well as a preset interval prior to and following the moment indicated by the caregiver.  However, it is currently unclear how well these caregiver implemented systems are utilized in real world situations.

Objectives: The purpose of the current study was to investigate the effectiveness of selective archiving systems in identifying relationships between environmental events and problem behavior when operated by a caregiver in the home. Specifically, the accuracy of caregiver collected video and missed occurrences of the target behavior were investigated.

Methods: Continuous Recording and Flagging Technology (CRAFT) was developed for this study so that caregivers could signal when problem behavior occurred by placing a timestamp on continuously recorded footage.  The timestamps were compared to the continuous footage to determine the accuracy of the data that would have been collected if a selective archiving system had been used.  False positives were marked when a timestamp was placed on footage that did not indicate problem behavior in the previous window.  False negatives were observed when problem behavior occurred in the absence of a timestamp.  True positives were scored when the timestamps and trained observer data both indicated problem behavior occurred.  Information was also collected in instances when the child was alone or out of the camera’s view.

Results: Results indicate that caregivers do not collect accurate data.  True positive accounted for only 16% (range, 0% to 50%) of the indicated instances of problem behavior.  False positives and false negatives occurred 22% (range, 0% to 50%) and 8% (0% to 21%) respectively.

Conclusions: Selective archiving systems have recently found their way into the commercial marketplace (Caring Technologies, 2010) and are becoming more widely available. Data from the current study indicate that, when provided minimal training, caregivers do not use these systems optimally.  Future investigations should explore ways to improve the effectiveness of selective archiving systems.