Validity of Behavior Evaluations for Shelter Dogs

In the Literature

Patronek GJ, Bradley J, Arps E. What is the evidence for reliability and validity of behavior evaluations for shelter dogs? A prequel to “No better than flipping a coin.” J Vet Behav. 2019;31:43-58.

The Research …

Canine behavior evaluations are performed in shelters ostensibly to determine a dog’s suitability for adoption. Evaluations typically consist of subtests in which the evaluator observes the dog’s responses to assess various behavior traits (eg, evaluating sociability and aggression when the dog is being pet, being restrained, approaching another dog, or responding to removal of a toy). Several published behavior evaluation protocols exist,^1,2 but no standard test has been established. Individual shelters may choose which, if any, evaluation to use, with some shelters modifying existing protocols or developing their own. Despite widespread use of behavior evaluations and numerous scientific studies, there continues to be confusion surrounding the validity of these tests and whether they can accurately predict future aggressive behavior. This confusion is concerning, as the results of these tests may be used to make life-or-death decisions for many shelter dogs and can impact public health and safety. 

The authors of this study searched online databases to determine the extent of the reported reliability, validity, and predictive ability of canine behavior assessments in previous studies. Seventeen studies from 8 countries were identified. The authors found that most studies did not report criteria necessary to meet the scientific standard of test validation, namely reliability (ie, reproducible measurements) and construct validity (ie, how strongly an evaluation measures what it claims to be measuring).

Predictive ability (ie, the likelihood of the assessment predicting the behavior of an individual dog in real life) is determined by the sensitivity and specificity of the assessment and is affected by the prevalence of behaviors in the general population. Sensitivity, specificity, false-positive rates, and false-negative rates were reported or calculated from the 8 studies for one or more behaviors, and all were found to have widely varying ranges; this led the authors to conclude that no canine behavior assessment or subtest has sufficient evidence to be considered a reliable test in shelters.

Five reasons were attributed to the discrepancy between the actual existence of validated evaluations that can reliably predict behavior and what clinicians believe has already been proven through research studies:

• Confusion resulting from mixing colloquial and scientific use of words (eg, validated, predictive, reliable, agreement)

• Erroneous interchangeable use of the terms “correlation” and “agreement” and the limitations of correlation and regression as statistical methods for demonstrating agreement or predictive ability

• The difference between predictive validity of an assessment used under research conditions versus the predictive ability of an assessment to accurately predict individual dog behavior in the real world

• Conflating statistical significance with clinical significance when interpreting results of behavior evaluations

• Presenting studies as validated despite actual results being less determinate

… The Takeaways

Key pearls to put into practice:

• Behavior assessments are not a valid or reliable predictor of aggression after adoption. Euthanasia decisions should not be based solely on a dog’s performance in a behavior assessment.

• Information used to understand and evaluate a dog’s behavior should come from and be corroborated by multiple sources and should include information provided by the previous owner and/or foster caretaker, as well as shelter staff observing the dog engaging in activities that would occur in a home (eg, walks, play, socialization).

• Shelters vary in how they obtain behavior information, how or whether they use behavior assessments, and how they determine suitability for adoption. Local shelters should be contacted to inquire about their policies.