Perspectives on observer effects in forensic science (D. Michael Risinger, Seton Hall Law School, Newark, NJ)

An elementary principle of modern psychology is that the desires and expectations people possess influence their perceptions and interpretations of what they observe. In other words, the results of observation depend upon the state of the observer as well as the thing observed. This insight is not new; long before cognitive scientists began formally studying the psychological foundations of such effects, the phenomenon was noticed and commented upon. Julius Caesar, for instance, noted that "men generally believe quite freely that which they want to be true."

Sensitivity to the problems of observer effects has become integral to the modern scientific method. Soon after Renaissance natural philosophers began creating the scientific method, they began paying specific attention to the problem of observer effects. The writings of Sir Francis Bacon in 1620, for example, recognized the problem. Bacon suggested that "the human understanding, when any proposition has once been laid down ... forces everything else to add fresh support and confirmation; and although ... instances may exist to the contrary, yet [the understanding] either does not observe or despises them ... ." Bacon also posited that "it is the peculiar and perpetual error of the human understanding to be more moved and excited by affirmatives than negatives, whereas it ought duly to be impartial; nay, in establishing any true axiom, the negative instance is the most powerful." In the first passage, Bacon anticipated what modern research has shown to be the cognitive phenomenon of selective attention: the tendency of observers to seek out some information and avoid other information. In both passages, Bacon anticipated what modern cognitive scientists refer to as confirmation bias, the tendency to test a hypothesis by looking for instances that confirm it rather than by searching for potentially falsifying instances, even though most scientists and philosophers of science today agree with Bacon that the best scientific method is to proceed by doing the latter. Bacon adds that "the human understanding resembles not a dry light, but admits a tincture of the will and passions, which generate their own system accordingly, for man always believes more readily that which he prefers." Like Caesar before him, Bacon took a step beyond cognition and raised the issue of motivational or attitudinal effects on what a person thinks he or she has observed.

Perhaps the first recorded instance of a scientist recognizing that the attributes of an observer were influencing the accuracy of particular observations occurred more than 200 years ago. In 1795, Nevil Maskelyne, Astronomer Royal at the Greenwich Observatory, realized that he and his assistant were obtaining different results for the times of stellar transits, even though they were using identical methods. These discrepancies reflected differences in complex judgments: "a coordination between the eye and the ear ... a spatial judgment dependent upon a fixed position ... an actual but instantaneous position of a moving object, and a remembered position no longer actual." In the 1820s, Bessel, an astronomer at Knigsberg, studied the problem and found that such differences were not only common, but in astronomical measurements they reflected predictable individual tendencies. By the 1830s astronomers had developed a method for calculating "personal equations" that enabled them to measure these particular kinds of observer error, adjust for them, and remove the distorting effects from their findings.

Scientists since that time have learned that observer factors can distort findings and produce misleading conclusions in myriad ways not so easily corrected for. The following are illustrations from a variety of fields.

While Newton failed to see something that did exist, scientists of the early twentieth century saw something that did not exist. First reported by Rene Blondlot in 1903, "N-rays" appeared to make reflected light more intense. So long as they were believed to exist, the effects of N-rays were "observed" by many scientists. Of course, once it was determined that N-rays did not exist, their effects ceased to be observed.

Observer effects also have been found in the reading of scales. That is, people do not always read dials and other readouts correctly, and their errors are nonrandom. Certain numbers or patterns are more likely to be "read" than others, resulting in systematic errors in the data read from the measuring instruments.

For many years, laboratory technicians who counted blood cells visually were taught that correct counting would keep blood cell counts within a certain range of variation. In 1940, using a more accurate photographic method to count blood cells, researchers discovered that for years technicians had been reporting blood cell counts that were within an impossibly narrow band of variability. The technicians made observations consistent with the expectations they held, but inconsistent with reality.

Mendel's counts of characteristics in pea plants came much closer to the theoretical predictions than is likely to have been possible. Mendel or his assistant either deliberately misreported, or were the victims of observer effects induced by expectation.

One medical researcher found observer errors in the use of the stethoscope in cardiac diagnostics, leading him to suggest that physicians as well as their stethoscopes needed to be calibrated. Another medical researcher, after finding medical students observing quite inaccurately when presented with two x-rays of hands to study, concluded that "our assumptions define and limit what we see, i.e., we tend to see things in such a way that they will fit in with our assumptions even if this involves distortion or omission."

A writer on marine biology, reflecting on problems of animal observation, commented that scientists may "equate what they think they see, and sometimes what they want to see, with what actually happens."

These realizations and attention to them have evolved into a "science of science," a careful study of the causes of the random and systematic errors induced by observer effects and the methods for their prevention. The results of such work can be found in the classrooms, textbooks, and laboratories of virtually all scientific fields, where methods and procedures have been developed to minimize the impact of such distorting influences. Today, awareness of such problems and their solutions is so widespread that concepts such as double-blind and placebo have become household words popularly understood well beyond the laboratory, and analogous error-prevention techniques are employed in settings beyond the domain of science. For example, in many schools, including of course nearly every law school, teachers are required to grade examinations without knowing the identity of the student. Other common examples of such anonymous evaluations include auditions for symphony orchestras where the candidates may play behind a screen, and academic journals, many of which conduct blind peer review of submissions.

Forensic science is one of a very few fields that has not yet profited from this "science of science." The most obvious danger in forensic science is that an examiner's observations and conclusions will be influenced by extraneous, potentially biasing information. However, there are other potentially error-producing sources of expectation beyond those induced by intentional or unintentional suggestion.

Materials

Presentation

DM Risinger, MJ Saks, WC Thompson, R Rosenthal. The Daubert/Kumho implications of observer effects in forensic science: Hidden problems of expectation and suggestion. California Law Review. Jan. 2002;90(1):1-56.

I Dror, D Charlton, A Péron. Contextual information renders experts vulnerable to making erroneous identifications. Forensic Science International. 2006;156(1):74-78.

L Miller. Procedural bias in forensic science examinations of human hair. Law and Human Behavior. Jun. 1987;11(2):157-163.