Introduction
Forensic document examiners want to minimize cognitive bias in their work. We must first understand the problem. Cognitive bias as it relates to forensic science is a popular topic today. A number of reasons contribute to this phenomenon. Forensic scientists are eager to improve the quality of their work and wish to remove the influence of cognitive bias from their practice. Popular culture in the form of news stories about forensic science have informed the general public about our work. Aggressive advocates are looking to exploit any perceive weakness in forensic science in order to create doubt in a court proceeding. Also, occasional errors in the work have focused attention on cognitive bias as a possible cause of those errors.
Cognitive bias results when people steadily depart from rational decisions [1]. Cognitive bias affects everyone and may take two forms as they relate to forensic science. Confirmation bias is the propensity to value information that supports a pre-existing view and discounts information that does not [2]. Context effects or context bias creates anticipations about what you should determine [3]. Information contained in a police report about the suspect’s previous criminal record may provide contextual bias to the forensic scientist working the case and interfere with their impartiality in the examination.
Forensic document examination (FDE) can be described as the application of science and analytical techniques to questioned document problems [4]. The work has existed for more than a century and has sections in most major law enforcement agencies worldwide along with private practitioners. The individuality of handwriting was demonstrated in Srihari’s 2002 study [5]. The proficiency of professional forensic document examiners (FDEs) in correctly identifying handwriting was proven in studies by Kam [6], Durina [7], Parrett and Szabo [8]. There is no doubt that a handwritten signature on a will can be identified as naturally written with black ballpoint pen ink and compared to a known sample of 20 signatures from everyday business to determine authenticity.
Critics suggest that forensic scientists, including forensic document examiners, are vulnerable to context effects and would benefit from double-blind testing and evidence line-ups [3]. This idea was tested in research by Parrett and Szabo [8] who used a line-up of 1 questioned document and 1,000 known writers and found that each FDE presented with the test correctly identified the questioned document with the known writer. Langenburg [9] evaluated statistics for latent print cases at the Minnesota Bureau of Criminal Apprehension in the Latent Print Unit. He compared the latent print identification rates for cases with high context/high police contact with no context/no police contact cases and found the rates to be nearly equal (21% versus 22%). The idea of limiting biasing information may be a good one but the direct effects on forensic scientists are debatable.
Basic Psychological studies
Psychological studies show that the brain can play visual tricks and Forensic Document Examiners’ work is accomplished through vision and perception so must be aware of visual and cognitive biases and tricks. Edgar Rubin’s vase illustrated the difference between figure and ground in his doctoral thesis in Denmark in 1915 [10]. The figure can be perceived two ways as a vase or as two faces. Rubin determined that this ambiguous, reversing two-dimensional form can be interpreted as two different shapes viewed one at a time.
Balcetis and Dunning [11] in their study demonstrated that motivation can influence the result. They showed subjects a shape that could be read as “B” or “13”. For those told that if they saw the letter B they could drink orange juice and if they saw 13 they would drink an awful smelling drink, 72% saw B. When subjects were told that if they saw 13 they got to drink orange juice and B they would drink the awful smelling liquid, 61% chose the number 13.
Flawed Studies Regarding Cognitive Bias and Forensic Document Examination
Miller’s 1984 study [12] is often cited by authors writing about cognitive bias and FDEs. In this study 12 college students were “trained” as FDEs. Group 1 was given only one suspect’s writing to compare to 3 questioned checks and were given biasing information. Group 2 was given 3 questioned checks and known writings from 3 suspects to determine if any of them wrote the questioned checks. Results in Group 1—4 students misidentified the suspect writer, 1 was inconclusive and 1 correctly eliminated the suspect. Results in Group 2—all correctly eliminated the 3 suspect writers. Flaws in this study include: slightly trained college students are not FDEs, professional standards—SWGDOC [13]- requires a 2 year fulltime training program for FDEs, the test lacks construct validity (it does not measure what it is meant to measure) and uses a very small sample size (12 students).
Stoel, Dror, and Miller examined Miller’s original work in their 2013 paper titled Bias Among Forensic Document Examiners: Still a Need for Procedural Changes [14]. This paper did not attempt to reproduce the 1984 experiment with fully trained professional FDEs rather it described the original study and concluded that it had some problems with methodology, but was still a good idea. The flaws in this paper included: no new data set to attempt to reproduce the 1984 results, it still relied on 12 slightly trained college students as subjects, and the additional authors dismissed the original study deficiencies. This would be like conducting a study to draw conclusions about bias of physicians using a few premed college students as the data set.
In a 2014 paper Handwriting: Cognitive Bias by Sulner [15], the author used 2 case studies to draw conclusions about the bias of FDEs from 2 court cases. This paper exhibited data problems. First, the very small sample size included only 2 case studies. Second, it is not possible to determine whether these 2 cases were cherry-picked by the author or whether they were truly representative. Third, the measurement that he used was the judge’s decisions in each case. The judges’ decisions were not a direct measure of cognitive bias on the part of FDEs. Too many other factors could contribute to the judges’ decisions in each case. Therefore it is not possible to draw any conclusions from the judges’ decisions.
Current Research Regarding Forensic Document Examination
Found and Ganas [16] initiated a context management system in the Document Examination Unit of the Victoria Police Forensic Services Department in Australia in 2009. They successfully separate forensic document examiners from biasing case information from police submitters through tools including a notice added to email signatures to remind police of the context management system at the laboratory and modified case submission forms.
Merlino et al. [17] completed a study titled Validity, Reliability, Accuracy, and Bias in Forensic Signature Identification. Eye-tracking information from 49 FDEs and 43 lay people was evaluated. In this study FDEs generally out performed lay persons. In a peer review protocol the effects of previous information about another person’s examination showed mixed results. FDEs followed their original calls more closely than lay persons when the signatures were genuine or simulated, but less closely when the signatures were disguised. Merlino et al. [17] concluded “Overall, these findings provide tentative evidence that domain irrelevant information may introduce bias into human decision making processes, although the extent to which our findings are due to the manipulation of the prior examination outcomes to demand characteristics is difficult to ascertain from these data.”
Method Insulate Forensic Document Examiners from Biasing Information in Case Work
In crime laboratories case managers can insulate FDEs from potentially biasing information from investigators by changing the case submission forms to eliminate information about the suspect unnecessary to work the case. They can also take phone calls from investigators limiting the direct involvement of the forensic scientist with the police.
Many questioned document cases are civil cases not criminal cases and are worked in small one or two person laboratories. In civil cases FDEs can insulate themselves from potentially biasing case information at the initial phone contact with attorneys presenting the case. Attorneys are very grateful for this instruction given by the FDE at the start of the first phone call. My phone script is placed next to my phone in the office to remind me to warn new clients, usually attorneys, not to impart biasing information in order to preserve my impartiality in the case. My phone call script reads – “Before you tell me too many details about your case, research suggests that exposure to contextual case information may lead to bias in the analysis. In order to preserve my impartiality, only give me information necessary for me to work the case. What documents are in question? Whose sample writings do you want me to examine and compare to the questioned items?”
When a civil case inquiry arrives via email, I send the attorney my curriculum vita, a fee schedule and an impartiality memo. My impartiality memo reads in part – “Forensic document examiners begin each case from a point of complete neutrality. Research suggests that exposure to contextual case information may lead to bias in the analysis. In order to prevent error from bias in this case, only supply information necessary for me to work the case…”
In the province of Ontario in Canada forensic scientists must sign an Acknowledgement of Expert’s Duty form for each case. The attorney hiring the expert provides this form. Part of the agreement reminds the expert that she must “provide opinion evidence that is fair, objective and non-partisan.”[18]
Attorney and Investigator Responsibilities
Attorneys most often present civil cases to FDEs by phone or email contact. They are typically eager to tell their story and offer their theory of the case during the initial contact with the FDE. Using a phone script and impartiality memo warns the attorney not to convey biasing case information. But attorneys have responsibilities not to convey potentially biasing information that may compromise the impartiality of the expert. The code of ethics of their professional organizations could be used to remind attorneys not to convey biasing information to the experts they hire. The American Academy of Forensic Sciences (AAFS) has a Jurisprudence Section. This section has sponsored workshops at the AAFS annual meetings in recent years on the topic of Cognitive Bias. Attorneys who are members of AAFS are well aware of the problem of cognitive bias and could institute a prohibition in the AAFS code of ethics for the Jurisprudence Section to refrain from giving biasing information about their cases to experts whom they hire.
Similarly investigators launch into the story of the case during the initial contact with a FDE. They convey information that is potentially biasing to the expert. They too have a responsibility to help preserve the impartiality of the expert. Many investigators in the United States are members of the International Association for Identification (IAI). The IAI could require their members who are investigators to agree not to convey biasing information to the experts with whom they interact in a case in their code of ethics.
Discussion
Research has shown [19] that cognitive bias in the forms of confirmation bias and contextual bias or context effects can occur in humans and may interfere with impartiality in forensic case work. The direct effects on FDEs, however, are debatable. Three of the studies [12, 14 and 15] that dealt directly with measuring the bias of FDEs were flawed studies. If the problem is as great as forensic critics contend, minimizing biasing information will improve the work of FDEs. Biasing information should be eliminated where possible and practical. FDEs should not work completely blind to information that may help them answer necessary case questions. Professional FDEs must be able to use their discretion with case information that they may require to properly work the case. Langenburg’s study of latent print examiners [9] reminds us that high police involvement with the examiners versus no involvement yielded nearly identical identification statistics.
Conclusion
Minimizing cognitive bias in FDE can be achieved with a few practical remedies easily incorporated in the everyday work of the expert. Professional societies’ codes of ethics can be used to prohibit attorneys and investigators from sharing biasing information with FDEs. Phone scripts and impartiality memos can be used to warn attorneys and case submitters to refrain from conveying potentially biasing case information to the expert with whom they interact. Training in cognitive bias should be part of the general education for investigators, attorneys and FDEs.