Technical Note: Signature Verification Algorithms: What They are, How They are Used, and How to Evaluate Them

2.a. What do SVAs do?

SVAs are designed to compare a questioned entry/signature to a control group and to present a result; however, specifically how they are implemented, the conditions they take into account, and how the results are utilized are subjects of discussion that are beyond the scope of this paper due to the significant differences that are present on an SVA-by-SVA and/or implementation-by-implementation basis.

SVAs can be developed for both static and dynamic signatures. Depending on the comparison metrics utilized by an SVA to analyze a signature, it is also possible within some SVAs to mix the two genres² of signatures in the same manner that an FDE can use both dynamic and static signatures during casework. [14].

Whilst the programming behind most SVAs are trade secrets, there are several common approaches that have been noted across the industry. For SVAs that examine static signatures, the SVAs can often be described as pictorial similarity calculators; they compare an image of a presented questioned signature to a series of known signatures, in order to determine whether or not there is agreement among any of the utilized geometric comparison metrics. For SVAs that examine dynamic signatures, SVAs calculate a conformity threshold—i.e. a level of conformity that characterizes the individual members of the control group (the known material) and keep that threshold as a pass/fail criterium to which the questioned signature is compared. As depicted in Figure 1, the general architecture of SVAs is the same between static and dynamic comparison; what changes is the data (depending on how the static image is digitized) and some of the actual metrics used.

Figure 1

Metrics that can be common between the two genres are geometric oriented parameters such as size and angles, area covered, placement etc., either of the entire configuration of writing or of specific segments of it [15]. Dynamic parameters that cannot be derived by static images can involve time derivatives of the captured parameters; the mainstream captured parameters (data channels) for DCS are the spatial coordinates X and Y (i.e. position on the vertical and horizontal axis), Force (usually expressed in pressure levels [16] and time. For example, the time derivative of X and Y can give velocity, acceleration and jerk³ for both axis but also for distance travelled (combining X and Y parameters via the Pythagorean theorem).

The selected metrics are calculated and compared, and according to the specifics of each SVA, each metric will be evaluated with a relative weight parameter (i.e. not all metrics are considered equally important). The metrics that are taken into account, as well as the relative weight given to each of them during analysis, is the core of an SVA and, as such, they are seldomly presented in the User Interface (UI) or even made public. Notwithstanding, the metrics and their relative weights need to be considered in the development of a validation study of an SVA [6, 17]. It has been observed that some analysis software will include metrics that are not trivial or obvious to the lay person or even the trained expert, e.g. area of bounding rectangle (D2_BOUNDING) or root mean square pen speed (V_RMS_SPEED) from SignatureScope [15]. It is to be expected that such metrics may be used in SVAs as well.

Furthermore, it has been observed [6, 17] that different alphabets, writing systems and/or scripts may require specific “tweaking” of those relative weights [18]. Therefore, class characteristics of the writing, such as writing system or alphabet, are a variable that should be taken into account in the development of a validation study depending on the proposed use case of an SVA.

2.b. How can SVAs be used?

SVAs have already been deployed as a means of identifying potential fraud in teller and retroactive deployment scenarios. Under these scenarios, the SVA produces a simple result stating whether a questioned signature has been executed by the author of the known signatures, based on the parameters of the deployment. In most circumstances, these results do not include detailed information of the metrics utilized to inform the result.

On the contrary, the theoretical forensic assistance use of SVAs would require exactly that—a reporting of the metrics used and their compatibility scores, either in the form of data sheets or as visual information (i.e. highlighting areas of interest on the signature XY visualization). This type of information can be used to assist the FHE by pointing out the features that they should focus their attention on (analysis phase) or to point out the features that they should have assessed during their analysis phase (validation phase). The potential of such use of SVAs is such that a possible integration of SVAs in the peer review process appears feasible as in other trace evidence fields [19]. Notwithstanding, the potential for introducing bias in a forensic application of an SVA must be studied in detail prior to implementation in any manner.

3.a. Methodology reasoning

With the various SVA uses in mind, an evaluation methodology is required to ascertain the validity of the results. Traditional logic for the collection of signatures revolves around obtaining writing samples that are representative of the range of variation of a particular writer, while minimizing the effects of any unwanted variables. Some variables that are generally considered include the collection timing as well as physical conditions [20].

When traditional logic for the collection of signatures has been utilized to generate samples for research, there are further considerations to ensure that the samples are representative of the various outcomes that may be possible. Standard outcomes may include genuine signatures, simulations, disguised signatures, and signatures that may be influenced by a miscellaneous factor [21]. These outcomes may be further elaborated to include simulations in which there is an attempt to copy the form, simulations in which there is not an attempt to copy the form, writers with poor signature repeatability/larger ranges of variation, alternate signature styles, medical issues, or unusual writing conditions.

Designing a signature sample set for a validation study of an SVA must take into account not only all of the above factors and outcomes, but many others as well.

For the evaluation of SVAs that compare static signatures, particular emphasis must be placed on the design of the collection sheets utilized for collecting the signatures. The collection sheets must be widely varied in design as the presence of a baseline, text, an overlapping signature, or any extraneous detail may lead to a misinterpretation of the algorithm [22].

For the evaluation of SVAs that compare dynamic signatures, the sample set should consider various properties of the deployed hardware. While the ideal scenario for the deployment of an SVA would include the deployment of one consistent hardware solution (i.e. digitizer and stylus) across the entire system, it is likely that there may be other hardware solutions deployed or even the intermingling of various hardware solutions. Furthermore, digitizers and stylus of different makes and models have different specifications, particularly with respect to their ability to record Force and the rate of capture of data points [16]. Simultaneously, hardware may fail or degrade over time due to wear and tear or aging of the pressure sensor [23]. As such, the sample set should be designed so that an SVA can be validated against numerous combinations of hardware configurations that may influence the data they evaluate. Finally, the sample set should also account for the physical conditions associated with the digitizer and stylus’ physical properties, such as whether they are tethered, slanted, or in some other configuration.

3.b. Signature sample creation

Signature sample creation can be divided in two categories: the samples intended for use as the group of “known” signatures (i.e. the reference point) and the samples intended for use as “questioned” ones to test the SVA. Even though there are different strategies in how to produce and maintain the “known” signature group (to account for changes in range of variation), this is beyond the scope of the present study and is not explored.

Sample signature creation should aim at accommodating all possible scenarios. As such, samples (for both categories) should include signatures executed over a period of time and not at one go. Different physical execution conditions should be taken into account if they are a possibility for the given intended use; e.g. if an SVA is to be evaluated for deployment at the teller, then the physical conditions (such as the height of the teller’s desk) should be replicated in the sample collection. An indicative flowchart of considerations is illustrated in Figure 2.

Figure 2

In addition to physical condition parameters, when examining DCS it may also be necessary to account for variables introduced by the hardware and software utilized as part of the deployed SVA. While an ideal scenario for an SVA is a completely consistent hardware/software solution, it is far more likely that some components of the solution may be replaced, such as individual digitizers or styli, each of which may have different technical specifications. It may be possible to overcome such differences (depending on how the SVA works when it comes to the unnormalized data), or to manufacture an SVA for specific deployment scenarios where only one type of software and hardware is deployed. Regardless, wear and tear (especially in EMR solutions [23]) can also affect the faithfulness of the dynamic data and hence possibly lead to false interpretations and rejections of samples by an SVA.

3.c. Signature sample considerations

Due to the variety of deployment scenarios and implementations for which SVAs are being developed, the testing procedures of a validation study for any SVA should mirror the procedures utilized by the particular SVA’s intended deployment scenario. It has been observed that in some studies [24] “problematic” samples or conditions are being left out in order to produce good results— this practice should not be followed when conducting validation testing of an SVA due to the potential for encountering these samples under real world scenarios, especially if aiming at a forensic application of such a tool.

The scenarios that need to be investigated for SVA performance (arguably after the “normal” samples have been dealt with) should certainly include simulations (of various degrees of sophistication), disguise, writers with poor signature repeatability/larger ranges of variation, alternate signature styles, medical issues, or unusual writing conditions, all of which may be encountered under real world scenarios.