Sequence Determination in Cases of Non-Intersecting Electrophotographic Print and Handwritten Entries

Introduction

A very common question in requests for expert examination is: “Was the document content printed first and then the signature was added or was it the other way round”?

A standard procedure is the examination of the sequence of color deposit layers in the point of intersection of two disputed entries. A suitable method for sequence of entries determination is to be chosen according to the writing instrument type and/or printing instrument combination. However, there are cases when the questioned entries do not intersect, usually they are signatures and printed text. In documents with electrophotographic print there are possibilities to evaluate the sequence of handwritten and printed entries. During the electrophotographic process, the whole sheet of paper is in contact with a printing cylinder from which the image is being transferred. Together with this image, random microscopic toner particles (Fig. 1) are being transferred onto the whole paper sheet area. These micro particles, inside handwriting strokes, are regarded as an intersection. The sequence of what is placed first on the paper/substrate is determined either by indirect evaluation of toner particle distribution or by direct optical evaluation of the toner particle and ink from the writing instrument.

Figure 1

Methods and Materials

The first step when examining the sequence of handwritten entries and printed text is verification of such facts as the document sheet size and coherence of the printing pattern. We have to be cautious with paper sheets different from the usual size, as there might be something wrong with the document integrity. It might mean that the original content of the document was cut off from the signature and replaced with new content printed on the remaining part of the paper, which is already covered in toner particles. The paper might also have gone through more than just one printing process, regardless of whether there is a printed image visible on the paper. If this is the case the results of the analysis might be distorted as it is impossible to pinpoint the printing process from which the random toner particles originate. Having verified the integrity of the printed image and excluded the possibility of multiple printing processes, we can proceed with examination of the sequence of entries. However, the hypothesis of multiple electrophotographic printing processes has to always be taken into consideration as well as the actual presence of toner particles both under the pen ink and on top of it. (Fig. 2).

Figure 2

Projectina DocuCenter Expert video spectral comparator is used for indirect optical evaluation of the sequence of entries, with the option to filter the image at different wavelengths of up to 1000 nm and under oblique illumination. In addition, a Dino-Lite® micro zoom camera is used with combined IR and UV illumination and with magnification of at least 230x—containing LEDs of 940 nm wavelength together with UV diodes of 395 nm wavelength.

For direct optical analysis and documentation close-ups a Hirox® KH 8700 digital microscope was used—initially with a MX 5040RZ objective. Together with adapters for double or half magnification it offers magnification ranging from 25x through 800x. There are also extensions available for coaxial or diffuse lighting, polarization of reflections and a variable-angle rota-head adapter. Based on partial results the system was upgraded with a new MX2500 objective lens with magnification range 30x through 2500x and coaxial or ring lighting. High quality optics and digital imaging makes it possible to observe objects at high magnification and with good depth of field at the same time. This property has proved to be very efficient with examinations on paper substrates. The topography of paper in micro-scale is very rugged and it is usually difficult to reach optimum focus of the object being examined in magnifications over 500x.

Experimental samples were prepared on plain sheets of office paper for digital print. Strokes were written with different types of writing instruments such as ballpoint pens, gel pens and rollers from the collection of writing instruments kept by the Institute of Criminalistics Prague. Various types of electrophotographic printing machines were used (both monochromatic and color, different manufacturers and different stages of wear) in print mode or copy mode—34 machines in total.

Research

In the preparatory stage of the project we analysed the order of writing against toner particles using a scanning electron microscope (SEM). Samples were analyzed in collaboration with our colleagues from the department of chemistry and physics of the Institute of Criminalistics Prague. Our intention was to use SEM with an integrated “focused ion beam” (FIB) to perform the toner particle cross section so as to determine the order of the layers in it. In the course of analyses we encountered a problem with the invisibility of the ink stroke in SEM imaging mode at high magnifications (5000 through 10 000x), which made it very difficult to correctly identify a toner particle placed in the area of the writing stroke. At the same time, it was not possible to observe / determine the order of ink and toner layers. Together with these SEM experiments, we also tested imaging possibilities of high-magnification digital microscopes, and thus verified the efficiency of such magnification for document examination.

On account of the above findings and also for the sake of practical reasons including the independence from the department of chemistry and physics, we decided to go with the option of optical methods combined with high magnification. For the purpose of the project we purchased a Hirox® digital microscope.

Based on observation and evaluation of the samples in the first stage of the research a new methodology for indirect determination of stroke sequence was created. Detailed description of the development and experimental verification of this methodology was published earlier [1]. For the sake of comprehensibility and completeness the description of the method is also covered in this article.

Indirect optical evaluation of the sequence of handwritten entries and electrophotographic print according to toner particle distribution

This brand new method is based on statistical evaluation of random toner particle dispersion in the blank region of the paper, i. e. outside of printed content. The level of contamination of the blank regions by toner particles is affected by a number of circumstances and varies for every printing machine. Apart from the wear of the photosensitive cylinder and the whole printing machine, contamination is also affected by tension settings for toner transfer from the photo cylinder onto the paper. The factors like air humidity, moisture of the paper and specific features of the paper such as thickness and surface structure also play their role. Printing on structured paper (rough surface with recessed areas) is a known weakness of the electrophotographic print. Toner powder transfers worse or does not transfer at all from the photosensitive cylinder onto areas not directly touching the cylinder surface during the printing process.

This property makes the ground for the indirect optical evaluation method according to the toner particle distribution detailed in the article called “Sequence Determination of Handwriting and Electrophotographic Print According to Distribution of Toner Particles” [1]. Assuming the document is being printed on a clean sheet of paper, the diffusion of toner particles must be more or less even. In cases when the handwritten entry was made first (assuming enough pressure was applied) and then the paper goes through a printing machine, we can observe the absence of or reduction in the number of toner particles in recessed areas of the strokes.

By means of statistical evaluation of random particle count in the recessed line region and its immediate surroundings, and by comparison of the two, it can be established if the handwritten entries were written before the print (Fig. 3).

Figure 3

The method is independent of the type and color of the writing instrument, with the exception of black archive inks that absorb IR (they contain carbonaceous pigments) and cannot be filtered off in the infrared region. No expensive microscopy equipment is necessary for practical use. A simple and affordable micro zoom camera is sufficient for this method to be effective. There are two basic conditions for successful application of the method; first—questioned strokes are indented into the paper substrate, second—the blank region of the document is covered by sufficient amount of random toner particles.

Results and discussion for the course of analysis by indirect method

For objective evaluation of toner particle count inside the handwritten stroke it is advisable to use spectral image separation by IR illumination (800 nm and higher) or filtering off the visible part of the spectrum up to 800 nm at magnification of at least 230x or ideally 300x. Spectral separation enables identification of toner particles even inside dark inks, or inhomogeneously colored handwritten strokes. Ideal solution is a Dino-Lite® micro zoom camera with combined IR and UV illumination and magnification of up to 250x (Fig. 4).

Figure 4

The first step of the analysis is verification of the random toner particles occurrence. The presence of toner particles is not necessarily uniform on the whole area of the paper sheet. That is why the examination is performed in the area surrounding the questioned entry. Based on experimental tests, we established the area for verification to be 0.5 mm². This area roughly corresponds to the region of the stroke in the microscopic field at optimum magnification of 300x. A typical quantity of toner particles in the area of this size does not exceed several dozen. For a conclusive difference in particles present inside and outside of the stroke the value was set to minimum 10 particles. See Fig. 5.

Figure 5

For the questioned entry analysis itself, it is necessary to choose such parts of the strokes that are the most recessed. For that reason, it is convenient to pre-examine the strokes in oblique lighting and then in combination with spectral image separation, which helps identify the most suitable parts. The chosen areas are the target of interest at 200x through 300x magnification. Together with automatic or manual selection of the colored stroke region we identify the best parts according to the edges of the stroke. Then we switch to spectral separation mode and count the particles in the selected area, see Figs. 6 and 7. In the next step we move the same selected area next to the stroke and count the number of toner particles, see Fig. 3. In order to eliminate random variation in the particle count, we repeat the procedure at least five times at other locations of recessed strokes.

Figure 6

Figure 7

If the resultant ratio of the number of particles inside the stroke to the particles outside of the stroke from at least five measurements falls within the range of 0 through 0.5 (i.e., ≤ 50 %) we can conclude that the strokes were made before the print. In case the resultant ratio is above 0.5 (i.e., > 50 %) the conclusion is “indeterminate”. There are three hypotheses that come into question:

1. Strokes were written after the print.

2. Handwritten strokes are not indented enough or a printer with better ability to transfer toner onto structured substrates was used.

3. The document in question could have gone through multiple printing processes without that being detected (the original image could have been cut off) and the first printing dispersed much more random toner particles than the other one.

In order to deal with such cases it is necessary to use the method of direct optical evaluation of the surface of individual toner particles in the stroke.

Direct optical sequence determination

The possibilities of determining the sequence of a handwritten entry against random toner particles by means of optical microscopy are given by the toner particle size, the position of the particle on the paper surface (e.g. on top of a fiber, in between fibers etc.) and the type of ink of the writing instrument. By all means, high enough magnification is critical here, at least 600x with coaxial lighting and with the option of adjustable polarization filter. It is necessary for all microscope types to have the resolution of the objective verified, to ensure that the 600x magnification is usable. In many types of microscopes it is necessary to have higher maximum magnification (e.g., 800x) since the highest possible magnification does not necessarily mean it is available for all applications due to insufficient image detail.

Some printing machines (especially new color multifunction printers) only transfer minimal amount of random toner particles, sized up to 5 μm, which limits the microscopic examination. For such cases, higher magnification of 1000 through 1500x (i.e., a lens with maximum mag. of 2500x) is necessary. Research has shown that in order to establish the sequence of entries using 600x through 1500x optical magnification, aggregates of yellow toner particles that are often present in areas of color electrophotographic content can be used in addition to black toner particles.

There are several articles dealing with the topic of direct analysis of toner particles in handwritten strokes [2],[3],[4],[5]. However, none of them has offered a comprehensive solution so far. The closest to our procedure is Francesco Dellavalle [6] who built and modified his own technical solution of a microscope. However, his article only covers ballpoint pens. Our first analyses were performed with lower magnification - 600x through 800x with a Hirox® KH 8700 digital microscope and MX 5040RZ objective lens. The method is based on the analysis of black toner particle surface in the area of a handwritten stroke and the aim is to determine whether or not it is covered by ink (writing paste). At magnifications of 600x to 800x the color ink deposit on toner particles cannot be distinguished. However, tests showed that reflective properties of some ballpoint pen inks help us distinguish between particles printed on top of the writing stroke and particles lying underneath the writing stroke. By the term “reflective properties” we mean the change of color hue (usually towards red-violet) in reflective diffuse illumination in contrast to a blue tint resulting from the use of polarization filter. At this magnification the method only worked for strongly reflective blue writing pastes (inks). To further improve the results it was necessary to equip the Hirox® microscope with a MX2500 objective lens for higher magnification (30x— 2500x). This lens was used in the second stage of the research. It is possible to examine and evaluate toner particles overlaid by ink from any writing instrument at 1000x through 1500x magnification with coaxial lighting.

The procedure being presented here was tested on a large amount of specimens created by various types of writing instruments and various blue and black inks combined with 34 different printing machines. Positive results can be obtained for even very tiny particles of 5 μm. At the same time it is possible to analyze clusters of yellow toner. Black inks are more difficult to analyze because detecting black toner particles in a black stroke area is very challenging. In order to facilitate detection of particles in the stroke we can pre-examine the position of toner particles with a Dino-Lite® micro zoom camera with combined IR and UV lighting, analogous to the indirect optical method. Still, maximum magnification of the micro zoom camera is only 230x and it is not possible to detect all particles including the smallest ones (5 μm). The auxiliary micro zoom camera is only used for better orientation and confirmation of particles found by Hirox®. Optimum solution would be infrared lighting used directly in combination with the digital microscope. However, like most of other microscopes, Hirox® has an IR filter in front of the camera chip that cannot be removed.

The reliability of the method was blind-tested on 22 specimens (ballpoint, gel and roller pens) with 100 % success rate. The only case when this method cannot be used is in combination with dark inks that absorb the IR and therefore cannot be filtered off. Detection and confirmation of toner particles in these cases is practically impossible.

Results and discussion for direct optical method

The test method is based on the fact that random dispersion of toner particles occurs during the printing process on the entire surface of the paper sheet. Toner particles or their clumps in the area of a disputed writing stroke are regarded as intersections of toner and ink (writing paste). Reflective properties of writing pastes/inks during microscopic examination with coaxial illumination are used to establish a conclusion.

If the examiner does not have enough experience or a writing instrument with poor reflective properties was used, it is necessary to use reference samples. Both captured variants of intersections for various writing instruments prepared in advance (e.g., at study testing) may be used as reference material. Alternatively, a reference standard of both intersection variants of selected writing instruments from personal collections can be used directly for specific cases. Determination of the writing instrument type is based on the questioned entry according to basic physical characteristics, such as the type (ballpoint pen/ gel or roller), color (blue/black), shade of blue (cyan/blue/purple), shade of black (absorbing IR/ not absorbing IR), reflection (low/high), and IR luminescence (yes/no). Following this procedure one or more writing instruments with identical parameters are chosen from the collection of writing instruments and used to prepare reference samples.

Examination with the electrostatic detection device (EDD) can only be performed after the analysis of toner particles in the writing stroke; otherwise there is a risk of contamination of the paper sheet surface with toner particles applied during EDD development.

The analysis itself is performed with a microscope with high magnification, i.e., 1000x to 1500x. With 300x through 600x magnification and polarization filter it is also possible, but this is limited to writing pastes with strong reflection. In blue inks we first examine a polarized image, which is completely glare-free. In polarized mode toner particles appear black and ink is blue (as opposed to coaxial reflective lighting when the surface of toner particles reflects purple glint and toner particles are difficult to distinguish). With black ink the identification of toner particles is very difficult. For this reason, it is necessary to use spectral separation of the writing stroke using IR. Some black inks absorb IR, just like black toner particles do, and that is why the writing stroke cannot be filtered off. If that is the case, the method cannot be applied. However, most black inks do not absorb IR. Detail of such a disputed entry is first captured in IR (above 800 nm) using the camera for the IR spectrum and at least 230x magnification, thereby filtering off the ink and leaving black toner particles visible. By means of comparing the filtered image detail with the enlarged image in the visible spectrum we can trace the identified toner particles that are to be examined using a reflective microscope with coaxial illumination.

For successful determination of the order in which toner particles and writing strokes appeared on a document, we search for particles expected to be covered by ink. Required properties are:

Particle size: from 5 microns

Position of the particles: in the middle part of the fiber, i.e. at the highest point of its surface or on a larger unstructured area between fibers in such a manner that the particle surroundings are twice as large as the particle diameter Particles located on the side (slope) of the fiber in slant position or in lower areas between fibers may not be covered by ink or they may reflect incident radiation at a different angle, which can cause misleading conclusions.

Having identified suitable particles, we can analyze them using coaxial illumination and 1000x through 1500x magnification. It is advisable to refocus continuously in order that the whole particle gradually comes in focus (particles usually protrude above the substrate surface).

If a toner particle is displayed in contrast mode to the surroundings, with black edges and only slightly tinted or white reflection in the middle (highest) part, then it means it is placed on top of the writing stroke surface. Particles on top of writing strokes often appear rounded. (Caution is necessary with toned glare of particles the position of which is inclined relative to the plane of observation and which may reflect the color of the ink). Particles covered by ink show strong reflection often with purple, golden or bronze tones. The reflective surface of particles often appears flattened.

In order to establish a conclusion, it is necessary to analyze at least three particles with optimal properties. In ambiguous cases, e.g., with weak reflection properties, comparative analysis with reference samples for both variants of intersections is performed, as mentioned above. In Fig. 8, 9 and 10 there are examples of reference samples for a blue writing paste, blue ink and black writing paste.

Figure 8

Figure 9

Figure 10

It is also possible to use yellow toner dots/clusters for analysis. During microscopic analysis printed yellow toner image is transparent and if combined with other colors it appears to be lying underneath them. A decisive factor in establishing the sequence of entries is the interaction of the writing paste/ink and the yellow toner layer formed by the melted toner particles. In electrophotographic printing the originally powdery toner particles form a compact melted surface with properties different from the properties of the substrate surface. If printing is first, the differences in surface properties are demonstrated in the quality of ink/writing paste adhesion. The writing paste forms clusters or runs to the edges of toner layer, especially to the approach side. Writing paste on the toner surface exhibits reflections with red hue. In contrast, with print over the writing stroke the writing paste adheres evenly in the whole area of yellow toner and is only dependent on the distribution of paper fibers. At the edges of the toner layer there is no perceptible change in ink/writing paste adhesion. The layout of ink/writing paste can be observed in polarized lighting mode. Blue writing paste combined with the transparent yellow layer acquires a greenish tint (not a reddish reflection). Color and reflective properties are examined in coaxial lighting mode. See reference sample in Fig. 11.

Figure 11

The topic of evaluating the shape of toner particles and shape differences between particles that are on top of the stroke surface compared to particles written over, has already been covered by several authors [3], [4], [5]. Just like these authors, we have registered increased incidence of rounded particles printed on top of handwritten strokes in the process of sample evaluation. However, this phenomenon did not show sufficient regularity in order to be the sole basis for sequence determination. Rounded particles are also found in cases when printing was there before writing. At the same time it is possible to observe flattened particles in the area underneath handwriting, but also anywhere else on the paper, even on top of handwritten strokes. Flattened particles are therefore not attributed to mechanical influence of the writing instrument but rather surface properties of paper. Toner particles are melted in a heat fixation unit (fuser unit) in the final stage of the printing process. In the liquid phase they may partially diffuse into the paper surface and hereby fixate. If there is some handwriting present on the paper sheet before it goes through printing, surface properties of the paper at that location change. Ink from the writing instrument can prevent the diffusion of the molten toner particles into paper, and they hereby retain their droplet shape. However, verification of this phenomenon was not the focus of our project and therefore it was not researched further.

Conclusion

Based on a series of conducted experiments, during which various combinations of order of written/ printed records were tested, it was verified that both options, a digital microscope with a minimum 600x magnification (ideally 1500x) and an indirect method for evaluation of toner particles distribution in handwritten strokes and their surroundings, offer a solution in some cases that previously seemed unsolvable.

The indirect method of establishing the order of entries is very easy to apply. Its implementation does not require costly special equipment or complex examiner training programs. Due to statistical comparison of toner particle occurrence the results are objective and can be comprehensibly interpreted during court proceedings. However, like other methods, this one also has its limits and application is not always possible.

The method of direct evaluation requires certain amount of skill and experience on the part of the examiner but once trained in the described procedure for both intersection variants using the test samples, obtained results are reliable. In most cases the method is successful but it requires expensive microscopic instrumentation. The most difficult step of the method is the search for toner particles in handwritten strokes made with dark inks. For this situation to improve, producers of microscopes would have to be involved in the technical solution of the IR filter in front of the camera so that it is made adjustable and IR could be used for sample illumination.

Not so long ago we considered the use of high magnification in the field of document examination ineffective; until 2010 usual working magnification range used to be 50 to 100x. The development of improved copy methods and materials (printing inks and toners) together with advances in microscope instrumentation has led us to change the approach and push the possibilities of optical methods for document examination. It still holds true that each case is specific. We have been gradually gaining experience, which leads us to recognize that it is possible to establish a relevant conclusion in growing numbers of seemingly intractable cases.