A Two-stage CNN Based Computer Vision Framework for Automated Validation of Indian Bank Cheques

Chakraborty, Debjani; Sahoo, Projjal; Biswas, Argha; Maitra, Sujaan; Saha, Sourav; Halder, Biswajit

doi:10.2478/ias-2024-0011

Figures & Tables

Automated Detection of an invalid Indian Bank Cheque Image

Overview of the process flow of the proposed framework

Overview of the Resnet 101 architecture of the Mask RCNN Model

Flow-diagram of First Stage validation error reporting module

Annotated Bank cheque image for stage-1 Mask RCNN model

Flow-diagram of Second Stage validation error reporting module

Annotated Bank cheque image for stage-2 Mask RCNN model

Different types of unacceptable overwritten/strike-through handwritten characters

A few Bank Cheque sample images used for developing Stage-1 Mask RCNN Model

A few Masked Bank Cheque sample images used for developing Stage-2 Mask RCNN Model

Loss Graph During Training Phase of Mask RCNN Models

Quantitative Results: Detection Accuracy of Stage-Validation Module

Model	Payee’s Name (%)	Legal Amount (%)	Courtesy Amount (%)	Signature (%)	Date (%)
Madasu [19]	90.0	91.6	85.2	90.3	87.5
Hakim [14]	97.5	72.0	75.4	69.0	75.8
Alirezaee [18]	94.4	93.2	94.15	90.8	92.1
U-Net [29]	95.3	94.9	95.1	95.2	91.8
YOLO v8 [30]	96.2	94.8	95.1	91.0	93.3
Proposed Model	98.1	98.0	97.4	97.2	98.2

Challenges of extracting and recognizing data fields from bank cheque images

Challenges	Difficulties
Data deterioration	The post-binarization nose, including stamps, dots and lines are observed. The decrease of quality resulting from the removal of noise, lines, and backgrounds.
The problem of Skewness	Misalignment of the cheque during the scanning process. The segmentation and recognition of bank cheques pose significant challenges.
Distinct handwriting	The distinctive handwriting style complicates the process of data recognition. Various persons use several font sizes, directions, thicknesses, and angles.
Data superposition	Data overlap caused by adjacent words resulting in insufficient differentiation of the data.
Perplexity	“/” occasionally contacts the adjacent digits. The “/” symbol poses a challenge in the process of segmenting and recognizing numbers.
Document torn and folded	The corners are predominantly folded or ragged.
Variation in image contrast	The image is either excessively bright or excessively dark, which hinders the extraction of data. Issues with image camera calibration, subpar printing, and incorrect thresholding of the background.
Cheque Streaks	The presence of dark or bright streaks can be attributed to various factors, including scratches in the scan window, dirt on the camera calibration target, and malfunctions in the camera electronics.
Image compression	Difficulties arise when the image is compressed below the minimal requirement or when the image size exceeds the maximum range.
Integrated algorithm	There is currently no standardized method available that can extract all the information from a bank cheque simultaneously.

Summary of the Dataset

Cheques with Missing Handwritten Field	Cheques with No Missing Handwritten Field	Cheques with Overwritten/Strike-through Characters	Cheques with no Overwritten/Strike-through Characters
90	30	78	12

Qualitative Observations of Stage-1 Validation Module’s Performances

U-Net	YOLO v8	Proposed Mask RCNN based model

Qualitative Observations of Stage-2 Validation Module’s Performances

U-Net	YOLO v8	Proposed Mask RCNN based model

Quantitative Results: Detection Accuracy of Stage-2 Validation Module

Overwritten/Strikethrough class types	U-Net based model (%)	YOLO v8 based model (%)	Proposed Mask R-CNN based model (%)
Type - A	94.30	95.40	98.20
Type – B	93.90	94.56	98.03
Type – C	94.75	95.2	97.57
Type – D	95.39	95.42	98.50
Type – E	92.67	93.71	98.90
Type – F	93.25	95.83	97.82
Type – G	95.28	95.07	98.23
Type – H	92.33	94.89	97.90
Type – I	93.80	94.77	97.88

A Two-stage CNN Based Computer Vision Framework for Automated Validation of Indian Bank Cheques

Figures & Tables

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Figure 9.

Figure 10.

Figure 11.

Figure 12.

Figure 13.

Quantitative Results: Detection Accuracy of Stage-Validation Module

Challenges of extracting and recognizing data fields from bank cheque images

Summary of the Dataset

Qualitative Observations of Stage-1 Validation Module’s Performances

Qualitative Observations of Stage-2 Validation Module’s Performances

Quantitative Results: Detection Accuracy of Stage-2 Validation Module

Paradigm

My account