CC BY
23Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 4 | 2023-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 4 | 2023 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 4 | 2023 год
VERIFICATION OF STATIC SIGNATURE USING CONVOLUTIONAL NEURAL NETWORK
Akhundjanov Umidjon Yunus ugli,
Candidate of Technical Sciences, Senior Lecturer of Software Engineering Department, Fergana Branch of Tashkent University of Information Technologies named
after Muhammad al-Khorazmi e-mail: [email protected]
Abstract. This article is devoted to the development of a method that provides verification of handwritten signatures based on real samples obtained by scanning with a resolution of 800 dpi. Handwritten signature remains one of the most common identification methods and consideration of the problems of this promising area contributes to the search for a solution to this problem One of the main stages of recognition is classification. This article describes the results of handwritten signature recognition using a convolutional neural network. A database of handwritten signatures of 10 people was used for experiments. The signatures are digitized as color images with a resolution of 850^550 pixels. There are 10 genuine and 10 fake signatures for each person. Experiments were carried out with the reduction of signatures to the size 128*128, 256*256, 512*512 pixels. As a result of the study of this model, it has shown its effectiveness and practical suitability for use in biometric identification systems.
Keywords: Recognition, verification, handwritten signature, classification, False Rejection Rate (FRR), False Acceptance Rate (FAR).
Introduction
Pattern recognition is one of the most studied tasks in such areas as digital image processing, computer vision, biometrics, creation of intelligent security systems, access control, etc. Nevertheless, such tasks as face recognition, gesture recognition, text recognition, iris recognition, fingerprint recognition, printed and handwritten texts, and signatures continue to be of great scientific and practical interest in the field of pattern recognition.
The term "recognition" combines two concepts - "verification" and "identification". Verification is the confirmation of conformity of the submitted biometric characteristic of a person to a certain identifier, which is specified by the user. The procedure is performed by comparing the code (from the submitted biometric characteristic) with the codes stored in the database (DB) and corresponding to the predefined identifier of the user. Identification is the comparison of the code calculated from the biometric characteristic of a person with the codes stored in the database in order to authorize the user [1]. The most common secure personal authentication in biometrics is handwritten signature. It is widely used in many aspects, in banks, business transactions and documents that are approved using signatures.
Handwritten signatures are one of the oldest and most widely used biometric authentication methods for administrative and financial institutions due to its simplicity and uniqueness [2]. As technology has evolved, authentication methods have evolved as well. Handwritten signatures are classified as static (online signatures) and dynamic (offline signatures).
Static (off-line), offline handwriting recognition is performed after the text sample has been created and digitally recorded. The optically captured image data is then converted into a bit pattern [3]. In dynamic (online) systems, during the handwritten signature setup, additional information about the x and y coordinate sequences of the signature, information about the pressure force, speed, etc. is collected.
The high complexity of the signature research task is determined by the following factors [2]:
- The signature is brief, simple and not very informative;
- it can be copied using technical means;
- the examiner may be influenced by confounding factors;
- handwriting of different people is naturally similar;
- a person's signature is variable.
70
Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 4 | 2023-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 4 | 2023 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 4 | 2023 год
Handwritten signature recognition has been widely researched in the last decade, but it remains an open research problem that could show high verification of signature recognition accuracy.
The main goal of the developed signature verification systems is to distinguish genuine signatures from forged signatures. This is a challenging task, especially in a static signature recognition method that uses images obtained by photographing or scanning, where dynamic information about the signing process is not available.
To evaluate the efficiency of recognition and verification, we use statistical indicators such as the first-order error False Rejection Rate - FRR (the ratio of the number of erroneously rejected genuine signatures to the total number of genuine signatures), the second-order error False Acceptance Rate - FAR (the ratio of the number of erroneously accepted forgeries to the total number of forgeries) and the recognition error measure EER (the level of equal error probability at which FAR and FRR are equal) [1].
Further FAR and FRR will be determined by the formula:
FP
FAR = FPR = ■
1)
2)
FP + TN FPR = False
Positive Rate FRR = FNR = -
FN
FNR = False
FN + TP Negative Rate
FP (False positive) - False positive solution, also called 1st kind of error. The model predicted a positive result, but in fact it is negative.
TP (Truepositive) - True positive solution. The model predicted a positive outcome, the prediction matched reality.
FN (False negative) - False negative solution, also called a 2nd kind of error. The model predicted a negative outcome, but in fact a positive outcome.
TN (True negative) - True negative solution. The model predicted a negative result, the prediction matched reality.
In most cases, the results of classifiers' performance are evaluated by confusion matrices. Table 1 presents the objects of correctly defined classes (true) and incorrectly defined classes (false) for one of such matrices [4].
Predicted class True classification
Class 1 Class 2
Class1 True Positive (TP) False Positive (FP)
Class 2 False Negative (FN) True Negative (TN)
Number of objects in class TP + FN = total number of class 1 objects FP + TN = total number of class 2 obj ects
Table 1. Confusion Matrix
A database containing 200 images of handwritten signatures from 10 people with a size of 850^550 pixels was used as data for training handwritten signature recognition. For each person there are 10 genuine and 10 fake signatures. (Fig. 2 and Fig. 3) show examples of handwritten signatures from 10 people before and after pre-processing.
Figure 1. Examples of samples of handwritten signatures
Images of handwritten signatures were converted to grayscale and then to binary. For this purpose, the Otsu method was used. This method calculates the threshold t minimizing the average segmentation error, i.e., the average error from deciding whether image pixels belong to an object or background [5-6].
71
Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 4 | 2023-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 4 | 2023 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 4 | 2023 год
Figure 2. Examples of handwritten signature samples after preprocessing
To distribute the classes of signature images, catalogs were created, and two subdirectories were created in each catalog according to the names of the classes: genuine and forced.
The experiments were performed with the reduction of signature images to the sizes of 128*128, 256*256, 512*512 pixels, since neural network architectures oriented to such sizes of the initial data were used.
Literature review and methodology
The idea of convolutional neural networks is to alternate convolutional and subsampling layers (subsampling layers). The network structure is unidirectional (without feedbacks), fundamentally multilayer. Standard methods are used for training, most commonly the back propagation of error method. The advantages of this approach are convenient parallelization of calculations and the possibility of implementing the algorithms of network operation and training on graphic processors, relative stability to rotation and shift of the recognized image. Training
was performed using the classical method of error back propagation.
The paper [7] gives formulas for computing 76 features, and [8] describes 44 features for evaluating binary classification results. The article [9] gives formulas for the five most common functions for evaluating binary classification results represented by the error matrix and investigates some properties of these functions.
The multilayer neural network model used to obtain the results is:
1. Convolution layer: 3*3 kernel size, number of feature maps - 32 pieces, ReLU activation function.
2. Subsampling layer: selecting the maximum value from a 2*2 square.
3. Convolution layer: 3*3 kernel size, number of feature cards - 32 pieces, ReLU activation function.
4. Subsampling layer: selecting the maximum value from the 2*2 square.
5. Convolution layer: kernel size is 3*3, number of feature cards is 64 pieces, ReLU activation function.
6. Subsampling layer: the maximum value in a 2*2 square is selected.
7. Conversion layer from two-dimensional to one-dimensional representation.
8. Fully connected layer: 64 neurons, ReLU activation function.
9. Dropout layer performs thinning which is used to average the training results obtained.
10. Output layer: one neuron, activation function sigmoid.
Layers one to six are used to extract important features in the image, and layers seven to ten are used to evaluate the classification result.
Figure 3 shows examples of genuine and forged signatures.
a) b)
Figure 3. Examples of signatures: a) genuine; b) forged
Figure 4 shows the plots of training and validation accuracy, as well as the change in accuracy and loss during training on an example of one person's
72
Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 4 | 2023-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 4 | 2023 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 4 | 2023 год
validation data for signature images with a signature image resolution of 256*256 pixels.
a) b)
Figure 4. Example of neural network training on one person's signatures: a) training and validation graph; b) loss function graph
The network was trained on 90% images of handwritten signatures from 10 people.
Figure 5 shows training graphs on signatures of 10 people with dimensions of 128*128, 256*256, 512*512 pixels.
a) b) c)
Figure 5. Graphs of training and validation of the
neural network on the signatures of 10 people
with image resolution: a) 118x118; b) 256x256; c)
512x512
Results
200 images of handwritten signatures were used to train, validate and test the model. Half of them were images of genuine signatures and the other half were fake signatures.
To perform the experiments, Python software was developed using deep learning neural network models. This software realized several functions: preparing the dataset, collecting images with simultaneous preprocessing, and training on the collected data through a trained learning model.
The Accuracy function was used to evaluate the classification of our model. The authors [4] of the article believe that the Accuracy function determines the proportion of correct answers and can be briefly translated as correctness, and it is not recommended to call it accuracy.
Accuracy = ■
TP + TN
TP + TN + FP + FN Table 2 shows the recognition results for the Accuracy metric.
Recognition Recognition Recognition
No. of each accuracy with accuracy with accuracy with
128*128 256*256 512*512
person resolution resolution resolution
(%) (%) (%)
Person №1 90,5 95 91,1
Person №2 98 100 94,2
Person №3 75,2 76 75,3
Person №4 91,2 96,4 90,1
Person №5 80,4 93,2 82,5
Person №6 85,8 90,1 89,8
Person №7 93,5 94,2 92,7
Person №8 88,7 95 96
Person №9 79,8 90,3 89,5
Person №10 100 99,7 96
Table 2. Results of recognizing the signatures of individuals
To train and test the model, 200 images of handwritten signatures were used in the proportion of 8:1:1 respectively, half of which were images of genuine signatures and the other half were fake signatures.
The computational experiment was conducted on a computer, with a discrete Intel (R) HD Graphics 5500 graphics card (1 GB of video memory).
To create a handwritten signature recognition system, several preparatory algorithms were developed in Python language using deep learning models. The work of these algorithm can be divided into several stages: preparation of the dataset, image acquisition with simultaneous preprocessing, and training on the collected data by means of a trained learning model.
73
Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 I Son: 4 I 2023-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 4 | 2023 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 4 | 2023 год
Conclusion
As a result of model testing TP = 85, TN = 83,
FP = 7, FN = 5
Accuracy =-85 + 83-= 0,9333
85 + S3 + 7 + 5
International Conference on Computer Science and Engineering. - IEEE, 2017. - P. 821-826.
9. Sokolova M., Lapalme G. A systematic analysis of performance measures for classification tasks // Information processing & management. - 2009. - Vol. 45., iss. 4. - P. 427-437.
The correctness of training the neural network model showed the best result at the resolution of handwritten signatures of 256*256 pixels (Table 2).
The results of these experiments can be improved with the application of augmentation. Further research will focus on big data experimentation. This work aims to create tools for automatic analysis for detection of forged signatures.
References
1. Starovoitov V.V. Processing of iris images for recognition systems. Minsk: LAP LAMBERT Academic Publishing, 2018. - 188 p.
2. Bobovkin M. V., Ruchkin V. A., Protkin A. A. Actual problems of theory and practice of forensic signature research // Vestnik of the Moscow University of the Ministry of Internal Affairs of Russia. - 2017. -Vol. 2. - P. 109-115.
3. Hafemann L. G., Sabourin R., Oliveira L. S. Offline handwritten signature verification— literature review // 2017-th international conference on image processing theory, tools and applications. -IEEE, 2017. - P. 1-8.
4. Starovoitov V. V., Golub Y. I. Comparative study of quality estimation of binary classification. Informatics. - 2020. - Vol. 17, iss. 1. -P. 87-101 (in Russian).
5. Israfilov H. C. Investigation of image binarization methods // Science and Education Bulletin. - 2017. - Vol. 2. iss. 6 (30). - P. 43-50.
6. Yankovskiy A. A., Bugrii A. N. Criteria for selecting a binarization method for the processing of images of laboratory analyses // Automated control systems and automatic devices. control systems and automation devices. - 2010. - Vol. 153. - P. 53-56.
7. Choi S. S. et al. A survey of binary similarity and distance measures // Journal of systemics, cybernetics and informatics. - 2010. - Vol. 8., iss. 1. - P. 43-48.
8. Canbek G. et al. Binary classification performance measures/metrics: A comprehensive visualized roadmap to gain new insights // 2017
74
