CC BY
13Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 1 | 2024-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 1 | 2024 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 1 | 2024 год
THE METHODS OF AUTOMATIC LICENSE PLATE RECOGNITION
Zulunov Ravshanbek Mamatovich
Associate Professor of the Department of software engineering, Fergana branch of the Tashkent University of Information
Technologies, Fergana, 150118, st. Mustakillik, 185 E-mail: [email protected]
Akhmadjonov Ikhtiyorjon Rovshanjonovich
Master's student at the Faculty of Computer Engineering, Fergana branch of the Tashkent University of Information
Technologies, Fergana, 150118, st. Mustakillik, 185
Ergashev Otabek Mirzapulatovich
Associate Professor of the Department of Information Technologies, Fergana branch of the Tashkent University of
Information Technologies, Fergana, 150118, st. Mustakillik, 185 E-mail: [email protected]
Abstract: The results of the methods research of automatic license plate recognition are presented. The procedures of the localization region license plates in the image picture, normalizing the license plate image, license plate image segmentation, character of license plate recognition and syntactic analysis of the elements of the license plate. It is shown that an effective approach can be based on the applying methods of the Viola-Jones, Hough, the analysis of histograms of brightness and support vector machine. The described approach allows to obtain high recognition accuracy in different corners of the license plate relative to the camera. Also, the article outlines the key procedures involved in automatic license plate recognition, including localization, normalization, segmentation, recognition, and syntactic analysis. It also describes the use of algorithms based on binarization, contour extraction, and morphological image processing to localize license plate areas in images. Finally, it proposes an algorithm for localizing license plate areas in images, involving grayscale conversion, gradient operators, adaptive binarization, morphological operations, contour detection, and area selection based on license plate parameters.
Keywords: automatic license plate recognition, localization, normalization, segmentation, character recognition, syntactic analysis.
Introduction. Automatic License Plate Recognition (ALPR) systems have emerged as indispensable tools in various domains, ranging from motor transport enterprises to car parking facilities. These systems offer a plethora of functionalities, including monitoring vehicle presence, managing parking spaces, recording time durations, and enhancing road safety through effective vehicle tracking. The growing ubiquity of vehicles on roads
necessitates efficient ALPR systems to streamline operations and ensure compliance with regulatory standards.
Fundamental to ALPR systems are the underlying procedures of localization, normalization, segmentation, character recognition, and syntactic analysis. Localization involves identifying and isolating the region containing the vehicle's registration number within an image. Subsequent normalization
135
Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 1 | 2024-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 1 | 2024 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 1 | 2024 год
procedures aim to standardize the size and orientation of the license plate image. Segmentation facilitates the delineation of individual characters within the license plate image, enabling accurate recognition. The recognition process involves generating a string of characters corresponding to the license plate. Finally, syntactic analysis deciphers the elements of the license plate string, adhering to predefined standards set by regulatory authorities.
In summary, this study presents a novel approach to automatic license plate recognition, addressing the increasing demand for efficient and reliable systems in diverse applications. By leveraging advanced algorithms and methodologies, our proposed ALPR system offers a robust solution for enhancing vehicle monitoring, safety, and management across different sectors.
Methods. Study Design: The study aimed to develop an effective automatic license plate recognition system by implementing various methods including localization, normalization, segmentation, character recognition, and syntactic analysis. The methods employed in this research encompassed techniques such as Viola-Jones for localization, Hough methods and pixel brightness histogram analysis for normalization, contour analysis for segmentation, and the Support Vector Machine (SVM) method for character recognition.
Data Collection: The experiments were conducted using images containing vehicle license plates. These images were obtained from various sources to represent diverse conditions such as different lighting, angles, and distances. The data collection process involved capturing images with varying distances between the camera and the vehicle, as well as images with different angles of rotation in the plane, vertically in space, and horizontally in space.
Software Development: The algorithms for automatic license plate recognition were developed using the C# programming language and the Emgu computer vision library. The software was designed to run on a dual-core personal computer with Intel Core 5i processors, clock frequency 1.6 GHz, and 4 GB RAM, operating on the Windows operating system.
Experimental Procedures: The experiments focused on evaluating the performance of the developed algorithms in terms of localization time, recognition time, and overall processing time. Localization time was measured as the duration taken to detect and localize the license plate area in the image. Recognition time represented the time taken to generate a string of license plate characters. Total processing time was the sum of localization time and recognition time.
Results. Localization Performance: The experiments demonstrated the effectiveness of the Viola-Jones method for localizing the license plate area in the image. The results showed that the system could accurately detect and localize license plates across various distances from the camera, with localization times ranging from 0.67 to 0.94 seconds.
Recognition Accuracy: The recognition accuracy of the system was evaluated under different conditions, including rotations in the plane, vertically in space, and horizontally in space. The results indicated high recognition rates, with correct recognition percentages ranging from 37% to 100% depending on the angle of rotation. Notably, the system achieved 100% correct recognition when the license plate was not rotated.
Overall Performance: The total processing time of the system ranged from 0.93 to 1.23 seconds, demonstrating efficient performance in real-time applications. Despite variations in distances and rotation angles, the system maintained high recognition accuracy, meeting the requirements of GOST R 5057793 (group 1, type 1) with an accuracy of at least 85%.
Currently, automatic license plate recognition systems are in demand in a wide variety of areas. For example, they are used in the work of motor transport enterprises, service stations, car parking lots, etc. Such systems make it possible to control the presence of cars in the service area, determine the time for servicing customer cars, the number of free parking spaces, and record the time of stay car in a specific zone, organize automatic entry and exit of cars, etc. In addition, the ability to automatically recognize vehicle license plates is an important aspect of monitoring and ensuring road
136
Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 1 | 2024-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 1 | 2024 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 1 | 2024 год
safety due to the constantly increasing number of vehicles on the roads.
Automatic license plate recognition systems mean a software or hardware-software complex that implements automatic license plate recognition algorithms for recording events related to the movement of cars, i.e. to automate data entry and subsequent processing [5].
Automatic license plate recognition is based on the following procedures [7]:
1) localization;
2) normalization;
3) segmentation;
4) recognition;
5) syntactic analysis.
The first procedure is designed to detect and localize the area with the vehicle registration number on the image. Next, the found area is cut out from the original image and examined separately. Normalization consists of bringing the size and orientation of the image with the number obtained in the previous step to the required form. Here geometric transformations, noise reduction, brightness changes and contrast. The segmentation procedure ensures that the image is divided into familiar locations, i.e. highlighting areas of individual characters. The recognition procedure is designed to generate a string of license plate characters. The last procedure (parsing procedure) is performed to determine the elements of the string containing the number characters. These elements may vary according to the standards of the vehicle registration countries. For example, in the Russian Federation, the standard defined by GOST R 50577-93 is used (as amended by orders of the Federal Agency for Technical Regulation and Metrology)1.
Various algorithms are used to localize the license plate area in the image. For the most part, they are based on binarization, contour extraction, and morphological image processing. In this case, it is assumed that the original color image is converted into a halftone form. Image binarization consists of dividing all the pixels of a halftone image by brightness into two classes - object and background. Automatic license plate recognition systems can use both global and local
image binarization methods. However, adaptive approaches are more preferable due to the ability to compensate for the influence of noise on various parts of the image, for example, the distribution of shadows due to illumination inhomogeneity. Along with binarization, edge detection can also be used [1]. To eliminate small details and breaks, morphological image processing methods are often used. As a result of these procedures, connected sequences of binary image points contours are formed. At the final stage of the localization procedure, it is determined which of the resulting contours is the boundary of the license plate area.
Thus, we can propose the following algorithm for localizing the license plate area in the image:
1) convert the original color image containing the license plate into a grayscale form;
2) perform image processing with gradient operators, for example, Sobel or Prewitt, the result of which is an image whose value of each pixel is equal to the gradient module at the corresponding point of the original image;
3) perform adaptive binarization of the image obtained in the previous step;
4) perform a morphological closure operation with a rectangle as a structuring element;
5) determine the contours in the image;
6) identify areas limited by contours;
7) select the area whose parameters correspond to the license plate number.
An alternative approach for localizing the license plate area in an image is based on the ViolaJones method, developed and presented in 2001 by P. Viola and M. Jones [8]. The Viola-Jones method is one of the most famous methods for searching for objects in an image in real time. This approach allows you to find the number area in complex and atypical conditions. The Viola-Jones method is based on the use of a set of Haar characteristics. The Haar sign consists of adjacent rectangular areas that are positioned in the image, then the pixel intensities in the areas are summed, then the difference between the sums is calculated. The region detection stage of the ViolaJones method uses a window of a certain size that
137
Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 1 | 2024-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 1 | 2024 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 1 | 2024 год
moves across the image. The Haar sign is calculated for each area of the image over which the window passes. The presence or absence of an object in the window is determined by the difference between the value of the feature and the trained threshold. High accuracy of detection of specified objects in the image is ensured by the cascade classifier.
Approaches based on contour analysis can find numbers of different sizes and at different angles. However, they have several disadvantages:
1) the image of a car may contain many rectangular objects, similar in outline to a license plate;
2) relatively high computational complexity -even on a small image, the detection time can reach several seconds;
3) they are based on the analysis of number boundaries, which is not always possible in real conditions. For example, images of dusty cars may not have clearly defined boundaries.
The approach based on the Viola-Jones method seems to be more effective for localizing the number. The corresponding procedure actually analyzes the desired area for the presence in it of relations, points or gradients characteristic of the number, identified at the training stage using positive and negative examples. At the same time, using some known relationships, you can further improve the search efficiency. For example, the initial value of the scanning window can be set based on the dimensions of the license plate 520x115 mm according to GOST R 50577-93. The disadvantage of this approach is the relatively low degree of invariance to affine and projection distortions of objects in images.
After localizing the image area with the license plate, it is necessary to generate an image containing only the license plate and perform its normalization. In the simplest case, normalization consists of rotating the corresponding rectangular area so that its orientation coincides with the orientation of the axes of the image coordinate system. But, as a rule, you often have to trim the license plate horizontally and vertically. You can also perform filtering to reduce noise or enhance contrast.
As a rule, the selected area with a number is relatively small in size and contains the boundaries of the number, horizontal stripes outlining the bumper and radiator grille. Therefore, to determine the angle of rotation of the number frame, you can apply an algorithm based on the Hough transform for lines [2]. The algorithm is as follows:
1) lines are determined whose length is more than half the width of the number area;
2) a straight line is formed from the average values of all points of the resulting lines;
3) the angle between the resulting straight line and the horizon line is calculated. The next stage of normalization is to find the boundaries of the frame of the no-
license plate number plate. For this purpose, intensity histograms are used horizontally and vertically, respectively. To construct histograms, it is necessary to sum the values of all pixels of a binary image in rows or columns, then select the maximum and filter out all values less than 20% of the maximum.
The next procedure, as mentioned above, consists of identifying familiar places in the image (segmentation). The simplest approach for character segmentation is to use a predefined template [4]. The template can be represented as an image of dark rectangles corresponding to the symbols on a light background, as shown in Fig. 1. The main requirement for effective segmentation here is the correct determination of the license plate frame at the normalization stage, and any deviation from the actual will contribute to the deterioration of the segmentation of license plate symbols dimensions.
Fig. 1. Template for the arrangement of symbols on the vehicle license plate
The next approach is based on constructing a horizontal projection of average intensity [2]. The essence of this approach is that the average intensity in
138
Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 1 | 2024-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 1 | 2024 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 1 | 2024 год
each column of the number image is calculated and the columns in which the average intensity differs significantly from the threshold value are determined (Fig. 2).
Fig. 2. Histogram of brightness distribution of license plate pixels
Another method is based on contour analysis [6]. After obtaining contours that represent connected sequences of the binary image points, those that are the boundaries of the car number symbol areas are determined. This is achieved by filtering contours that satisfy certain requirements for the relationship of geometric characteristics [2]. An example of the operation of the corresponding algorithm is shown in Fig. 3.
f \
Fig. 3. Result of license plate character segmentation
The method based on the use of templates is very simple to implement and does not require complex operations associated with image analysis to search for characters, but for its operation it is necessary to accurately select the boundaries of the license plate frame, which is not always feasible in real conditions. Histogram analysis of an image is highly sensitive to noise and defects in the image, especially in the spaces between characters. As a result, histograms may not reveal clear peaks in the spaces between symbols. In this method, difficulties may also arise when selecting regional symbols, since under them there are symbols of country affiliation. Segmentation based on contour analysis is less demanding on implementation conditions, since it uses generalized geometric features.
To recognize characters, it is necessary to reduce each of them to a predetermined standard form. Typically, this comes down to scaling and binarizing the characters to achieve the best contrast between the characters and the background. After pre-processing, various recognition methods can be applied to the corresponding selected familiar places. Template matching methods, methods based on moment analysis, and support vector machines are often used [1, 3].
The advantages of template methods are ease of implementation and resistance to defects in the image of characters. The main disadvantage of template methods is the inability to recognize characters that are subject to affine and projection distortions and differ from the templates embedded in the system.
The advantage of using image moments to extract character features is that it is highly resistant to changes in image scale and other geometric transformations. The disadvantage of using them is their high sensitivity to noise and defects in the image, as a result of which characters may be incorrectly classified.
The advantage of the SVM method is that to build a classifier for character recognition, a relatively small training sample size is sufficient. In addition, it has a low error rate. The disadvantage of this method is that the classification process does not take into account the entire data set, but only the part closest to the boundary separating the classes. The support vector machine seems to be the most effective for recognizing license plate characters.
The drawn conclusions on the effectiveness of the implementation of each of the procedures were confirmed as a result of experiments with the developed software. The experiments were carried out on a dual-core personal computer with Intel Core i5 processors, clock frequency 1.6 GHz, 4 GB RAM, running the Windows operating system. The software is developed in C# language using the Emgu computer vision library. The results of the experiments are given in table. 1-4. In table Figures 2-3 show the results of the analysis of images of license plates obtained by rotating the image plane relative to the camera (Fig. 4).
139
Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 1 | 2024-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 1 | 2024 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 1 | 2024 год
Table 1. Recognition time depending on the distance to the object
Distance to Localization Recognition Total
object, m time, s time, s processing
time, s
1 0,67 0,26 0,93
2 0,69 0,28 0,97
3 0,84 0,29 1,13
4 0,94 0,29 1,23
5 0,81 0,32 1,13
6 0,89 0,31 1,20
7 0,85 0,29 1,14
8 0,91 0,29 1,20
9 0,79 0,31 1,10
10 0,85 0,31 1,16
Table 2. License plate recognition when turning in plane
Table 4. License plate recognition when rotating horizontally in space
Angle of rotation -25 -20 -15 -10 0 10 15 20 25
Percentage of correct recognition 50 82 95 100 100 98 96 79 37
Table 3. License plate recognition when otated vertically in space
Angle of rotation -70 -60 -40 -20 0 20 40 60 70
Percentage of correct recognition 53 87 98 98 100 97 98 90 52
Angle of rotation -40 -30 -20 -10 0 10 20 30 40
Percentage of correct recognition 63 95 98 100 100 100 99 92 57
Fig. 4. Rotations of the image plane on the plane (left), in space vertically (center), in space horizontally (right)
Thus, an effective automatic license plate recognition system can be built based on the ViolaJones method for localizing the license plate area in the image, Hough methods and pixel brightness histogram analysis for normalization, contour analysis for segmentation and the SVM method for character recognition.
The developed algorithms made it possible to ensure the recognition of license plates that comply with GOST R 50577-93 (group 1, type 1) with an accuracy of at least 85% at various rotation angles.
Conclusion.The study concluded that the developed automatic license plate recognition system, based on the Viola-Jones method for localization, Hough methods and pixel brightness histogram analysis for normalization, contour analysis for segmentation, and the SVM method for character recognition, was effective in accurately recognizing license plates under various conditions. The system's performance met the standards specified by GOST R 50577-93, making it suitable for practical applications in areas such as motor transport enterprises, service stations, and car parking lots.
References
1. Gonzalez R., Woods R. Digital Image processing. 3rd ed. New Jersey, Prentice Hall, 2008 (Russ. ed.: Tsifrovaya obrabotka izobrazhenii. 3-e izd., ispr. i dop. Moscow, Tekhnosfera Publ., 2012, 1104 p.).
2. Elizarov A.I., Apanasenko A.V. Metodika postroeniya sistem raspoznavaniya
140
Muhammad al-Xorazmiy nomidagi TATU Farg'ona filiali "Al-Farg'oniy avlodlari" elektron ilmiy jurnali ISSN 2181-4252 Tom: 1 | Son: 1 | 2024-yil
"Descendants of Al-Farghani" electronic scientific journal of Fergana branch of TATU named after Muhammad al-Khorazmi. ISSN 2181-4252 Vol: 1 | Iss: 1 | 2024 year
Электронный научный журнал "Потомки Аль-Фаргани" Ферганского филиала ТАТУ имени Мухаммада аль-Хоразми ISSN 2181-4252 Том: 1 | Выпуск: 1 | 2024 год
avtomobil'no- go nomera [Method of construction of systems of recognition of license plate]. Metodika postroeniya sistem raspoznavaniya avtomobil'nogo nomera [Bulletin of the Tomsk Polytechnic University], 2006, vol. 309, no. 8, pp. 118-122.
3. Kirpichnikov A.P., Lyasheva S.A., Obukhov A.V., Shleymovich M.P. Avtomaticheskoe raspoz- navanie avtomobil'nykh nomerov [Automatic license plate recognition]. Vestnik Kazanskogo tekhnologi- cheskogo universiteta [Herald of Kazan Technological University], 2015, vol. 18, no. 4, pp. 218-222.
4. Murygin K.V. Normalizatsiya izobrazheniya avtomobil'nogo nomera i segmentatsiya simvo-lov dlya posleduyushchego raspoznavaniya [Normalization of the Image of a Car Plate and Segmen- tation of Symbols for the Subsequent Recognition]. Iskusstvennyi intellekt [Artificial intelligence], 2010, no. 3, pp. 364-369.
5. R.Zulunov, S.Xadjyev. Staff Incentives Based on Kpi Principles. Periodica Journal of Modern Philosophy, Social Sciences and Humanities. 2023. - Т. 17. - p. 101-105.
6. Р.Зулунов, М.Махмудова. Технологии Искусственного Интеллекта и Нейрокибернетики. Periodica Journal of Modern Philosophy, Social Sciences and Humanities, 2023, Т.17, с. 90-93.
7. Р.Зулунов, А.Абдукодиров. Этические и правовые аспекты внедрения искусственного интеллекта. Research and implementation, 2023, 1/6, c. 14-20.
8. R.Zulunov. Sun'iy intellektni axloqiy va huquqiy muammolari. Journal of technical research and development. 2023, 1/1, c. 120124.
9. Р.Зулунов, Д.Ирматова. Использование технологий искусственного интеллекта. Журнал интегрированного образования и исследований. 2022, 1/6, с. 53-56.
10. Р. Зулунов, А.Горовик. Внедрение технологий искусственного интеллекта, нравственные и правовые нормы.
Conference on Digital Innovation: "Modern Problems and Solutions", 2023.
11. Абдулхамидов, А. (2023). СВЁРТОЧНАЯ НЕЙРОННАЯ СЕТЬ ДЛЯ КЛАССИФИКАЦИИ ХЛОПКА ПО СТЕПЕНИ РАСКРЫТИЯ. Engineering problems and innovations, 1(3), 23-34.
12. Азизжон А. и Эркин У. (сентябрь 2022 г.). Выбор камеры для распознавания хлопка-сырца и анализа его основных параметров. В 2022 году Международная конференция по информационным наукам и коммуникационным технологиям (ICISCT) (стр. 1-4). IEEE.
13. Abdulkhamidov, A. A. (2021). PROSPECTS FOR TELECOMMUNICATIONS SERVICES IN UZBEKISTAN. In СОВРЕМЕННЫЕ ТЕНДЕНЦИИ РАЗВИТИЯ ФУНДАМЕНТАЛЬНЫХ И ПРИКЛАДНЫХ НАУК (pp. 183-187).
14. Абдулхамидов, А.А. (2021). К ВОПРОСУ ОБ ОСНОВНЫХ ТРЕБОВАНИЯХ К ДАТЧИКАМ ЛИНЕЙНЫХ ПЕРЕМЕЩЕНИЙ.
141
