CC BY
17
Smart Video Number Plate Character Recognition and Speed measurement using Hybrid Optimization-based YoloV3
Manoj K. Bhosale, Shubhangi B. Patil, Babasaheb B. Patil, Dnyaneshwar S. Mantri
Abstract— In the present growing era of vehicular technology, number plate recognition is the prim in order to solve the multilevel problems of security. The Number Plate Recognition (NPR) using hybrid techniques (image processing + Optical Character Recognition (OCR)) is utmost important in design of security system, that automatically read and recognize the characters on a vehicle's number plate. The application areas considered are toll roads, parking areas, and other restricted zones. The parameters used for detection and validation are accuracy, precision recall and F1 score. The NPR system begins by capturing an image of the number plate using a camera or other imaging device. Then, the image is processed using image processing techniques to enhance the quality of the image and identify the number plate's location. Next, the OCR algorithm is applied to the image to read and recognize each character on the number plate accurately. The accuracy of the NPR system depends on the quality of the captured image and the efficiency of the OCR algorithm. According to the research papers, the NPR system's accuracy in recognizing Indian number plates is between 75-85%. For high speed vehicles Optimized YOLOV3 is used for detecting the number plates. Once the number plates are detected, character recognition can be performed using the Improved Convolutional Neural Network (ICNN). The NPR system has several practical applications in the transportation industry, law enforcement agencies, and parking management systems. The system can help automate toll collection, improve traffic flow, and enhance the security of restricted areas by identifying and tracking vehicles. Overall, the NPR system is an essential technology that can improve the efficiency and security of various transportation-related operations. Its effectiveness and accuracy make it a valuable tool for various industries and organizations.
Keywords— ICNN, NPR, OCR, YOLOV3.
I. INTRODUCTION
It seems that the advancements in information technologies have led to a demand for integrating vehicles into information systems. This can be accomplished by
Manuscript received October 27, 2023.
Manoj K. Bhosale, Shivaji University, Kolhapur, Research Scholar, Department of E&TC Engineering, MS, India.
Shubhangi B. Patil, Shivaji University, Kolhapur, Dr. J. J.M College of Engineering, Jaysingpur., MS, India.
Babasaheb B. Patil, Shivaji University, Kolhapur, PVPIT Budhagaon, Sangli., MS, India.
Dnyaneshwar S. Mantri, SPPU University, Pune, Sinhgad Institute of Technology, Lonavala, MS, India.
researching significant data offered by vehicles for informational and factual purposes, either by a person or by a smart system that can recognize vehicles by their number plates in the real world and divert the data to a theoretical method. Furthermore, with the increasing number of vehicles on the roads, it seems that there is a need for automated systems that can manage and monitor car parks more efficiently, using advanced technologies such as sensors, cameras, and machine learning algorithms. Such systems can help in keeping track of the number of vehicles. In addition, the integration of vehicles into information systems can also help in providing useful data for traffic management, such as real-time traffic flow, congestion patterns, and accident detection. The optimized information can be used to rate traffic flow, improve road safety, and reduce travel time for commuters. Overall, the advancements in information technologies have opened up new opportunities for integrating vehicles into information systems, which can have a significant impact on various fields and areas of work.
Advance number plate recognition system is grown into vehicle detection and it mainly consists of detection of number plate are, finding out breakdown character with recognition. The detected license plate image is split from each individual character at the conclusion of the license plate identification procedure. By doing this, unnecessary information is discarded in favour of gathering only the pertinent data required for character identification.
Automatic license plate recognition (ALPR) systems have become increasingly popular and widely used in recent years, thanks to the advancements in computer vision and machine learning technologies. These systems take pictures of license plates, extract the characters, and turn them into machine-readable text using cameras and specialized software. The data can then be utilized to automate a variety of processes, such as tracking and identifying vehicles, enforcing traffic regulations, and keeping track of parking infractions.
The application areas considered are toll roads, parking areas, and other restricted zones. The parameters used for detection and validation are accuracy, precision recall and F1 score. The NPR system begins by capturing an image of the number plate using a camera or other imaging device. Then, the image is processed using image processing techniques to enhance the quality of the image and identify the number plate's location. Once the number plates are
detected, character recognition can be performed using the Improved Convolutional Neural Network (ICNN). The NPR system has several practical applications in the transportation industry, law enforcement agencies, and parking management systems. The system helps to automate toll collection, improve traffic flow, and enhance the security of restricted areas by identifying and tracking vehicles.
The paper has different sections to go ahead with, section-II focuses on Literature survey and findings of other authors to explore, Section-III focuses on the research methodology and proposed work, the mathematical modeling of proposed MCSMO is explored in Section IV, while section -V discuss the results and with conclusions in section -VI.
II. LITERATURE SURVEY
It provides the in-depth details of the work carried out and is the basis for formulation of the problem analyzed in the paper.
This section explores various CNN based algorithms for number plate identifications, which have been shown to achieve high levels of accuracy in this task. These techniques involve training a CNN on a large dataset of license plate images to learn to recognize and classify license plates in different contexts. The study gives a summary of different CNN-based algorithms for license plate identification that have been demonstrated to be highly accurate for this task.
License plate detection and recognition can be challenging due to various factors such as variations in plate size, color, and format, poor lighting conditions, variations in fonts and characters, and inclination of the plate [1]. Degradation and fading of the license plate can also pose significant challenges for license plate detection and recognition systems. Over time, license plates can become faded, scratched, or dirty, which can affect the visibility and legibility of the characters on the plate [2]. Convolutional Neural Networks (CNNs) have drawn a lot of interest recently because of their effectiveness in image and video processing applications. CNNs algorithms are used to extract features from input data. [3] CNNs have demonstrated great potential in solving a wide range of problems, making them a popular area of research and development in the field of machine learning. CNNs are a special type of artificial neural network that have been shown to excel at a variety of tasks, including image recognition and classification. CNNs have been particularly successful is in license plate recognition (LPR) systems. LPR systems are used to automatically read license plates in images or videos, and CNNs have been shown to achieve high accuracy in this task [4].
In [5] paper that discussed various CNN-related techniques for image classification. In this study, three different CNN architectures - Inception, ResNet, and Dense-Net - were merged together to create a more powerful model. an architecture that uses new modules, including convolution layers, to enhance the network's performance.
ResNet, on the other hand, utilizes skip connections to help combat the vanishing gradient problem, allowing the network to learn more effectively. Finally, DenseNet adds dense feature connections to improve the model's performance. It also focusses on the technique to avoid overfitting is new data is close to training set with complex model. Two methods discussed were "Auto Augment," which increases the amount of training data by automatically generating augmented versions of the original images, and "Mix-up," which creates new training examples by blending pairs of images together. "Drop Block" and Dropout reduces the complexity of CNN.
In [6] a YOLO (You Only Look Once)-based network was used for object detection. YOLO is a popular object discovery algorithm that is known for its speed and accuracy. Unlike traditional object detection algorithms that use a sliding window approach, YOLO takes an image as a whole and divides it into a grid of cells, each responsible for predicting the presence of objects. For each grid cell, the YOLO meth-od predicts bounding boxes and class probabilities using a single neural network. This approach is faster than traditional object detection methods. the YOLO-based net-work was chosen for its speed and efficiency in object detection tasks. The use of this network allowed for real-time object detection, which is essential in applications where quick response times are critical, such as in autonomous driving or surveillance systems.
In [7] new type of neural network architecture unlike CNNs is proposed to reduce the spatial dimensions of feature maps, Capsule Networks use a type of capsule layer that retains spatial information in the form of vectors, called capsules. Convolutional layers, capsule layers, and fully connected layers make up the architecture of the capsule network. The convolutional layers' extract feature maps from the input im-age, while the capsule layers use dynamic routing to group capsules that represent similar properties together. The fully connected layers are used to make the final predictions based on the outputs of the capsule layers.
In [8] proposes a method for detecting and recognizing license plates in uncon-strained scenarios. The proposed method consists of several steps, including plate candidate generation, plate candidate verification, character segmentation, and character recognition it uses a combination of handcrafted features and machine learning algorithms to perform the various steps of the license plate detection and recognition process. They use techniques such as HOG (Histogram of Oriented Gradients) features, SVM (Support Vector Machine) classifiers, and connected component analysis to detect and recognize license plates in images.
Automatic Traffic Surveillance using Video Tracking proposed most economical video surveillance technique in different light condition. The proposed technique 2-lines algorithm and vehicle classification using kalman filter for day time and headlight based detection for night time which helps in successful tracking of vehicles. The license plate detection uses Edge detection, Gaussian Analysis, Feature extraction and character recognition which makes it robust to detect license plates in both day and night conditions [3].
Vehicle speed measurement and number plate detection using a real-time embedded system can be achieved by combining computer vision techniques and an embedded hardware platform. This setup allows for accurate and realtime processing of video or image data to estimate vehicle speeds and detect number plates. Install a camera in a suitable position to capture the view of the road or the designated area where vehicles will pass. The camera should have sufficient resolution and frame rate to capture clear images or video footage of the vehicles. Choose an embedded hardware platform capable of real-time processing, such as a microcontroller with sufficient computational power or a dedicated embedded system-on-chip (SoC). This system will be responsible for running the computer vision algorithms and making decisions in realtime. The camera captures continuous video or images of the moving vehicles in the designated area. Utilize computer vision algorithms for real-time vehicle detection. Techniques like deep learning-based object detection or feature-based methods can be used to identify and locate vehicles in the video frames. Employ number plate detection algorithms to locate and extract number plates from the detected vehicles. This could involve using image processing techniques like edge detection, character recognition, or deep learning-based methods. Perform number plate recognition (Optical Character Recognition - OCR) on the extracted number plates to obtain the alphanumeric information from the plates. Ensure that the processing pipeline meets real-time constraints to provide immediate feedback and responses. Optimization of algorithms and hardware acceleration (e.g., using hardware accelerators like GPUs or specialized coprocessors) may be necessary to achieve real-time performance. The implementation details and choice of specific algorithms will depend on the complexity of the system, the hardware platform, and the performance requirements. Real-time embedded systems offer a compact and efficient solution for on-the-spot vehicle speed measurement and number plate detection, making them suitable for traffic management, surveillance, and other applications where real-time information is critical [13].
Single-camera vehicle speed measurement refers to the process of estimating the speed of vehicles using only one camera. This method is commonly used for traffic monitoring, speed enforcement, and surveillance applications where a cost-effective and straightforward solution is required. Position the camera in a way that it captures the view of the road or a specific section of the road where the vehicle's speed needs to be measured. The camera should be installed at an appropriate height and angle to obtain clear images of the vehicles as they pass through the monitored area. Calibrate the camera to determine its intrinsic and extrinsic parameters. This process involves identifying the camera's focal length, distortion, and its position relative to the road surface. Calibration is essential to account for any perspective distortions and ensure accurate speed measurements. Utilize computer vision techniques to detect and track vehicles in the captured video or image frames. Object detection algorithms can be employed to identify the bounding boxes around the
vehicles, and tracking algorithms help associate the same vehicle in different frames to establish its trajectory. Determine the time taken by each vehicle to traverse the specified distance. The timestamps of the video frames or the frame rate of the video can be used to calculate the time accurately. Once the distance and time information are related as (Speed = Distance / Time).
Display the speed measurements on a screen or record the data for further analysis and reporting. Depending on the application, the speed information can be used for traffic management, enforcement actions, or traffic flow analysis. It's important to consider several factors that can affect the accuracy of single-camera vehicle speed measurement, including camera resolution, frame rate, calibration accuracy, environmental conditions, and the presence of occlusions or vehicles traveling at varying speeds. While single-camera methods provide a cost-effective solution, they may not be as accurate as more complex multi-camera systems, especially in challenging scenarios. However, with proper calibration and processing techniques, single-camera speed measurement can still provide valuable speed estimates for various applications. [15].
Vehicle Speed Determination from Video Stream Using Image Processing suggested the hybrid algorithm for detecting the moving object which will overcome the drawbacks of background subtraction algorithm. Here in adaptive background subtraction technique is combined with three frame difference methods. First to construct matrix of corresponds to the current frame to decide whether the pixel is in motion or stationary. After that adaptive background subtraction method is applied. For noise removal trial and error method is applied. After tracking the vehicle next to calculate the centroid of the object. Now the speed is measured on the basis of number of frame required to enter and leave the object to pass the scene and distance it cover [16]. According to the survey it is found that, the NPR system's accuracy in recognizing Indian number plates is between 75-85%. For high speed vehicles Optimized YOLOV3 is used for detecting the number plates.
III. RESEARCH METROLOGY AND PROPOSAL
The proposed approach shown in Figure1 is composed by three main steps: Vehicle detection this step involves detecting the presence of vehicles in an image or video feed. License plate detection is to locate the license plate within the image. Optical character recognition (OCR) is to extract the alphanumeric characters from the plate using OCR.
Algorithm Development Fig. 1. Proposed Work Flow.
Traffic management systems play a promising and difficult role in computer vision applications and intelligent traffic systems. Many standard models identify the vehicles by
utilizing the bounding box representations and lacks in it performance for providing the vehicle locations.
A number of the present ways are analyzed that are represented in Table 1. YOLOv3 [8] provides higher accuracy on a sunny day and lowers the minimum error rate when police investigation multiple vehicles. However, it provides less accuracy with rainy dataset whereas scrutiny with sunny dataset. RSE-Net and YOLO [9] provides increased cryptography potency by victimization marginal variety-ty of parameters. Yet, it will increase the procedure quality in detecting with the sweetening procedure. Improved YOLOv3 [10] increases the detection speed and accuracy even trained with larger quantity of traffic videos. On the opposite hand, it causes slow convergence and coaching speed and also leads to enlarged delay. YOLOv3 [11] utilizes the error correction technique for correcting the real-world coordinates. However, considering the similarity and vehicle space ratio, its low performance that might cause detection failure. DNN [12] provides improved accuracy with less time interval for camera frames. However, this needs advanced learning that is computationally integrated. Deep CNN [13] used to accurately establish accidents in live video. Also, it has powerful representation with less parameter. But, it provides less stability regarding the extraction of attention region. Hence, on considering these challenges it is necessary to introduce a new traffic management system through the deep learning method.
A. Research Methodology
The ever-increasing variety of police investigation cameras places a high demand on efficient video encoding. Though today's video encoding standards have greatly improved the efficiency of video encoding, they're designed for general purpose video rather than surveillance video. Multiple vehicle detection could be a promising and difficult role in intelligent transportation systems and laptop vision applications. Most existing strategies find vehicles with bounding box illustration and fail to supply the situation of vehicles. However, the location info is vigorous for many time period applications corresponding to the motion estimation and flight of vehicles moving on the road. Monumental advance has tested throughout the years within the space of traffic police investigation by the expansion of intelligent traffic video surveillance system. The main objective of this proposal is to style and develop a new video traffic monitoring system mistreatment advanced deep learning. This model is able to handle the video traffic police investigation by measurement speed of vehicles and recognizing its variety plates. The process steps of the projected models will be (a) knowledge collection, (b) vehicle detection (c) number plate detection, and (d) plate character recognition. The initial method will be the data collection, during which the traffic video data will be gathered. Further, the vehicle detection will be performed by the Optimized Yolov3 deep learning classifier, in which the parameter optimization will be performed by the Coyote optimization algorithmic rule (COA) [14] and Spider Monkey Optimization (SMO) [15]. For the high speed vehicles, the Optimized Yolov3 will be used for detecting
the number plates. Once the number plates are detected, plate character recognition will be performed by the Improved Convolutional Neural Network (ICNN). Thus, the information about the vehicles, which are violating the traffic rules, can be conveyed to the vehicle owners and RTO to take further action for avoiding accidents. Experimental results show that the proposed method improves performance when comparing over conventional models on different light and whether conditions, this ensures the safety of the transport system.
B. Pre-processing
YOLOv3 (You Only Look Once version 3) is an object detection algorithm that uses a single neural network to detect objects in an image. YOLOv3 is based on a convolutional neural network (CNN) architecture and can detect objects in real-time with high accuracy. Optimization with Coyote Spider Monkey Optimization MCSMO can improve the performance of the YOLOv3 network and reduce the computational complexity. By using YOLOv3 with MSMCO optimization for vehicle, speed, and license plate detection, it is possible to improve the accuracy and speed of the detection process.
IV. MATHEMATICAL MODELING OF PROPOSED MCSMO
Here, the newly recommended MCSMO algorithm has been developed with a combination of COA as well as SMO algorithms. The COA algorithm has utilized a very low amount of fuel cost as well as performed well over other models. Similarly, the SMO has been effective, and independent in performing the function. It has also been defined as the most reliable as well as robust over other models. Even though these algorithms have various advantages, it has some limitations, so in order to overcome the restriction as well as to design the enhanced model, the algorithm MCSMO has been implemented. In the recommended MCSMO algorithm, the position updating has been made with the aid of position and it is defined in Eq. (1).
NW _ FT (T )
mean{{NW _FTa")COA, (nw _FTKl)smo} (1) = + std jNW _FTa")COA, (nw _FThJ)mq} 2
COA [26]: In the COA algorithm, coyotes have been regarded as the type of species, which belongs to the Canis Latran variety in the North American region. It has been considered as the population depended on algorithms that have been based on swarm intelligence as well as the evolutionary heuristic and motivated by the adaption of the coyotes. The COA has the ability to offer a balance among both the exploitation and exploration phases during the process of optimization issues.
In COA, the coyotes have the potential to form several packs, as well as the leader of each pack, is defined as the alpha. The population in COA has been split into Ta and Tb
Here, the number of coyotes in the pack is denoted as Tb and the number of the pack is termed as Ta. In the initial phase, the amounts of coyotes are similar for each pack. Hence, the total population has been sorted through Ta and Tb.
In order to resolve the issues in optimization with COA, each coyote is considered as the solution as well as the social situation of the coyotes that includes several decision variables like, snowpack hardness, gender, temperature, snow depth, and social status are the cost for objective functions. The social condition of the coyotes for the pack at the time is derived in Eq. (2).
SoCri^ = c =(cl, c1,■■■, cA) (2)
Here, the adaptation in the coyotes to the environmental conditions is determined as the cost for fitness function that is represented as FTb'" e B.
In COA, the population of coyotes has been initiated in the beginning phase. Each of the coyotes is generally in social condition initially, because it is a stochastic-based algorithm. In accountant with random values for b'h coyotes in the pack for d'h dimension has been equated in Eq. (3).
SoCnati _ LBd + ed (UBd - LBd )
(3)
FTba'" _ g (soCnab-" )
(4)
prb _ 0.005 • T2
(5)
alpa'" _ {SoCn
b argb={1,2,-Tb} ming
(SoC<,fi)}
(6)
All the information from the coyotes has been computed as the cultural tendency expressed in Eq. (7).
CTa/ =
Ca
Ca
+Ca
Tbisodd
otherwise
(7).
Here, the ranked social condition that belongs to the coyotes has been termed as Ca'u . Then, the cultural tendency has been computed as the median social conditions for all the coyotes. Then, the term AGab" e B denotes the age of each coyote as well as sorted in the COA algorithm. On regarded to the randomly selected parents as well as social situations, the birth of the new coyotes has been derived as in Eq. (8).
put _
SoCna*d
soCna? D„
rdd < SPord _ dj rdd > SP + SAord _ d 2 otherwise
(8)
Here, the random dimensions on the issues have been termed as dx and d2 , and the random coyotes that belong to the pack is indicated as el and e2. The association and scatter probability that offers the cultural tendency has been denoted as SA and SP accordingly. The random values among [0, 1] attained through uniform probability are indicated as Dd.
The probability SA SP is derived by using the Eq. (9) 1
Sp _
Here, the random number by utilizing the uniform probability range over [0, 1] is termed as ed the upper and
lower bounds for the d 'h decision variables are indicated as UBd and LBd accordingly. The adaption of the coyotes with the environmental condition is derived as in Eq. (4).
DN
(9)
Here, the term DN defines the dimension of the search
space.
S _(1 - Sp ) A 2
(10)
Then, the coyotes have the ability to go out from packs or participate in other packs. The coyotes, which go out from the pack appeared along with the probability prb as well as is given in Eq. (5).
In terms of COA, each of the coyotes' packs has a maximum value of 14 coyotes; because the value prb is not greater than 1. Hence, cultural and interaction diversity has been given between coyotes.
In the COA algorithm, the term alpha has been derived for a pack at the time given in Eq. (6).
In the COA, there are packet effects ^ and alpha effects recommended for the purpose of depicting the cultural interaction inside the pack between coyotes. Both the effect has depicted the cultural difference, but the first one is through the random coyotes to the pack CR1 into alpha as well as the one through the random coyotes CR2 into the cultural tendency of the pack. Uniform distribution probability has been utilized in selecting the random coyotes. Then, the term ^ and has been derived as in Eq. (11) and (12)
£ _ alpaM - SoCnaM
CR1
£ _ CTa'u - SoCn,
a,ti CR1
(11)
(12)
Therefore, the new social condition of the coyotes along with the alpha as well as the pack impact has been derived as in Eq. (13).
NW _ SoCnab"= + g2.#2 (13)
Here, the weight of the alpha coyotes and the pack impact has been termed as g1 and g2 . Both the terms g1 and g2 are considered as the random values between the range [0. 1]. Then, the new social situation has been equated as in Eq. (14).
d
2
d
d
2
2
NW _ FTr= g Ko^'J
(14)
At the end, the social condition of the coyotes that adapts itself better to the environmental conditions is utilized as the solution for global issues.
SMO [27]: SMO is defined as the meta-heuristic methodology that has been inspired by the foraging behaviour of the spider monkey that has been based on the fission-fusion social structure.
Global Leader Phase (GLP): in this phase, the solution updating has been dependent on the selection probability that is defined as the fitness value function. From utilizing the fitness function Ih the fitness FTh has been computed by using Eq. (15).
1 ifIh > 0
FF = FTl =
1 + h
1 + ABS (Ih ) iflh < 0
(15)
PRBh = e
FF,
-or
S FFh
h=1
PRK = 0.9 x-
FT,
(16)
mx FF
+ 0.1
speed. But, it consumes more time for training the dataset. In order to overcome the difficulties, the MCSMO algorithm is utilized.
OBFU =
{epro-v3.ePcNN ,hncLY
argmm
i,hnc
1
,LRTO_V3 acc + prn
(18)
Here, the term PRBh is denoted as selection probability that has been defined as the roulette wheel selection. The value of fitness function FTh on the hth SM has been defined and then its probability of being chosen in the global leader phase has been calculated by utilizing the following Eq. (16).
Here, the term OBFU denotes the objective function, is given as epochs in the YOLOV3 between [2-20], epCNN is the epochs in CNN between [5- 255] the term hncYO-V3 depicts the hidden neuron count in YOLOV3 between [5255], hnCNN is the hidden neuron count in CNN between [50-100] and the term LRYO-V3 is defined as the learning rate in YOLOV3 between [0.01-0.99] are optimized in order to attain the traffic vehicle surveillance system along with maximized accuracy and precision. Here, prn precision as well as accuracy is termed as acc and represented in Eq. (19) and (20)
acc =
(ja +jb)
( ja + jb + is + iv)
prn = ■
ja
ja + is
(19)
(20)
In order to update the position, the SM has utilized the knowledge of the global leader that has experienced the neighbouring SM as well as its own persistence. Then, the position updating the formula for SMO in this phase is given in Eq. (17).
NW_FThj = NW_FThj + F(0,1)x(glt -NW_FThl) + F(- 1,1)y.(NW _ FTkl- NW _ FThi) (17)
Here the terms is ja , jb , is and iv refers to the false positive, true negative, false negatives, and the true positive accordingly.
Start
Here, ini dimension, the position of the global leader is
termed as gli . Then, the position updating has been split into three major components. They are a) the first component has shown the persistence of the parent of SM b) it has shown the attraction of the parent SM onwards global leader and c) it has been utilized to manage the stochastic behaviour in the algorithm.
Improved Convolutional Neural Network-based number plate Character Recognition
In this phase, the detected number plates are given as the input for character recognition; here the improved ICNN techniques have been used to detect the characters in the number plate. Here, the YOLOV3 model has the ability to detect the position of the vehicle as well as it is more robust for lighting adaptability, and speed calculation. But the optimization and dataset are limited in this model. CNN has the ability to outperform well in terms of accuracy and
Pmilion I |*Jjlini; \ MivjSVf)
Fig 2. Number plate detection for automatic traffic video surveillance system using optimized YOLOV3.
At the end, the character recognition is detected and the alert has been conveyed to the RTO and vehicle owner to enhance the road safety.
V. RESULTS AND DISCUSSIONS
The proposed traffic surveillance system using videos has been implemented in Python, and the experimental analysis has also been carried out. Here, positive measures and negative measures are used for evaluating performance. The Chromosome length taken was 3, and the maximum iteration were 20. The algorithms like Particle Swarm Optimization (PSO) [18], (JAYA) [19], Region-Based Convolutional Neural Networks (RCNN) [21], FAST-RCNN [22], FASTER-RCNN [23] are used.
The dataset contains the information about car stock video footage, which is attained using the given link "https://www.videvo.net/stock-video-footage/car/", "Access Date: 2022-10-28.
Real Time Access Date: 2022-10-28. The real time Data set used for the performance evaluation of MCSMO is given in Fig3(a), (b), (c), (d).
f 1 - S = -
(21)
prn + SS
Here, the sensitivity is denoted as and it is equated in Eq. (22)
ja
snstvy = -
(22)
Re cl = ■
ja
ja + iv
(23)
B. Performance estimation for vehicle, speed and number plate detection over algorithm
Table 1: Performance estimation for vehicle, speed and number plate detection over all algorithm
TERMS PSO [28] JAYA [29] COA[26 ] SMO [27] MCSM O Proposed
"Accuracy" 80.2 4 86.4 1 93.82 94.0 4 97.53
"F1-Score" 71.5 4 77.0 6 83.69 85.5 7 87.01
"Precision" 79.5 4 85.7 1 93.12 93.3 4 96.83
"Recall" 65 70 76 79 79
Also The Performance estimation for vehicle, speed and number plate detection on the traffic video surveillance system for dataset 1 over various algorithms in terms of Accuracy, F1-Score, Precision, and d) Recall" is Shown in
Fig 4(a)," (b), (c) & (d).
-0- PSO
-0- JftYA -0- COA
-<2>" SMO -Q- PROPOSED
Learning Percentage ,' !
Fig. 4(a). Accuracy.
Fig 3: Real time data set A. Performance metrics
The various performance measures, which are used for traffic video surveillance systems are given below. (a) F1 score: The F1 score is denoted as and it is equated in Eq. (21)
SS ■ prn
The Accuracy of proposed MCSMO is 21.55%, 12.87%, 3.95% and 3.71 % more as compared to PSO, JAYA, COA and SMO shown in Fig 4(a).
S eo-
ja + iv
(b) Recall: The Recall is denoted as and it is equated in Eq. (23)
O- O-
//^ PSO
y -0- JAYA
COA
-O" SMO
-Q- PROPOSED
Learning Percentage (%)
Fig. 4(b). FI-Score.
The FI-Score of proposed MCSMO is 21.62%, 12.91%, 3.97% and 1.61 % more as compared to PSO, JAYA, COA and SMO shown in Fig 4(b).
Fig. 4(c). Precision.
The Precision of proposed MCSMO is 21.74%, 12.97%, 3.98% and 3.74 % more as compared to PSO, JAYA, COA and SMO shown in Fig 4(c).
The Recall of proposed MCSMO is 21.54%, 12.86%, 3.95% and 0.00 % more as compared to PSO, JAYA, COA and SMO shown in Fig 4(d).
I
Fig. 4(d). Recall.
C. Performance estimation for vehicle, speed and number plate detection over classifiers.
Table 2: overall performance analysis for number plate detection over various classifiers.
TERMS PSO [28] JAYA [29] COA[26 ] SMO [27] MCSM O Proposed
"Accuracy" 87.8 7 85.2 9 82.60 88.0 5 95.38
"F1-Score" 81.9 6 80.8 0 78.84 82.7 8 88.56
"Precision" 87.1 7 84.5 9 81.90 87.3 5 94.68
"Recall" 77.3 3 77.3 3 76 78.6 6 85.33
Also The Performance estimation for vehicle, speed and number plate detection on the traffic video surveillance system for dataset 1 over classifiers in terms of Accuracy, F1-Score, Precision, and d) Recall" is Shown in Fig 5 (a), (b), (c) & (d).
35 55 £5 TS SS
Learning Percentage (W)
Fig. 5(a). Accuracy.
The Accuracy of proposed MCSMO-ICNN is 8.55%, 11.83%, 15.47% and 8.32 % more as compared to PSO, JAYA, COA and SMO shown in Fig 5(a).
«0
I
Fig. 5(b). FI-Score.
The FI-Score of proposed MCSMO-ICNN is 8.05%, 9.60%, 12.33% and 6.98 % more as compared to PSO, JAYA, COA and SMO shown in Fig 4(b).
s ss sb
Learning Percentage (%)
Fig. 5(c). Precision.
The Precision of proposed MCSMO-ICNN is 8.62%, 11.93%, 15.60% and 8.39% more as compared to PSO, JAYA, COA and SMO shown in Fig 5(c).
35 55 65 7S 85
Learning Percentage (%}
Fig. 5(d). Recall.
» 55 Û5 11
L,e*mtno Percentage
The Recall of proposed MCSMO-ICNN is 10.35%, 10.35%, 12.28% and 8.48% more as compared to PSO, JAYA, COA and SMO shown in Fig 5(d).
VI. CONCLUSIONS
The paper presents the hybrid MCSMO algorithm for vehicle number plate detection and character recognition. It uses the combination of OCR and YOLOV3 as classifier to obtain the results for real time data set. The proposed MCSMO shows the maximum accuracy and precision values for number plate detection as compared to PSO, JAYA, COA and SMO algorithms. The MCSMO has acquired 10%, 10%, 12% and 8% higher values for Accuracy, FI-score, Precision and Recall over other classifier model. Hence the MCSMO achieves the maximum accuracy and precision values for real time traffic video surveillance system. The future scope is to detect the number plate accurately in night vision and abnormal weather conditions.
REFRENCES
[1] B. S. K, V. S and V. V, "Perspective Vehicle License Plate Transformation using Deep Neural Network on Genesis of CPNet," in Elsevier, 2020.
[2] J. Yonten, R. Panomkhawn and W. Rattapoom, "Real time Bhutanese license plate localization using YOLO," Elsevier, 2020.
[3] D. K. Francisco and M. Rodrigo, "CON VOLUTION AL NEURAL NETWORKS FOR LICENSE PLATE DETECTION IN IMAGES," in IEEE, 2017.
[4] S. Ibtissam, Z. Abdelmoghit, A. O. Wahban, A. Issam and H. Abdellatif, "An automated license plate detection and recognition system based on wavelet decomposition and CNN," Elsevier, September 2020.
[5] L. Jungkyu, W. Taeryun, K. L. Tae, L. Hyemin, G. Geonmo and H. Kiho, "Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network," in IEEE, 2020.
[6] M. S. Sergio and R. J. Claudio, "Real-time license plate detection and recognition using deep convolutional neural networks," Elsevier, March 2020.
[7] S'ergio Montazzolli Silva and Cl'audio Rosito Jung, "License Plate Detection and Recognition in Unconstrained Scenarios" ECCV 2018.
[8] J. Priyadarshini and D. Sudha , "An intelligent multiple vehicle detection and tracking using modified vibe algorithm and deep learning algorithm, "Soft Computing, vol. 24, pp. 17417-17429, 2020.
[9] Junchao Tong, Dandan Ding and Lingyi Kong, "A deep learning approach for quality enhancement of surveillance video," Journal of Intelligent Transportation Systems, Vol. 24, pp. 304-314, no. 3, 2020.
[10] Songtao Ding, Shaohua Wan and Chen Chen, "Edge computing enabled video segmentation for real-time traffic monitoring in internet of vehicles," Pattern Recognition, Vol. 121, no. 108146, January 2022.
[11] Bhargava Rama Chilukuri and Rohan Dhatbale, "Deep Learning Techniques for Vehicle Trajectory Extraction in Mixed Traffic," Journal of Big Data Analytics in Transportation, vol. 3, pp. 141-157, 2021.
[12] O.S.Amosov, F.F.Pashchenko, S.G.Amosova, Y.S.Ivanov and S.V.Zhiganov,"Deep Neural Network Method of Recognizing the Critical Situations for Transport Systems by Video Images," Procedia Computer Science, Vol. 151, pp. 675-682, 2019.
[13] Miguel A.Molina-Cabello, Jorge García-González, Rafael M.Luque-Baena, Juan M.Ortiz-de-Lazcano-Lobato and Ezequiel López-Rubio," Road pollution estimation from vehicle tracking in surveillance videos by deep convolutional neural networks," Applied Soft Computing, Vol. 113, Part B, no. 107950, December 2021.
[14] Arefe Esalat Nejad and Morteza Romoozi,"Presenting a traffic management and control system in driver assistance form based on vehicular networks,"Automatic Control and Computer Sciences, vol. 51, pp. 1-12, 2017.
[15] Arefe Esalat Nejad and Morteza Romoozi,"Presenting a traffic management and control system in driver assistance form based on vehicular networks,"Automatic Control and Computer Sciences, vol. 51, pp. 1-12, 2017.
[16] ZhiYuan, WeiqingWang, HaiyunWang, and AbdullahYildizbasi, "Developed Coyote Optimization Algorithm and its application to optimal parameters estimation of PEMFC model", Energy Reports, vol.6, pp. 1106-1117, November 2020.
[17] Vani Agrawal, and Ratika Rastogi, and D. C. Tiwari, "Spider Monkey Optimization: a survey", International Journal of System Assurance Engineering and Management, vol.9, no.4, pp 929-941, August 2018.
[18] Ahmed G. Gad, "Particle Swarm Optimization Algorithm and Its Applications: A Systematic Review", Archives of Computational Methods in Engineering, vol. 29, pp. 2531-2561, 2022.
[19] Venkata Rao, R., "Jaya: A simple and new optimization algorithm for solving constrained and unconstrained optimization problems", International Journal of Industrial Engineering Computations, vol. 19-34, 2016.
[20] Ge, L., Dan, D., & Li, H., "An accurate and robust monitoring method of full-bridge traffic load distribution based on YOLO-v3 machine vision", Structural Control and Health Monitoring, vol. 12, 2020.
[21] Karel Lenc, A. Vedaldi, "R-CNN minus R", Computer Science, 2015.
[22] Ross Girshick, "Fast R-CNN", Microsoft Research, pp. 9, 2015.
[23] Cao, C., Wang, B., Zhang, W., Zeng, X., Yan, X., Feng, Z., ... Wu, Z., "An Improved Faster R-
[24] CNN for Small Object Detection", IEEE Access, pp. 1-1, 2019.
Mr. Manoj K. Bhosale has Completed B.E. in electronics and telecommunication and M.E. degrees in Electronics from Shivaji University, Kolhapur, India, in 1999 and 2011, respectively. Later, I am working with the Department of Electronics and telecommunication Engineering in Sinhgad institute of technology Lonavala affiliated by Savitribai P hule Pune University, Pune as an Assistant Professor since 2006. My research interests include image and signal processing, watermarking, information hiding and multimedia security. Has published 2 international conference paper and 2 journal papers.
Dr. Shubhangi B. Patil has Completed B.E. in electronics and telecommunication, M.E. degrees in Electronics and PhD. From Shivaji University, Kolhapur, India, in 1994, 2001, and 2011 respectively. Later, she is working with the Department of Electronics and telecommunication engineering in dr. J. J. Magdum college of Engineering Jaysingpur affiliated by Shivaji University, Kolhapur as an Professor since 2006. her research interests include image and signal processing, watermarking, information hiding and multimedia security. Has published 110 Papers in international conference and journal.
Dr Babasheb S. Patil, Principal In charge, PVPIT, Budhgaon. Has 33 years of experience, having more than 6 patents and two copy writes on his name. Has published more than 30 papers in conferences and journals.
Dr. Dnyaneshwar S Mantri graduated in Electronics Engineering from Walchand Institute of Technology, Solapur (MS) India in 1992 and received Master's from Shivaji University in 2006. He has been awarded Ph.D. in Wireless Communication at the Center for TeleInFrastruktur (CTIF), Aalborg University, Denmark.He has teaching experience of 25 years. From 1993 to 2006 he was working as a lecturer in different institutes [MCE Nilanga, MGM Nanded, and STB College of Engg. Tuljapur (MS) India]. Since 2006 he is associated with Sinhgad Institute of Technology, Lonavala, Pune and presently he is working as a Professor in
the Department of Electronics and Telecommunication Engineering. He is a recognized master's and Ph.D guide at Savitibai Phule Pune University, Pune in the subjects of Electronics and Telecommunication Engg, Computer and Information Technology. He is a Senior Member of IEEE, a Fellow of IETE, and Life Member of ISTE. He has published 06 books, 20 Journal papers in indexed and reputed Journals (Springer, Elsevier, and IEEE, etc.), and 19 papers in IEEE conferences. He is a reviewer of international journals (Wireless Personal Communication, Springer, Elsevier, IEEE, Communication Society, MDPI, etc.) and conferences
organized by IEEE. He worked as TPC member for various IEEE conferences and also organized IEEE conferences GCWCN2014 and GCWCN2018. He worked on various committees at University and College. He was a member of the Board of Studies in Electronics at Dr. Babasaheb Ambedkar Marathwada University, (Dr.BAMU) Aurangabad, He is a guest editor in STM Journals, Elected as Executive Council member (EC), and vice chair of IETE Pune Local Center (2022-24), His research interests are in Adhoc Networks, Wireless Sensor Networks, Wireless Communications specific focus on energy and bandwidth.
