CC BY
103
УНИВЕРСИТЕТ итмо
НАУЧНО-ТЕХНИЧЕСКИИ ВЕСТНИК ИНФОРМАЦИОННЫХ ТЕХНОЛОГИИ, МЕХАНИКИ И ОПТИКИ май-июнь 2022 Том 22 № 3 http://ntv.ifmo.ru/
SCIENTIFIC AND TECHNICAL JOURNAL OF INFORMATION TECHNOLOGIES, MECHANICS AND OPTICS May-June 2022 Vol. 22 No 3 http://ntv.ifmo.ru/en/
ISSN 2226-1494 (print) ISSN 2500-0373 (online)
ИНФОРМАЦИОННЫХ ТЕХНОЛОГИЙ. МЕХАНИКИ И йПТИКИ
doi: 10.17586/2226-1494-2022-22-3-459-471
Detection of potholes on road surfaces using photogrammetry and remote sensing methods (review)
Shahrul Nizan Abd Mukti1, Khairul Nizam Tahar2«
1 Dewan Bandaraya Kuala Lumpur, Kuala Lumpur, 50350, Malaysia
2 Universiti Teknologi MARA (UiTM), Shah Alam, 40450, Malaysia
1 shahrulnizan58@gmail.com, https://orcid.org/0000-0002-0498-0654
2 nizamtahar@gmail.com«, https://orcid.org/0000-0003-2065-7993
Abstract
An overview of methods for obtaining 2D and 3D models of defects on the pavement is given. The integrity of the pavement can be affected by factors such as temperature, humidity, weathering and loads. Potholes are one of the most common types of pavement failure. These defects are the signs of structural failures in an asphalt road. The process of collecting and analyzing data is critical to pavement maintenance. Finding and quantifying pothole geometry information is essential to understand road maintenance forecasts and to determine the right asphalt maintenance strategies. Visual detection of road defects is costly and time consuming. Today, there are quite a lot of studies in the scientific literature showing methods for automatic detection and recognition of potholes. In our work, we consider methods for automatic detection and classification of potholes using tools — sensors integrated with a positioning system. The technique of processing two-dimensional (2D) images using various methods of machine classification allows you to determine the precise geometry of the pothole. Algorithmic methods such as artificial neural networks, decision trees, support vector machines, and fuzzy classification are used to improve the accuracy of image processing and highlight the edges of potholes. A three-dimensional model of the pothole (3D) can be obtained based on laser scanning data and photogrammetry methods. The paper summarizes various methods and proposed techniques for extracting a 3D pothole model. The results of the work can be used to improve the infrastructure for maintaining road surfaces. Keywords
classification, image, processing, model, pavement defect, pothole Acknowledgements
The authors thank the Faculty of Architecture, Planning and Surveying of the Universiti Teknologi MARA (UiTM), the Research Management Center (RMC) and the Ministry of Higher Education (MOHE) for providing the GPK 600-RMC/ GPK 5/3 (223/2020) Foundation grant, as well as FRGS for grant FRGS/1/2021/WAB07/UITM/02/2. For citation: Abd Mukti S.N., Tahar K.N. Detection of potholes on road surfaces using photogrammetry and remote sensing methods (review). Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2022, vol. 22, no. 3, pp. 459-471. doi: 10.17586/2226-1494-2022-22-3-459-471
УДК 004.932
Обнаружения выбоин на дорожных покрытиях с использованием методов фотограмметрии и дистанционного зондирования
(обзор)
Шахрул Низан Абд Мукти1, Хайрул Низам Тахар2Н
1 Администрация Куала-Лумпура, Куала-Лумпур, 50350, Малайзия
2 Технологический университет MARA (UiTM), Шах-Алам, 40450, Малайзия
1 shahrulnizan58@gmail.com, https://orcid.org/0000-0002-0498-0654
2 nizamtahar@gmailxoms, https://orcid.org/0000-0003-2065-7993
Аннотация
Приведен обзор методов получения двухмерных (2D) и трехмерных (3D) моделей дефектов на дорожном покрытии. На целостность дорожного покрытия могут влиять такие факторы, как температура, влажность,
© Abd Mukti S.N., Tahar K.N., 2022
атмосферные воздействия и нагрузки. Один из самых распространенных видов разрушения дорожного покрытия выбоины, которые являются признаками структурных разрушений асфальтовой дороги. Процесс сбора и анализа данных имеет решающее значение при обслуживании дорожного покрытия. Обнаружение и количественная оценка информации о геометрии выбоин необходима для понимания прогноза работ по содержанию дорог и для определения правильных стратегий ухода за асфальтовым покрытием. Визуальное обнаружение дорожных дефектов дорогостоящее и трудоемкое. В настоящее время в научных работах представлены многочисленные исследования, показывающие способы автоматического обнаружения и распознавания выбоин. В настоящей работе рассмотрены методы автоматического обнаружения и классификации выбоин с использованием инструментальных средств — датчиков, интегрированных с системой позиционирования. Техника обработки 2D-изображений с использованием методов машинной классификации позволяет определить и уточнить геометрию выбоины. Для повышения точности обработки изображений и выделения краев выбоин применяются такие алгоритмические методы как искусственные нейронные сети, деревья решений, методы опорных векторов и нечеткой классификации. 3D-модель выбоины может быть получена на основе данных лазерного сканирования и методов фотограмметрии. В работе обобщены различные методы и предложенная техника для извлечения 3D-модели выбоины. Результаты работы могут найти применение для улучшения инфраструктуры обслуживания дорожных покрытий. Ключевые слова
классификация, изображение, обработка, модель, дефект дорожного покрытия, выбоина Благодарности
Авторы выражают благодарность Факультету архитектуры, планирования и геодезии Университета Технологии MARA (UiTM), Центру управления научными исследованиями (RMC) и Министерству высшего образования (MOHE) за предоставление грант Фонда ГПК 600-РМЦ/ГПК 5/3 (223 /2020), а также FRGS за предоставление гранта FRGS/1/2021/WAB07/UITM/02/2.
Ссылка для цитирования: Абд Мукти Ш.Н., Тахар Х.Н. Обнаружения выбоин на дорожных покрытиях с использованием методов фотограмметрии и дистанционного зондирования (обзор) // Научно-технический вестник информационных технологий, механики и оптики. 2022. Т. 22, № 3. С. 459-471 (на англ. яз.). doi: 10.17586/2226-1494-2022-22-3-459-471
Introduction
Deterioration of the road surface accumulates over time and traffic makes the situation worse. If maintenance does not perform by its time, the asphalt tear can create a pothole. The conventional method for road detection needs technicians for manual data collection and many working hours for a rough estimation of damage on the road. Making a repair decision and budgeting depending on the information of pothole damaged information is covered in [1]. Current practice implements now sophisticated digital inspection use (time-consuming and costly task). Secondly, the procedure is too dependent on the worker's precision during measure the pothole (subjective and biased thread of features).
There are many road assessment procedures, but not all are applied and used in practice by the department of authorities. The method of identifying defects in the main road varies depending on a person involved and background author (Fig. 1). But the same is the case in terms of product entry; however, different perspectives and detailed research are done based on the expertise of
the author. Defect detection is to distinguish the part of the defect from other parts in the image [2]. The process is to locate the region pixels of the defect and classify the type of defect.
Reference [3] presents 3 main types of road defect methods: vibration method, vision-based method and 3D reconstruction. According to a different author, reference [2] proposed image processing method, machine learning method and 3D imaging method. However, the features of detection are more concentrated on crack defect. Nevertheless, the conceptual procedure still remains the same. Image processing have the same concept with vision based method, while 3D imaging is a same method as explained in [3] using 3D reconstruction method. Table 1 shows a method of defect detection. Defects including all deteriorated road surfaces such as potholes crack, rutting and others are considered in [4].
Sensorial method
The sensorial-based method is a technique that uses an accelerometer on a vehicle platform based for data
CRACK POTHO INPAT APPAT OPJOI
Fig. 1. Sample surface defect types [2]
Table 1. Method of defect detection [4]
Item Vibration method Vision based method/2D method 3D reconstruction
Equipment Vibrator, gyroscope sensor, absorber sensor, mobile phone gyro RGB camera, multispectral camera, satellite imagery Stereo camera, RGB camera, laser scanner
Technique Z-direction displacement RGB image processing, multispectral classification Stereo vision, photogrammetry, Light Detection and Ranging (LiDAR)
Platform Car, van and mobile phone Drone, car, van, satellite Drone, car
collection. The advantages of this method are the small storage requirement, cost-effectiveness and feasibility for automatic real-time data processing. However, it does not provide detailed distress conditions of the road. The service condition of the vehicle also has a different impact on the result obtained from another vehicle; hence it needs constant vehicle parameters to make sure the data is comparable. Table 2 shows a previous study for sensorial method.
The same concept applies in [7] using a gyroscope sensor. The vibration data was obtained from vehicles running over four road types. Fig. 2 shows a proposed flowchart of the pothole detection. The collected data was used to extract the vehicle maximum rates of pitch, roll and yaw. When applying the machine learning ANN, the classification damage rates at 85 % of accuracy. Sensor equipment is a non-based technique that needs machine learning to classify road damage. In analogy, the image shows the surface of the pavement, but the vibration-based system feels the surface condition based on mechanical responses of the testing vehicle. This method needs high maintenance as the vehicle needs to purposely hit the pothole and other road defects. Sometimes, it picks up the joint bridge as a pothole road defect. Most often the acceleration is used as their input of data translation of road defects.
Due to the rapid economic growth and the advent of fast mobile technologies, there are applications available in a mobile phone [11]. The utilization of mobile for real-time pothole detection is possible now. The accelerometer data is normalized by computing the Euler angle adapted to
the pothole detection algorithm. Together they add spatial information to GPS data. Acquired data is sent to the Intelligent Transportation System (ITS). The availability of this system lies in the fact that the mobile phone itself has G-sensors, electronic compass, gyroscope, GPS, microphone and cameras. Basically, the detection system is crucial to acquire real-time information. The algorithm connects to the accelerometer, for example, the current G-zero analysis and in the context of various classes of road irregularities shows true positive indicators up to 90 %.
Under the same concept, this method also used in [8, 9]. The sensor method provides real-time pothole detection. It is a simple and economical method. This method even uses a sensor for detection, but the data processing to finalize the road defect is done by machine learning. The system consists of accelerometers or tilt switches to measure irregularities and also provides real-time data acquisition via two-way wireless communication and reuse of the same. It also has an interface for irregularity monitoring via a cloud-based service. The system consists of GPS and map index provided. The cloud service performs the conversion of the data received from all the devices on the ground simultaneously. The cloud service performs the conversion of the data received from all the devices on the ground simultaneously. This is how different authors use it to differentiate the method. Whether it is a machine learning method based on data segmentation or sensory method based on data input, it is somehow different from the author's point of view from his field of expertise.
The sensory method is a technique of intact measurement to record a defect area. The system functions
Table 2. Sensorial method's previous study
Reference Method How it works
[5] Mobile sensing with Global Positioning System (GPS) interpolation- z-threshold developed to detect a pothole The accelerometer data is normalized by Euler angle computation and is adopted in the pothole detection algorithm
[6] Smart phone accelerometer based on Android Operating System (OS) Provide z-threshold and g-zero which tuple below threshold
[7] Android smart phone with develop Artificial Neural Network (ANN) Collect optimum size of the training data, the effects of the number of the neurons in the hidden layer, and the level of the achievable classification accuracy
[8] Developed by microcontroller, accelerometer, GPS and zig-bee transmitter for two-way communication Accelerometers or tilts switches to measure the irregularity and also provides a real time data collection via a two-way wireless communication and reuse of the same
[9] 32-bit ARM Cortex M4 micro-controller and a LSM9DS0 Inertial Measurement Unit module (IMU system) Record the positioning system and attitudes of platform
[10] Mobile phone Finding a threshold impact on accuracy of true positive
Door slams, Acceleration, Minor Expansion joint, Smaller curbs braking turns anomalies rail crossing, highway
speed bumps anomalies
Fig. 2. Proposed flowchart of the pothole detection procedure [7]
by a sudden change of elevation and impact received by the sensor. Thus, a vehicle or user system has to hit the pothole defect purposely moving lane-by-lane for that purpose. However, non-hitting potholes do not cause any vibration to the sensorial-system failing the operating detection system [3, 12]. Decision making of the system is inseparable from machine learning applications, especially in data processing. This is a part of the processing to get results. The study distinguishes the method by its method of data input. This includes tools/hardware of sensor method, or image processing product, or 3D point cloud data of 3D reconstruction. All of the mentioned is related to machine learning, supervised, unsupervised, parametric, and non-parametric or any kind of algorithm for data classification purposes. Now, let's look into other methods which have been explained before.
2D Image processing
The development of cameras digital photography has greatly advanced road distress detection [13]. Distress detection is to distinguish the type of defect and segmented it from smooth asphalt of the road. Image segmentation is known as a process of assigning a label to every pixel in an image (Fig. 3). Pixel was classified with certain characteristics of a group such as a digital number of spectral reflectance, texture and geometry [14]. Vision based 2D detection originate from image processing and classification method, mostly in RGB format. Image processing mainly includes three categories: threshold segmentation, edge detection and region growing methods;
while classification method included pixel-wise, sub-pixel-wise, and object-based image classification methods [15].
The edge detection method determines the edges of the features through edge detection operators such as the Sobel operator, Prewitt operator, and Canny operator (Fig. 4). Reference [16] detects the edge of the defect area using an ant colony algorithm on thermal images. It is studied using a pulsed thermographic system to indicate a defect of SiC Coated High Temperature Super Alloy Substrate. The traditional edge detection method is based on the grey value of the image to determine the edge, but its drawback is various noises that effect to the quality of detection. And much more techniques have been introduced by the researchers, such as Wavelet transform, Wavelet packet edge detection, Edge detection mathematical morphology-based algorithm, Fuzzy theory, and neural network edge detection. Ant colony algorithm advantages are computational speed and less noise of detection, plus it gives complete detection of edge features.
Lastly, the image segmentation is done using a region growing method. This method depicts the information inside the pothole by combining the pixels with similar characteristics to form a group of features. The region is growing from the seed pixel by adding in neighboring pixels that are similar, increasing the size of the region. It's a method for serial segmentation of images. The advantage of this method is relatively simple and usually segmented with the same property region which is associated with the area. And it can provide good boundary information and segmentation. However, the region growing method is an iterative one. The space and time costs are relatively
Fig. 3. Method of detection from image-based processing
Fig. 4. The input infrared image: (a), the image after grey level transformation (b) and Canny preator edge detection (c) [16]
high. The noise and grey levels may cause gaps and over-segmentation. And the region growing method is not very good in image processing [17].
In contrast to the above technologies, reference [18] has addressed automatic object detection in sidewalk images using a unified framework. It then trains a Support Vector Machine (SVM) classifier for each feature separately in a one-against-all paradigm. Reference [19] proposed an automatic detection of potholes in images. The method has segmented asphalt into defective and non-defective regions using a histogram-based threshold. Based on the geometric properties of a defect region, the potential pothole shape is estimated using morphological thinning and elliptical regression. Then, the texture within high degree-defect geometry is extracted and compared to the non-defective roadway to determine if the region in question is indeed a pothole. The shape extraction procedure is shown in Fig. 5.
Reference [21] proposed crack detection by image pre-processing which consists of binary segmentation, morphological operations and remove algorithm. The idea is to remove the isolated dots and area. Then it is proposed a novel algorithm to join the break line of the crack. Another reference [22] presented a method of integrating the processing of grayscale and texture features for detection and segmentation of cement concrete pavement pothole. A machine learning model based on the Library of Support Vector Machines (LIBSVM) is built to distinguish potholes from longitudinal cracks, transverse cracks, and complex cracks (Fig. 6). Single based image method is the most applied for road distress detection. The input of single based image had been used in [1]. The reference [1] has used spectral clustering to identify regions using histogram-based data from greyscale image.
The use of single image-based for pothole detection is widely used. However, no quantification of geometry information such as width and length are available. The calibrated camera is required to obtain camera parameter, to compute its exterior parameter (object coordinate). Thus, for any intention of geometry information extraction, the calibration process before data collection is compulsory.
Image classification
Image classification of remote sensing-based methods has the potential for pothole detection. Supervised learning and unsupervised learning has been utilized for land use classification and analysis. Both are basic approaches,
but they are often combined to form hybrid methods. The most popular output from the classification process is the land use/land cover map. It's a main output of the remote sensing method. However, some use for other purposes such as urban hydrological map [23], urban classification map [24, 25], glacier facies mapping [26], crop map [27, 28], and oil palm [29]. The standard classification procedure of the multispectral and hyperspectral image starts with image correction and includes several steps such as geometric correction, radiometric correction, and hinge cleaning. Secondly, the image contrasts are restored; this includes contrast manipulation, spatial characteristic manipulation, spectrum ratio, main component analysis, additional component and vegetation index analysis. From a different perspective, classification is an overall process of image sensors, image pre-processing, object
Fig. 5. Shape extraction procedure based on a pothole shade:
morphological thinning (a), shrink the shade region (b), skeleton's branching point (c), connected major path of shade region, (e) boundary of the shape is almost smooth (d), entire skeleton defines the major path f) [20]
Fig. 6. Pothole extraction process: original image (a), texture image (b), grayscale (c), binary image (d), remove small object (e),
morphological f), extraction results (g), and mask image (h) [22]
detection, object segmentation, feature extraction and object classification [30].
Many classification techniques have been developed for image classification such as ANN, Decision Trees (DT), SVM and fuzzy classification. Image classification approaches are defined by different techniques, e.g., based on the features used (shape-based and motion-based), based on the training sample used (supervised and unsupervised), based on the assumption of parameters for the data (parametric classifier and nonparametric classifier), based on the pixel information used (classifier per pixel, sub-pixel classifier, per-field classifier, object-oriented classifier), based on several outputs for each spatial element (hard classification, soft classification), based on spatial information (spectral classifiers, contextual classifier, spectral-contextual classifier) and finally multiple classifier approach. The input for the classification process is a multispectral/hyperspectral image.
Reference [31] used Hyperspectral (HS) cameras to identify road defects. The HS cameras are considered more discriminative clues for crack detection. HS images used have a spectral range of 350 nm-2500 nm that contains spectra beyond the human vision range (400 nm-700 nm). A new spectral descriptor was proposing to describe the spectrum of road pavement. Reference [32] achieved modest success with Classification and Regression Tree (CART). By using CART, the abundance of cracked road is overestimated, while maximum overall accuracy is only at 68 %. Reference [33] is presenting pattern recognition techniques of pavement defect through Geographic Information System. The pavements degradation rate is representing by distinct spectral characteristics. It uses image indices with the spatial variance measures relating to the remote sensing signal of the pavement condition index. Detection of asphalt is carried out in
a two-classification process: a pixel supervised and unsupervised. The unsupervised classification uses linear operation classifier onto a monochromatic format image; supervised classification implements a maximum likelihood algorithm.
Reference [34] combined field spectrometry, in-situ road surveys and hyperspectral remote sensing to explore the potential in mapping road conditions. The proposed method used image ratios and spatial variance measures to relate the remote sensing signal to Peripheral Computer Interconnect (PCI). Image ratios showed the best correlation with the PCI, and variance is correlated with an index describing structural road damages. The authors mentioned the potential of other techniques using short-wave infrared and small absorption features identified in the spectral library analysis. Reference [35] evaluated asphalt pavement surface defects using airborne emissivity data. The authors used a surfacing limestone granules occurrence as a suitable indicator for surface defect assessment. The proposed hybrid of segmentation procedure and emissivity shape-based analysis is applying in this study. This technique allows you to produce rapid discrimination. Reference [36] extracted road asphalt condition using Object-Based Image Analysis (OBIA) and feature selection technique. The chi-square algorithm outperformed SVM and Random Forest (RF) techniques. SVM, RF and feature selection were evaluating to indicate the most effective algorithm. This test can identify which OBIA attributes are best set (spatial, spectral, textural, and color). An algorithm is a medium for classifying road distress that uses learning algorithms with different parameters. The authors used two sub-datasets RGB channels with 12 image bands to represent the Multispectral (MS) images. They concluded to incorporate spectral, geometrical, and textural features as attributes in RF classifier that can produce better delineation
Table 3. Review of spectral wavelength, platform and classifier algorithm used
Reference Band Platform Algorithm
[31] Hyperspectral (450 nm-950 nm) across 125 channels and fitted with a lens of 23 mm focal length Unmanned Aerial Vehicle (UAV) Not Available (NA)
[32] 227 spectral bands spanning the VNIR and SWIR ranges UAV CART algorithms
[33] RGB UAV Maximum likelihood
[34] 224 individual bands with a nominal bandwidth of 8-11 nm UAV NA
[35] Thermal infrared UAV Object-oriented approach and Band Depth analysis
[36] MS UAV SVM, RF, and chi-square
[14, 37] multispectral camera Micro-Miniature Multiple Camera Array System fixed-wing UAV KNN, SVM, ANN, RF
[38] 24 continuous spectral bands from 380.1 nm to 1,033.1 nm UAV NA
[39] MS IKONOS and QuickBird Multivariate kernel statistic
of the potholes and cracks. Table 3 shows a review of study of image classification.
Fig. 7 shows a fraction of 3D reconstruction method. The division presents laser scanning, photogrammetry applications and stereovision. Both are not the same in conceptual method but produce the same data (point cloud), so it can be used to derive 3D information such as point cloud elevation, mesh, and Digital Elevation Model (DEM). The laser scanner method receives data from another platform UAV or vehicles for terrestrial view, and uses a technique that employs active laser pulse to generate point cloud data [40, 41]. The data consists of elevation information captured on specific distress features by means of a grid-based processing approach. The irregular surface of the road is automatically revealed by a specific algorithm that detects different elevation and convex hull creation to find the parameter of the defect region. The result accurate quantified the amount of filling material.
3D imaging conceptual design was used for various detection and road monitoring. This section reviews focuses on the hardware side. The accuracy comparison is more appropriate since many laser-based systems use the same technical concept. Laser scanners were used in the study for road mapping. The system is considered as active scanner. It has a double-sided scanning mirror for the dual purpose of projecting and reflecting the light. The imaging system is mounted on a vehicle for road
scanning. Deployment of laser scanners has also resulted in several other applications related to pavements. In this regard, a measurement of pavement thickness was reported in [42]. Confidence arises as this method can improve the visualization of the pavement, and finer features have begun to attract researchers' attention. 10 attempts have been made to deploy a 3D laser scanner to measure pavement roughness using the system benchmarking International Roughness Index as their reference. The noted laser-based measurements have a high potential to quantify pavement roughness. In a related effort [43], an integrated multisensorial vehicle was used with laser range scanners for road imaging. The paper provides the system-level details of the mobile platform, a transverse resolution of the system achieved 1 cm. Reference [44] developed a laser triangulation-based mobile system with 2 mm horizontal and vertical resolution respectively. 3D geometric features were obtained and they classify the types of distress such as potholes and rutting. Sub-millimeters accuracy attracts the researchers, who basically use more than 95 % of the research on this technology [45]. However, a laser scanner needs a sturdy UAV platform because the micro-UAVs can't carry scanner due to workload limitations. Additionally, laser scanners capture big data and they require a workstation with the highest specification to handle.
Stereo vision requires high computational effort to construct a 3D pothole or other road defects. The matching
Laser scanning
UAV Terrestrial
3D method
Photogrammetry
UAV Close-range
Stereovision
Fig. 7. A fraction of 3D reconstruction method
feature point between two views requires time consumption so it is being used for post-processing. For vehicle platform cameras, it is important to align the camera properly otherwise it will affect the quality of the outcome. Reference [46] reports the developed techniques for 3D road surface reconstruction in support of unpaved road assessment. By using a non-metric camera from the UAV platform, it captures multiple images of unpaved roads from an altitude of about 45 m above ground, with an image scale of approximately 1:9000. The ground resolution is about 5 mm. The UAV is travelling at 4 m/s, acquiring road images with 80 % overlap along the path. The road surface model is constructed from a combination of the matching results of feature points, normal image points and grid points following a coarse-to-fine hierarchical strategy. Some errors still exist in the 3D models, due to shadows and mismatches caused by the low greyscale variation of road surface and lack of texture.
In [47], an acquisition of 3D information of potholes features is easily carried out. Disparity calculation method is described by graph cut, belief propagation and dynamic programming. The disparity calculation method is carried out in three steps: matching cost computation, search range recalculation and disparity enhancement. The first method of a stereo-vision based technique is found in [48] and reported by [45]. A stereo pair of cameras is coupled with a structured light projector to get image cracks and potholes. The same research method is applied. The experiments are conducted by mounting the measurement system on a car. During the annual study, the accuracy in centimeters level was obtained. Another stereo system achieved a depth accuracy of 5 mm [49].
To cover the whole pavement width of 4 m, two stereo sets are used. In [50] a texture depth measurement is established that texturizes the road but no absolute measurement was assessed. Another study of stereo vision was reported in [51]. Study concentrates on defect detection but does not mention the geometry of measured defect which was the real objective of two calibrated camera setup. Reference [52] introduced a low-cost detection system using the same technique with infrared laser line projector and digital camera. The purpose of the camera is to capture a multi-view coplanar scheme as a patch and register colored point cloud from the image to the laser scanner point cloud. Reference [53] combined a 2D method with 3D reconstruction. It deduced the number of potholes by the 2D recognition process and validated the geometry of the defect with 3D reconstruction. The method is used in different ways by others, who dependently use
point cloud data for recognition and geometry analysis. Thus, the low computational cost and acceleration data processing are guaranteed. Table 4 shows a technology solution classification of road distress detection. 3D laser scanning has highest spatial resolution with 1 mm accuracy.
Apart from the technologies of laser scanners and stereo vision cameras, a photogrammetric technique was reported in reference [2]. The photogrammetry method provides 3D reconstruction and 2D image correction. They are useful for detection purposes. When capturing multiple images, their orientation is applied for image matching to generate Orthophoto and DEM. Reference [55] covers up cracks analysis from an image. 3D model and 2D image format were used to perform analysis of potholes profile and orthophoto. Besides, 2 mm achievable accuracy from reconstruction method was presented in [49], it uses DHDV (Digital Highway Data Vehicle) developed by University of Arkansas and Way Link System Co. A 3D model of the road surface has been constructed from image resection. Tolerances of 1 mm accuracy were achieved in both longitudinal and transverse directions. The final pavement resolution images, which are captured from a speed above 100 km/h, yield good quality. It is a significant judge to have optimal distance from the camera to the road to compensate with such an optimal speed.
Mentioned above 3 methods of 3D reconstruction provide a point cloud-based data. Derived of the image-based point, the cloud is also able to characterize the texture of the road. After advancement of UAV technology, the aerial photogrammetry produces sensational road-mapping work. Nevertheless, terrestrial photogrammetry can be applied to the same objective. Many studies propose a photogrammetry method as a source of point cloud information of the interest features as well as mesh product analysis for the same purposes. In a revolutionary approach, an UAV mounted RGB camera has been used for the reconstruction of unpaved roads [46]. Mass production of micro-UAV provides inexpensive platform of data capture. The platform evolution for data capture from the land into aerial drastically changes the way of road defect detection. The photogrammetry technique starts a revolution in the mapping industry [56, 57] including the method of road mapping. Inexpensive and rugged UAV combination with high-resolution camera helps to get better output of high intensity of point cloud data. Two images captured from two different viewpoints are analyzed to derive the 3D information of the identified and matched feature points; hence the system is stereo in effect, although it has a monocular vision at any given point in
Table 4. Technology solution classification [54]
Technology Information Precision Accuracy Spatial Resolutio, mm Productivity Cost Automation
Camera 2D Medium Medium 3-6 High Low High
Linear scan camera 2D High High 2 Medium Medium Medium
3D laser imaging 2D High High 1 High High Medium
Terrestrial Laser Scanner (TLS) 2D Very high High 3-6 Medium High Low
Table 5. Previous review on reconstruction method
Reference 3D Method Proposed Techniques
[47] Stereovision camera Using stereo camera vision to create a disparity map, applied the surface fitting algorithm to estimate the road surface
[41] LiDAR Estimate road from point thinning then apply Multi-level Otsu thresholding to divide the intensity surface and apply convex hull for pothole polygonal creation
[46] Aerial Photogrammetry Image matching from combining the matching results of feature points, normal image points and grid points
[51] Laser imaging The distress measure in an image is calculated by accumulating the differences between adjacent histogram values
[44] Stereovision camera Stereovision camera for 3D Reconstruction
[45] UAV with LiDAR attached Features were extracted based on the strength of the point cloud elevations and reflection intensities
[59] Photogrammetry 2D image for detection and 3D reconstruction for measurement estimation
[40] LiDAR and camera fusion using fully convolutional neural networks An unstructured and sparse point cloud is first projected onto the camera image plane and then up sampled to obtain a set of dense 2D images encoding spatial information
[60] Aerial Photogrammetry Image stitching and DEM
[61] Aerial Photogrammetry Developed algorithm based from verticality value of point to automatically detect and measure road distress
time. The reconstruction process is helped by collinearity equation from input of the relative displacement of the UAV movement by two on board sensors: a GPS and an inertial measurement unit [46]. In another work, aerial mapping for various features has questioned the state-of-art in the parameter set for each desired mapping feature. Various range accuracy is reported in the work [58]. Table 5 shows a previous study of 3D reconstruction method and how it works.
Conclusion
Many proposed methods for pothole detection used image-based detection. The rough and coarser road surface
is the simplest indicator to detect and analyze the road surface condition. By that indicator, an algorithm of machine learning is evolving rapidly today. The symbiont of this technique contributes to the complex development of the algorithm. Day by day a detection system adds on by hardware, software, or classifier algorithm application for better detection. The concern of pothole detection arises since it is a dangerous obstacle on the road. Difference in pothole detection techniques results in different outputs. However, the same objective is to address the existence of the pothole to alert road users. Each method has advantages and a significant impact.
References
1. Buza E., Omanovic S., Huseinovic A. Pothole detection with image processing and spectral clustering. Proc. of the 2nd International Conference on Information Technology and Computer Networks (ITCN '13), 2013.
2. Cao W., Liu Q., He Z. Review of pavement defect detection methods. IEEE Access, 2020, vol. 8, pp. 14531-14544. https://doi.org/10.1109/ aCCESS.2020.2966881
3. Kim T., Ryu S.-K. Review and analysis of pothole detection methods. Journal of Emerging Trends in Computing and Information Sciences, 2014, vol. 5, no. 8, pp. 603-608.
4. Sarie F., Bisri M., Wicaksono A., Effendi R. Types of road pavement damage for road on peatland, a study case in Palangka Raya, Central Kalimantan, Indonesia. Journal of Environmental Science, Toxicology and Food Technology, 2015, vol. 9, no. 12, pp. 53-59. https://doi. org/10.9790/2402-091235359
5. Wang H.-W., Chen C.-H., Cheng D.-Y., Lin C.-H., Lo C.-C. A realtime pothole detection approach for intelligent transportation system. Mathematical Problems in Engineering, 2015, vol. 2015, pp. 869627. https://doi.org/10.1155/2015/869627
6. Mednis A., Strazdins G., Zviedris R., Kanonirs G., Selavo L. Real time pothole detection using Android smartphones with accelerometers. Proc. of the 7th International Conference on
Литература
1. Buza E., Omanovic S., Huseinovic A. Pothole detection with image processing and spectral clustering // Proc. of the 2nd International Conference on Information Technology and Computer Networks (ITCN '13). 2013.
2. Cao W., Liu Q., He Z. Review of pavement defect detection methods // IEEE Access. 2020. V. 8. P. 14531-14544. https://doi.org/10.1109/ aCCESS.2020.2966881
3. Kim T., Ryu S.-K. Review and analysis of pothole detection methods // Journal of Emerging Trends in Computing and Information Sciences. 2014. V. 5. N 8. P. 603-608.
4. Sarie F., Bisri M., Wicaksono A., Effendi R. Types of road pavement damage for road on peatland, a study case in Palangka Raya, Central Kalimantan, Indonesia // Journal of Environmental Science, Toxicology and Food Technology. 2015. V. 9. N 12. P. 53-59. https:// doi.org/10.9790/2402-091235359
5. Wang H.-W., Chen C.-H., Cheng D.-Y., Lin C.-H., Lo C.-C. A realtime pothole detection approach for intelligent transportation system // Mathematical Problems in Engineering. 2015. V. 2015. P. 869627. https://doi.org/10.1155/2015/869627
6. Mednis A., Strazdins G., Zviedris R., Kanonirs G., Selavo L. Real time pothole detection using Android smartphones with accelerometers // Proc. of the 7th International Conference on
Distributed Computing in Sensor Systems and Workshops (DCOSS), 2011, pp. 86376. https://doi.org/10.1109/DCOSS.2011.5982206
7. Gunawan F.E. Detecting road damages by using gyroscope sensor. ICIC Express Letters, 2018, vol. 12, no. 11, pp. 1089-1098. https:// doi.org/10.24507/icicel.12.11.1089
8. Jamakhandi H.A., Srinivasa K.G. Internet of Things based real time mapping of road irregularities. Proc. of International Conference on Circuits, Communication, Control and Computing (I4C), 2014, pp. 448-451 https://doi.org/10.1109/CIMCA.2014.7057842
9. Lei T., Mohamed A.A., Claudel C. An IMU-based traffic and road condition monitoring system. HardwareX, 2018, vol. 4, pp. e00045. https://doi.org/10.1016/j.ohx.2018.e00045
10. Strazdins G., Mednis A., Kanonirs G., Zviedris R., Selavo L. Towards vehicular sensor networks with android smartphones for road surface monitoring. The Second International Workshop on Networks of Cooperating Objects (CONET'11), Electronic Proceedings of CPSWeek'11, 2011, pp. 1-4.
11. Rau J.Y., Hsiao K.W., Jhan J.P., Wang S.H., Fang W.C., Wang J.L. Bridge crack detection using multi-rotary UAV and object-base image analysis. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives, 2017, vol. 42, no. 2W6, pp. 311-318. https://doi.org/10.5194/isprs-archives-XLII-2-W6-311-2017
12. Eriksson J., Girod L., Hull B., Newton R., Madden S., Balakrishnan H. The Pothole Patrol: Using a mobile sensor network for road surface monitoring. MobiSys'08 — Proc. of the 6th International Conference on Mobile Systems, Applications, and Services, 2008, pp. 29-39. https://doi.org/10.1145/1378600.1378605
13. Chambon S., Moliard J.M. Automatic road pavement assessment with image processing: Review and comparison. International Journal of Geophysics, 2011, vol. 2011, pp. 989354. https://doi. org/10.1155/2011/989354
14. Pan Y., Zhang X., Cervone G., Yang L. Detection of asphalt pavement potholes and cracks based on the unmanned aerial vehicle multispectral imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, vol. 11, no. 10, pp. 3701-3712. https://doi.org/10.1109/JSTARS.2018.2865528
15. Li M., Zang S., Zhang B., Li S., Wu C. A review of remote sensing image classification techniques: The role of Spatio-contextual information. European Journal of Remote Sensing, 2014, vol. 47, no. 1, pp. 389-411. https://doi.org/10.5721/EuJRS20144723
16. Qingju T., Jingmin D.A., Chunsheng L.I.U., Yuanlin L.I.U., Chunping R. Study on defects edge detection in infrared thermal image based on ant colony algorithm. International Journal of Signal Processing, Image Processing and Pattern Recognition, 2016, vol. 9, no. 4, pp. 121-130. http://dx.doi.org/10.14257/ijsip.2016.9.4.11
17. Luo Z.M., Wen J. The image segmentation algorithm of region growing and Wavelet Transform Modulus Maximum. Proc. of the 5th International Conference on Instrumentation and Measurement, Computer, Communication, and Control (IMCCC), 2015, pp. 11711174. https://doi.org/10.1109/IMCCC.2015.251
18. Sultani W., Mokhtari S., Yun H. Automatic pavement object detection using superpixel segmentation combined with conditional random field. IEEE Transactions on Intelligent Transportation Systems, 2018, vol. 19, no. 7, pp. 2076-2085. https://doi.org/10.1109/ TITS.2017.2728680
19. Koch C., Jog G.M., Brilakis I. Automated pothole distress assessment using asphalt pavement video data. Journal of Computing in Civil Engineering, 2013, vol. 27, no. 4, pp. 370-378. https://doi. org/10.1061/(ASCE)CP. 1943-5487.0000232
20. Koch C., Brilakis I. Pothole detection in asphalt pavement images. Advanced Engineering Informatics, 2011, vol. 25, no. 3, pp. 507-515. https://doi.org/10.1016/j.aei.2011.01.002
21. Huang W., Zhang N. A novel road crack detection and identification method using digital image processing techniques. Proc. of the 7th International Conference on Computing and Convergence Technology (ICCCT), 2012, pp. 397-400.
22. Gao M., Wang X., Zhu S., Guan P. Detection and segmentation of cement concrete pavement pothole based on image processing technology. Mathematical Problems in Engineering, 2020, pp. 1360832. https://doi.org/10.1155/2020/1360832
23. Paul A., Yang C., Breitkopf U., Liu Y., Wang Z., Rottensteiner F., Wallner M., Verworn A., Heipke C. Automatic classification of aerial imagery for urban hydrological applications. International Archives of the Photogrammetry, Remote Sensing and Spatial Information
Distributed Computing in Sensor Systems and Workshops (DCOSS). 2011. P. 86376. https://doi.org/10.1109/DCOSS.2011.5982206
7. Gunawan F.E. Detecting road damages by using gyroscope sensor // ICIC Express Letters. 2018. V. 12. N 11. P. 1089-1098. https://doi. org/10.24507/icicel.12.11.1089
8. Jamakhandi H.A., Srinivasa K.G. Internet of Things based real time mapping of road irregularities // Proc. of International Conference on Circuits, Communication, Control and Computing (I4C). 2014. P. 448-451 https://doi.org/10.1109/CIMCA.2014.7057842
9. Lei T., Mohamed A.A., Claudel C. An IMU-based traffic and road condition monitoring system // HardwareX. 2018. V. 4. P. e00045. https://doi.org/10.1016/j.ohx.2018.e00045
10. Strazdins G., Mednis A., Kanonirs G., Zviedris R., Selavo L. Towards vehicular sensor networks with android smartphones for road surface monitoring // The Second International Workshop on Networks of Cooperating Objects (CONET'11), Electronic Proceedings of CPSWeek'11. 2011. P. 1-4.
11. Rau J.Y., Hsiao K.W., Jhan J.P., Wang S.H., Fang W.C., Wang J.L. Bridge crack detection using multi-rotary UAV and object-base image analysis // International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives. 2017. V. 42. N 2W6. P. 311-318. https://doi.org/10.5194/isprs-archives-XLII-2-W6-311-2017
12. Eriksson J., Girod L., Hull B., Newton R., Madden S., Balakrishnan H. The Pothole Patrol: Using a mobile sensor network for road surface monitoring // MobiSys'08 - Proc. of the 6th International Conference on Mobile Systems, Applications, and Services. 2008. P. 29-39. https://doi.org/10.1145/1378600.1378605
13. Chambon S., Moliard J.M. Automatic road pavement assessment with image processing: Review and comparison // International Journal of Geophysics. 2011. V. 2011. P. 989354. https://doi. org/10.1155/2011/989354
14. Pan Y., Zhang X., Cervone G., Yang L. Detection of asphalt pavement potholes and cracks based on the unmanned aerial vehicle multispectral imagery // IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2018. V. 11. N 10. P. 37013712. https://doi.org/10.1109/JSTARS.2018.2865528
15. Li M., Zang S., Zhang B., Li S., Wu C. A review of remote sensing image classification techniques: The role of Spatio-contextual information // European Journal of Remote Sensing. 2014. V. 47. N 1. P. 389-411. https://doi.org/10.5721/EuJRS20144723
16. Qingju T., Jingmin D.A., Chunsheng L.I.U., Yuanlin L.I.U., Chunping R. Study on defects edge detection in infrared thermal image based on ant colony algorithm // International Journal of Signal Processing, Image Processing and Pattern Recognition. 2016. V. 9. N 4. P. 121-130. http://dx.doi.org/10.14257/ijsip.2016.9.4.11
17. Luo Z.M., Wen J. The image segmentation algorithm of region growing and Wavelet Transform Modulus Maximum // Proc. of the 5th International Conference on Instrumentation and Measurement, Computer, Communication, and Control (IMCCC). 2015. P. 11711174. https://doi.org/10.1109/IMCCC.2015.251
18. Sultani W., Mokhtari S., Yun H. Automatic pavement object detection using superpixel segmentation combined with conditional random field // IEEE Transactions on Intelligent Transportation Systems. 2018. V. 19. N 7. P. 2076-2085. https://doi.org/10.1109/ TITS.2017.2728680
19. Koch C., Jog G.M., Brilakis I. Automated pothole distress assessment using asphalt pavement video data // Journal of Computing in Civil Engineering. 2013. V. 27. N 4. P. 370-378. https://doi.org/10.1061/ (ASCE)CP. 1943-5487.0000232
20. Koch C., Brilakis I. Pothole detection in asphalt pavement images // Advanced Engineering Informatics. 2011. V. 25. N 3. P. 507-515. https://doi.org/10.1016/j.aei.2011.01.002
21. Huang W., Zhang N. A novel road crack detection and identification method using digital image processing techniques // Proc. of the 7th International Conference on Computing and Convergence Technology (ICCCT). 2012. P. 397-400.
22. Gao M., Wang X., Zhu S., Guan P. Detection and segmentation of cement concrete pavement pothole based on image processing technology // Mathematical Problems in Engineering. 2020. P. 1360832. https://doi.org/10.1155/2020/1360832
23. Paul A., Yang C., Breitkopf U., Liu Y., Wang Z., Rottensteiner F., Wallner M., Verworn A., Heipke C. Automatic classification of aerial imagery for urban hydrological applications // International Archives of the Photogrammetry, Remote Sensing and Spatial Information
Sciences — ISPRSArchives, 2018, vol. 42, no. 3, pp. 1355-1362. https://doi.org/10.5194/isprs-archives-XLII-3-1355-2018
24. Maas A., Alrajhi M., Alobeid A., Heipke C. Automatic classification of high resolution satellite imagery - A case study for urban areas in the Kingdom of Saudi Arabia. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives, 2017, vol. 42, no. 1W1, pp. 11-16. https://doi.org/10.5194/isprs-archives-XLII-1-W1-11-2017
25. Jabari S., Fathollahi F., Zhang Y. Application of sensor fusion to improve UAV image classification. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives, 2017, vol. 42, no. 2W6, pp. 153-156. https://doi.org/10.5194/isprs-archives-XLII-2-W6-153-2017
26. Jawak S.D., Wankhede S.F., Luis A.J. Comparison of pixel and object-based classification techniques for glacier facies extraction. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives, 2018, vol. 42, no. 5, pp. 543-548. https://doi.org/10.5194/isprs-archives-xlii-5-543-2018
27. Verma A.K., Garg P.K., Prasad K.S.H., Dadhwal V.K. Classification of LISS IV imagery using decision tree methods. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives, 2016, vol. 41, pp. 10611066. https://doi.org/10.5194/isprsarchives-XLI-B8-1061-2016
28. Saini R., Ghosh S.K. Crop classification on single date sentinel-2 imagery using random forest and suppor vector machine. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives, 2018, vol. 42, no. 5, pp. 683-688. https://doi.org/10.5194/isprs-archives-xlii-5-683-2018
29. Shaharum N.S.N., Shafri H.Z.M., Ghani W.A.W.A., Samsatli S., Yusuf B., Al-Habshi M.M.A., Prince H.M. Image classification for mapping oil palm distribution via support vector machine using scikit-learn module. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives, 2018, vol. 42, no. 4/W9, pp. 139-145. https://doi.org/10.5194/isprs-archives-XLII-4-W9-133-2018
30. Kamavisdar P., Saluja S., Agrawal S. A survey on image classification approaches and techniques. International Journal of Advanced Research in Computer and Communication Engineering, 2013, vol. 2, no. 1, pp. 1005-1009.
31. Abdellatif M., Peel H., Cohn A.G., Fuentes R. Hyperspectral imaging for autonomous inspection of road pavement defects. Proc. of the 36th International Symposium on Automation and Robotics in Construction (ISARC), 2019, pp. 384-392. https://doi.org/10.22260/isarc2019/0052
32. Jengo C.M., Hughes D., LaVeigne J.D., Curtis I. Pothole detection and road condition assessment using hyperspectral imager. Proc. of the Annual Conference "Geospatial Goes Global: From Your Neighborhood to the Whole Planet", 2005, pp. 461-473.
33. Branco L.H.C., Segantine P.C.L. MaNIAC-UAV — A methodology for automatic pavement defects detection using images obtained by Unmanned Aerial Vehicles. Journal of Physics: Conference Series, 2015, vol. 633, no. 1, pp. 012122. https://doi.org/10.1088/1742-6596/633/1/012122
34. Herold M., Roberts D., Smadi O., Noronh V. Road condition mapping with hyperspectral remote sensing. Available at: https://www.ugpti. org/smartse/research/citations/downloads/Herold-Road_Condition_ Mapping_with_HSI-2004.pdf (accessed: 30.12.2021)
35. Pascucci S., Bassani C., Palombo A., Poscolieri M., Cavalli R. Road asphalt pavements analyzed by airborne thermal remote sensing: Preliminary results of the venice highway. Sensors, 2008, vol. 8, no. 2, pp. 1278-1296. https://doi.org/10.3390/s8021278
36. Shahi K., Shafri H.Z.M., Taherzadeh E., Mansor S., Muniandy R. A novel spectral index to automatically extract road networks from WorldView-2 satellite imagery. Egyptian Journal of Remote Sensing and Space Science, 2015, vol. 18, no. 1, pp. 27-33. https://doi. org/10.1016/j.ejrs.2014.12.003
37. Pan Y., Zhang X., Sun M., Zhao Q. Object-based and supervised detection of potholes and cracks from the pavement images acquired by UAV. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives, 2017, vol. 42, no. 4W4, pp. 209-217. https://doi.org/10.5194/isprs-archives-XLII-4-W4-209-2017
38. Resende M.R., Bernucci L.L.B., Quintanilha J.A. Monitoring the condition of roads pavement surfaces: proposal of methodology using hyperspectral images. Journal of Transport Literature, 2014, vol. 8, no. 2, pp. 201-220. https://doi.org/10.1590/s2238-10312014000200009
Sciences — ISPRS Archives. 2018. V. 42. N 3. P. 1355-1362. https:// doi.org/10.5194/isprs-archives-XLII-3-1355-2018
24. Maas A., Alrajhi M., Alobeid A., Heipke C. Automatic classification of high resolution satellite imagery - A case study for urban areas in the Kingdom of Saudi Arabia // International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives. 2017. V. 42. N 1W1. P. 11-16. https:// doi.org/10.5194/isprs-archives-XLII-1-W1-11-2017
25. Jabari S., Fathollahi F., Zhang Y. Application of sensor fusion to improve UAV image classification // International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives. 2017. V. 42. N 2W6. P. 153-156. https:// doi.org/10.5194/isprs-archives-XLII-2-W6-153-2017
26. Jawak S.D., Wankhede S.F., Luis A.J. Comparison of pixel and object-based classification techniques for glacier facies extraction // International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives. 2018. V. 42. N 5. P. 543-548. https://doi.org/10.5194/isprs-archives-xlii-5-543-2018
27. Verma A.K., Garg P.K., Prasad K.S.H., Dadhwal V.K. Classification of LISS IV imagery using decision tree methods // International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives. 2016. V. 41. P. 1061-1066. https://doi.org/10.5194/isprsarchives-XLI-B8-1061-2016
28. Saini R., Ghosh S.K. Crop classification on single date sentinel-2 imagery using random forest and suppor vector machine // International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives. 2018. V. 42. N 5. P. 683-688. https://doi.org/10.5194/isprs-archives-xlii-5-683-2018
29. Shaharum N.S.N., Shafri H.Z.M., Ghani W.A.W.A., Samsatli S., Yusuf B., Al-Habshi M.M.A., Prince H.M. Image classification for mapping oil palm distribution via support vector machine using scikit-learn module // International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives. 2018. V. 42. N 4/W9. P. 139-145. https://doi.org/10.5194/isprs-archives-XLII-4-W9-133-2018
30. Kamavisdar P., Saluja S., Agrawal S. A survey on image classification approaches and techniques // International Journal of Advanced Research in Computer and Communication Engineering. 2013. V. 2. N 1. P. 1005-1009.
31. Abdellatif M., Peel H., Cohn A.G., Fuentes R. Hyperspectral imaging for autonomous inspection of road pavement defects // Proc. of the 36th International Symposium on Automation and Robotics in Construction (ISARC). 2019. P. 384-392. https://doi.org/10.22260/isarc2019/0052
32. Jengo C.M., Hughes D., LaVeigne J.D., Curtis I. Pothole detection and road condition assessment using hyperspectral imager // Proc. of the Annual Conference "Geospatial Goes Global: From Your Neighborhood to the Whole Planet". 2005. P. 461-473.
33. Branco L.H.C., Segantine P.C.L. MaNIAC-UAV — A methodology for automatic pavement defects detection using images obtained by Unmanned Aerial Vehicles // Journal of Physics: Conference Series. 2015. V. 633. N 1. P. 012122. https://doi.org/10.1088/1742-6596/633/1/012122
34. Herold M., Roberts D., Smadi O., Noronh V. Road condition mapping with hyperspectral remote sensing [Электроный ресурс]. URL: https://www.ugpti.org/smartse/research/citations/downloads/Herold-Road_Condition_Mapping_with_HSI-2004.pdf (дата обращения: 30.12.2021)
35. Pascucci S., Bassani C., Palombo A., Poscolieri M., Cavalli R. Road asphalt pavements analyzed by airborne thermal remote sensing: Preliminary results of the venice highway // Sensors. 2008. V. 8. N 2. P. 1278-1296. https://doi.org/10.3390/s8021278
36. Shahi K., Shafri H.Z.M., Taherzadeh E., Mansor S., Muniandy R. A novel spectral index to automatically extract road networks from WorldView-2 satellite imagery // Egyptian Journal of Remote Sensing and Space Science. 2015. V. 18. N 1. P. 27-33. https://doi. org/10.1016/j.ejrs.2014.12.003
37. Pan Y., Zhang X., Sun M., Zhao Q. Object-based and supervised detection of potholes and cracks from the pavement images acquired by UAV // International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives. 2017. V. 42. N 4W4. P. 209-217. https://doi.org/10.5194/isprs-archives-XLII-4-W4-209-2017
38. Resende M.R., Bernucci L.L.B., Quintanilha J.A. Monitoring the condition of roads pavement surfaces: proposal of methodology using hyperspectral images // Journal of Transport Literature. 2014. V. 8. N 2. P. 201-220. https://doi.org/10.1590/s2238-10312014000200009
39. Raziq A., Xu A., Li Y., Zhao X. Extraction of urban road network from multispectral images using multivariate kernel statistic and segmentation method. Geoinformatics & Geostatistics: An Overview, 2017, vol. 5, no. 1. https://doi.org/10.4172/2327-4581.1000158
40. Caltagirone L., Bellone M., Svensson L., Wahde M. LiDAR-camera fusion for road detection using fully convolutional neural networks. Robotics and Autonomous Systems, 2019, vol. 111, pp. 125-131. https://doi.org/10.1016/j.robot.2018.11.002
41. Kumar P., Angelats E. An automated road roughness detection from mobile laser scanning data. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives, 2017, vol. 42, no. 1W1, pp. 91-96. https://doi.org/10.5194/isprs-archives-XLII-1-W1-91-2017
42. Walters R.C., Jaselskis E. Using scanning lasers for real-time pavement thickness measurement. Proc. of the 2005 ASCE International Conference on Computing in Civil Engineering, 2005, pp. 375-385. https://doi.org/10.1061/40794(179)36
43. Yu S.J., Sukumar S.R., Koschan A.F., Page D.L., Abidi M.A. 3D reconstruction of road surfaces using an integrated multi-sensory approach. Optics and Lasers in Engineering, 2007, vol. 45, no. 7, pp. 808-818. https://doi.org/10.1016/j.optlaseng.2006.12.007
44. Li Q., Yao M., Yao X., Yu W., Xu B. A real-time 3D scanning system for pavement rutting and pothole detections. Proceedings of SPIE, 2009, vol. 7447, pp. 74470B. https://doi.org/10.1117/12.824559
45. Mathavan S., Kamal K., Rahman M. A review of three-dimensional imaging technologies for pavement distress detection and measurements. IEEE Transactions on Intelligent Transportation Systems, 2015, vol. 16, no. 5, pp. 2353-2362. https://doi.org/10.1109/ TITS.2015.2428655
46. Zhang C., Elaksher A. 3D Reconstruction from UAV-acquired Imagery for Road Surface Distress Assessment. Proc. of the 31st Asian Conference on Remote Sensing (ACRS), 2010, pp. 386-391.
47. Zhang A., Wang K.C.P., Li B., Yang E., Dai X., Peng Y., Fei Y., Liu Y., Li J.Q., Chen C. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Computer-Aided Civil and Infrastructure Engineering, 2017, vol. 32, no. 10, pp. 805-819. https://doi.org/10.1111/mice.12297
48. Kertesz I., Lovas T., Barsi A. Measurement of road roughness by low-cost photogrammetric system. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives, 2007, vol. 36, no. 5/C55, pp. 4.
49. Hou Z., Wang K.C.P., Gong W. Experimentation of 3D pavement imaging through stereovision. Proc. of the International Conference on Transportation Engineering (ICTE), 2007, pp. 376-381. https:// doi.org/10.1061/40932(246)62
50. El Gendy A., Shalaby A., Saleh M., Flintsch G.W. Stereo-vision applications to reconstruct the 3D texture of pavement surface. International Journal of Pavement Engineering, 2011, vol. 12, no. 3, pp. 263-273. https://doi.org/10.1080/10298436.2010.546858
51. Yu X., Salari E. Pavement pothole detection and severity measurement using laser imaging. Proc. of the IEEE International Conference on Electro Information Technology, 2011, pp. 5978573. https://doi. org/10.1109/EIT.2011.5978573
52. Jokela M., Kutila M., Le L. Road condition monitoring system based on a stereo camera. Proc. of the IEEE 5th International Conference on Intelligent Computer Communication and Processing (ICCP), 2009, pp. 423-428. https://doi.org/10.1109/ICCP.2009.5284724
53. Tan Y., Li Y. UAV photogrammetry-based 3D road distress detection. ISPRS International Journal of Geo-Information, 2019, vol. 8, no. 9, pp. 409. https://doi.org/10.3390/ijgi8090409
54. Ragnoli A., de Blasiis M.R., Di Benedetto A. Pavement distress detection methods: A review. Infrastructures, 2018, vol. 3, no. 4, pp. 58. https://doi.org/10.3390/infrastructures3040058
55. Wang X., Al-Shabbani Z., Sturgill R., Kirk A., Dadi G.B. Estimating earthwork volumes through use of unmanned aerial systems. Transportation Research Record, 2017, vol. 2630, pp. 1-8. https:// doi.org/10.3141/2630-01
56. Tahar K.N., Ahmad A., Akib W.A.A.W.M., Mohd W.M.N.W. A generic approach for photogrammetric survey using a six-rotor unmanned aerial vehicle. IOP Conference Series: Earth and Environmental Science, 2014, vol. 18, no. 1, pp. 012003 https://doi. org/10.1088/1755-1315/18/1/012003
57. Abd Mukti S.N., Tahar K.N. Low altitude photogrammetry for urban road mapping. Built Environment Journal, 2021, vol. 18, no. 1, pp. 31. https://doi.org/10.24191/bej.v18i1.10205
39. Raziq A., Xu A., Li Y., Zhao X. Extraction of urban road network from multispectral images using multivariate kernel statistic and segmentation method // Geoinformatics & Geostatistics: An Overview. 2017. V. 5. N 1. https://doi.org/10.4172/2327-4581.1000158
40. Caltagirone L., Bellone M., Svensson L., Wahde M. LiDAR-camera fusion for road detection using fully convolutional neural networks // Robotics and Autonomous Systems. 2019. V. 111. P. 125-131. https:// doi.org/10.1016/j.robot.2018.11.002
41. Kumar P., Angelats E. An automated road roughness detection from mobile laser scanning data // International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives. 2017. V. 42. N 1W1. P. 91-96. https:// doi.org/10.5194/isprs-archives-XLII-1-W1-91-2017
42. Walters R.C., Jaselskis E. Using scanning lasers for real-time pavement thickness measurement // Proc. of the 2005 ASCE International Conference on Computing in Civil Engineering. 2005. P. 375-385. https://doi.org/10.1061/40794(179)36
43. Yu S.J., Sukumar S.R., Koschan A.F., Page D.L., Abidi M.A. 3D reconstruction of road surfaces using an integrated multi-sensory approach // Optics and Lasers in Engineering. 2007. V. 45. N 7. P. 808-818. https://doi.org/10.1016/j.optlaseng.2006.12.007
44. Li Q., Yao M., Yao X., Yu W., Xu B. A real-time 3D scanning system for pavement rutting and pothole detections // Proceedings of SPIE. 2009. V. 7447. P. 74470B. https://doi.org/10.1117/12.824559
45. Mathavan S., Kamal K., Rahman M. A review of three-dimensional imaging technologies for pavement distress detection and measurements // IEEE Transactions on Intelligent Transportation Systems. 2015. V 16. N 5. P. 2353-2362. https://doi.org/10.1109/ TITS.2015.2428655
46. Zhang C., Elaksher A. 3D Reconstruction from UAV-acquired Imagery for Road Surface Distress Assessment // Proc. of the 31st Asian Conference on Remote Sensing (ACRS). 2010. P. 386-391.
47. Zhang A., Wang K.C.P., Li B., Yang E., Dai X., Peng Y., Fei Y., Liu Y., Li J.Q., Chen C. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network // Computer-Aided Civil and Infrastructure Engineering. 2017. V. 32. N 10. P. 805-819. https://doi.org/10.1111/mice.12297
48. Kertesz I., Lovas T., Barsi A. Measurement of road roughness by low-cost photogrammetric system // International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences — ISPRS Archives. 2007. V. 36. N 5/C55. P. 4.
49. Hou Z., Wang K.C.P., Gong W. Experimentation of 3D pavement imaging through stereovision // Proc. of the International Conference on Transportation Engineering (ICTE). 2007. P. 376-381. https://doi. org/10.1061/40932(246)62
50. El Gendy A., Shalaby A., Saleh M., Flintsch G.W. Stereo-vision applications to reconstruct the 3D texture of pavement surface // International Journal of Pavement Engineering. 2011. V. 12. N 3. P. 263-273. https://doi.org/10.1080/10298436.2010.546858
51. Yu X., Salari E. Pavement pothole detection and severity measurement using laser imaging // Proc. of the IEEE International Conference on Electro Information Technology. 2011. P. 5978573. https://doi. org/10.1109/EIT.2011.5978573
52. Jokela M., Kutila M., Le L. Road condition monitoring system based on a stereo camera // Proc. of the IEEE 5th International Conference on Intelligent Computer Communication and Processing (ICCP). 2009. P. 423-428. https://doi.org/10.1109/ICCP.2009.5284724
53. Tan Y., Li Y. UAV photogrammetry-based 3D road distress detection // ISPRS International Journal of Geo-Information. 2019. V. 8. N 9. P. 409. https://doi.org/10.3390/ijgi8090409
54. Ragnoli A., de Blasiis M.R., Di Benedetto A. Pavement distress detection methods: A review // Infrastructures. 2018. V. 3. N 4. P. 58. https://doi.org/10.3390/infrastructures3040058
55. Wang X., Al-Shabbani Z., Sturgill R., Kirk A., Dadi G.B. Estimating earthwork volumes through use of unmanned aerial systems // Transportation Research Record. 2017. V. 2630. P. 1-8. https://doi. org/10.3141/2630-01
56. Tahar K.N., Ahmad A., Akib W.A.A.W.M., Mohd W.M.N.W. A generic approach for photogrammetric survey using a six-rotor unmanned aerial vehicle // IOP Conference Series: Earth and Environmental Science. 2014. V. 18. N 1. P. 012003 https://doi. org/10.1088/1755-1315/18/1/012003
57. Abd Mukti S.N., Tahar K.N. Low altitude photogrammetry for urban road mapping // Built Environment Journal. 2021. V. 18. N 1. P. 31. https://doi.org/10.24191/bej.v18i1.10205
58. Slimane A.B., Khoudeir M., Brochard J., Do M.T. Characterization of road microtexture by means of image analysis. Wear, 2008, vol. 264, no. 5-6, pp. 464-468. https://doi.org/10.1016/j. wear.2006.08.045
59. Jog G.M., Koch C., Golparvar-Fard M., Brilakis I. Pothole properties measurement through visual 2D recognition and 3D reconstruction. Proc. of the ASCE International Conference on Computing in Civil Engineering, 2012, pp. 553-560. https://doi. org/10.1061/9780784412343.0070
60. Saad A.M., Tahar K.N. Identification of rut and pothole by using multirotor unmanned aerial vehicle (UAV). Measurement: Journal of the International Measurement Confederation, 2019, vol. 137, pp. 647-654. https://doi.org/10.1016Zj.measurement.2019.01.093
61. Biçici S., Zeybek M. An approach for the automated extraction of road surface distress from a UAV-derived point cloud. Automation in Construction, 2021, vol. 122, pp. 103475. https://doi.org/10.1016/j. autcon.2020.103475
Authors
Shahrul N. Abd Mukti — PhD, Senior Surveyor, Dewan Bandaraya Kuala Lumpur, Kuala Lumpur, 50350, Malaysia, https://orcid.org/0000-0002-0498-0654, shahrulnizan58@gmail.com
Khairul N. Tahar — PhD, Associate Professor, Universiti Teknologi MARA (UiTM), Shah Alam, 40450, Malaysia, S3 38362352000, https:// orcid.org/0000-0003-2065-7993, nizamtahar@gmail.com
Received 10.03.2022
Approved after reviewing 14.04.2022
Accepted 30.05.2022
58. Slimane A.B., Khoudeir M., Brochard J., Do M.T. Characterization of road microtexture by means of image analysis // Wear. 2008. V. 264. N 5-6. P. 464-468. https://doi.org/10.1016Zj.wear.2006.08.045
59. Jog G.M., Koch C., Golparvar-Fard M., Brilakis I. Pothole properties measurement through visual 2D recognition and 3D reconstruction // Proc. of the ASCE International Conference on Computing in Civil Engineering. 2012. P. 5 5 3-560. https://doi. org/10.1061/9780784412343.0070
60. Saad A.M., Tahar K.N. Identification of rut and pothole by using multirotor unmanned aerial vehicle (UAV) // Measurement: Journal of the International Measurement Confederation. 2019. V. 137. P. 647-654. https://doi.org/10.1016/j.measurement.2019.01.093
61. Bijici S., Zeybek M. An approach for the automated extraction of road surface distress from a UAV-derived point cloud // Automation in Construction. 2021. V. 122. P. 103475. https://doi.org/10.1016/j. autcon.2020.103475
Авторы
Абд Мукти Шахрул Низан — PhD, старший геодезист, Администрация Куала-Лумпура, Куала-Лумпур, 50350, Малайзия, https://orcid.org/0000-0002-0498-0654, shahrulnizan58@gmail.com Тахар Хайрул Низам — PhD, доцент, Технологический университет MARA (UiTM), Шах-Алам, 40450, Малайзия, S3 38362352000, https:// orcid.org/0000-0003-2065-7993, nizamtahar@gmail.com
Статья поступила в редакцию 10.03.2022 Одобрена после рецензирования 14.04.2022 Принята к печати 30.05.2022
Работа доступна по лицензии Creative Commons «Attribution-NonCommercial»
