CC BY
14
Experimental assessment of the distance measurement accuracy using the active-pulse television measuring system and a digital terrain model
V.V. Kapustin1, A.S. Zahlebin1, A.K. Movchan1, M.I. Kuryachiy1, M.V. Krutikov1 1 Tomsk State University of Control Systems and Radioelectronics, 634050, Russia, Tomsk, Lenina ave, 40
Abstract
This paper considers an experimental study of the layout of an active-pulse television measuring system in the problem of assessing the accuracy of measuring the distance to objects using the depth maps. The main technical characteristics and structure of the active-pulse television measuring system layout are described, the description of the multi-zone ranging method used in the experiment is given. The field tests were carried out using a system for terrain orthophoto-maps construction by an unmanned aerial vehicle and a geodetic measuring instrument, which is a reference for building a terrain plan and fixing distances between objects on the ground. The technique of carrying out aerial work is described to obtain the necessary data array, on which a digital model and an orthophotomap of the area were subsequently built. Conclusions are drawn about the accuracy of digital terrain models built based on the results of aerial photography from an unmanned aerial vehicle with a geodetic receiver on board and the applicability of these data as reference data for testing a prototype of an active-pulse television measuring system.
Keywords: depth maps, range measurement, terrain orthophotomap, digital terrain model, active-pulse television measuring system.
Citation: Kapustin VV, Zahlebin AS, Movchan AK, Kuryachiy MI, Krutikov MV. Experimental assessment of the distance measurement accuracy using the active-pulse television measuring system and a digital terrain model. Computer Optics 2022; 46(6): 948-954. DOI: 10.18287/2412-6179-CO-1114.
Introduction
Experimental assessment of the distance measurement accuracy using the active-pulse television measuring system and a digital terrain model Currently, methods and tools for measuring the range to objects were widely used in various fields of human activity. Sonars, lidars, radars, and stereo cameras are used in autonomous unmanned vehicles and driver assistance systems to analyze the environment in order to prevent emergencies. On unmanned aerial vehicles (UAVs), such systems are used to measure distances to various obstacles during flight, which avoids collisions when flying at low altitudes.
The class of devices using the active method of determining the distance to observation objects includes active-pulse television measuring systems (AP TMS) [1 -4]. Basically, such systems are used to detect and recognize objects in difficult meteorological visibility conditions (with fog, smoke, dust, snowfall) and when observing objects underwater, due to the possibility of suppressing backscattering interference [5, 6]. The principle of AP TMS operation consists of illumination of observed space by optical pulses of a certain duration, as well as time gating of reflected optical pulses by the photodetector. To observe an object located at a certain distance from the AP TMS, it must be illuminated by optical space illumination pulses (SIP), and the amount of delay between the SIP's radiation and the moment of opening of the photode-tector shutter should be the time required for radiation to propagate to the object and back. In this case, the operator of AP TMS will see the object of interest and some part of
the space surrounding it. To describe this observed portion of space, the authors use the term active vision area (AVA). The depth of the AVA will be determined by both the exposure time of the photodetector and the duration of the optical SIP. AVA shape will generally be the convolution result of the optical SIP shape with the gating pulse shape of the photodetector [7 - 9]. A general principle of the active-pulse observation method is given in fig. 1 [10].
Gated Image Sensor
—.r
0 ro
Fig. 1. Active-pulse observation method
Fig. 1 shows that the photodetector will receive only photons reflected from objects located in the certain range of distances, which corresponds to the depth of the AVA. The photodetector gating delay (t = 2ro / co) defined by the distance r0, where the AVA will be formed, c0 -propagation speed of the optical radiation for the medium in which the observation is carried out.
Since the AP TMS works according to the principle of the optical locator, this feature can be used to determine the range to objects found in the field of view of the system, as well as to construct depth maps of the observed
space, where a pixel of a two-dimensional picture corresponds to the range to the object.
At the moment, there are several main methods and approaches for building a depth map using AP TMS.
1. The scene scanning by the depth during the change of the time delay between SIP and photo camera strobe [1, 11].
2. Range-intensity correlation method. The principle of distance measurement is based on the use of two AVA shifted by the gating delay for the duration of the SIP [12, 13].
3. Photodetector gain modulation method. Since the introduction of this method, two types of gain modulation have been developed, including linear modulation and exponential modulation [14, 15].
4. Training of neural networks to build a depth map on the obtained AVA [16].
To assess the accuracy of the range measurement with an exponential model of AP TMS, field tests were carried out at the training ground. The layout of AP TMS was developed at the Department of Television and Control of Tomsk State University of Control Systems and Radio Electronics. Tests on polygon were carried out using UAV and geodesic equipment.
The appearance of the AP TMS layout used during the tests is shown in fig. 2.
Fig. 2. AP TMS layout
Basic technical characteristics of the AP TMS layout: system vision range up to 100 m; angle of the system field of view -12°; optical illumination power in pulse - 320 W; wavelength of the illuminated radiation - 842 nm; frequency of illumination pulses repetition 5 kHz.
The AP TMS layout consists of the following main components: input lens, image intensifier tube (IIT), matching lens, TV sensor, illumination device, power supplies, pulse generators, control and image processing units (Fig. 3.).
The illumination device is a pulse laser semiconductor emitter (PLSE). The TV sensor is synchronized through the control unit with the IIT. The frame frequency of the TV sensor is 50 Hz, the frequency of IIT operation is up to 5 kHz. Thus, up to 100 frames generated on the IIT screen can be integrated into 1 frame in a TV sensor.
The purpose of the studies was field trials of the AP TMS layout at the test area. The following tasks were
solved: to evaluate the accuracy of measuring the AP TMS range to observation objects using the multizone range measurement method (MZRM), to compare the results with geodetic measurement methods; create a digital model of the test area based on the results of aerial photography; consider using a digital terrain model as a reference for future AP TMS tests.
Input lens
Matching lens
Image intensifier tube (IIT)
Image processing unit
Pulse laser semiconductor emitter (PI.SP)
Pump current pulse generator
Fig. 3. AP TMS layout Structure Diagram Experiment plan
1. Placement of the reference points on the test area.
2. Determination of the exact geographical coordinates of reference points with a geodetic device.
3. Drawing up a flight task for the UAV.
4. Aviation work.
5. Processing of the aerial photography results and building of the digital terrain model.
6. Installation of the AP TMS layout and banners on reference points
7. Distance measurement from the AP TMS to banners.
8. Evaluation of the distance measuring accuracy of the AP TMS relative to the digital terrain model.
Building a digital terrain model
The testing area was an uneven piece of terrain on which identification marks (reference points) were located. The location of the reference points was chosen so that after the aviation work in their place, it was possible to arrange numbered banners, the distance to which will be measured by the AP TMS layout. The geographical coordinates of these points were precisely determined using the Trimble R8 satellite geodetic receiver in RealTime Kinematic (RTK) mode. The geodetic base station was located at known coordinates and transmitted corrections to the receiver. Reference points are used to check the accuracy of orthomosaic geographical reference and digital model of terrain, which were obtained as a result of photogrammetric processing of aerial photographs [17, 18]. Also, with their help, the accuracy of distance measurement done by the AP TMS layout was checked.
As a UAV a DJI Phantom 4 Pro quadrocopter with an integrated two-frequency geodetic receiver on board was used. The geodetic receiver allows navigation in space
and recording of satellite observations data during flight (kinematics) [19, 20]. The UAV is equipped with a television camera with a spatial resolution of 20 megapixels, which is installed on a gyro-stabilizing three-axis gimbal. This makes it possible to achieve centimeter accuracy for coordinates determination of photographing centers during post-processing of satellite observation data from UAV board and base station installed at the point with known coordinates [21].
A flight task for the UAV was prepared. Main parameters of flight mission were: flight speed - 6 m / s; flight altitude - 80 m; longitudinal overlap - of 80 % aerial photograph; cross-section of aerial photograph - 60 %. On the quadcopter which was used, the exposure value of the photodetector was in the range from 1 / 500 s to 1 / 1600 s. If the exposure value were less than 1 / 500 s, images became blurred. If the exposure value were more than 1 / 1600 s, the mechanical shutter of the camera stopped working, which led to a sharp increase in the error of determining geospatial data of the constructed terrain model [22].
As a result of the flight mission, a video sequence of 706 aerial photographs was formed.
Photogrammetric processing of aerial photographs was carried out under laboratory conditions according to the procedure described in [23]. Agisoft Metashape was used as the main software to build a digital terrain model. Satellite observation data were processed, and photography centers were calculated for each image, which are elements of the external orientation of the image (coordinates of the projection center in the object coordinate system, angles of rotation of the image coordinate system relative to the object system) [24 - 26]. According to these data, using the collinearity equation, the coordinates of points of objects on the ground are calculated. As a result of the aerial photographs alignment and photographing centers optimization the following differences of control points coordinates determination on the ground using aerial photographs and results of full-scale measurements using a geodetic receiver in RTK mode (tab. 2) are obtained. The result of full-scale measurements in this work is considered a reference.
Tab. 1. MSE for reference points
Name of the reference Error Error Error
point X, cm Y, cm Z, cm
R1 - 4 6 - 2.5
R2 - 5.1 4.3 - 2
R3 - 2.8 5.1 - 4
R4 - 2.9 4.9 1.4
R5 - 2 6.7 - 1
Table 1 show that the difference in determining the coordinates of objects between the digital model and natural measurements using a geodetic instrument is not more than 7 cm. This fact indicates that the final digital terrain model will have a high accuracy of geospatial data
and correspond to a scale of 1:500. This technique can be used to create digital terrain models for testing AP TMS.
A situation was modeled when aviation operations were carried out on UAVs without a geodetic receiver on board. In this case, the processing was carried out without the exact values of the elements of the internal and external orientation of the aerial photographs, and their calculation took place from control points that were used as reference points. Tab. 2 shows an influence assessment of the points number for plan-altitude justification (control points) on the spatial error of the model during the processing of materials without accurate values of internal and external values of aerial photographs.
Tab. 2. Dependence of MSE in control points on their quantity
Number of MSE on MSE on MSE on
the refer- reference reference reference
ence points points X, cm points Y, cm points Z, cm
3 1 1 1.5
2 57.7 246 117.7
1 424.6 463.5 595.4
0 558.4 513.9 928.3
From tab. 3, it follows that SD at control points increases dramatically with a decrease in the number of SD involved in processing. A minimum number of reference points shall be at least three per small area (up to 10 Ha). Thus, the use of a UAV without a geodetic receiver onboard entails additional labor on the need to use reference points. This is especially true in large areas and in the territory with complex relief.
At the next stage of data processing, a dense cloud of points was built according to the method described in [23]. A dense cloud of points with known coordinates (in the coordinate system of the object) and brightness values took on the original image. A fragment of the constructed dense point cloud is represented in fig. 4.
Fig. 4. Dense point cloud
A dense cloud of points is a digital model of terrain on which it is possible to measure distances between objects with centimeter accuracy. Accordingly, such a model can be used to test the AP TMS. As an additional source of geospatial data, an orthomosaic of the area was built and then depicted in fig. 5.
Fig. 5. Local orthomosaic fragment
Then, the distances between the reference points at which the banners were located was measured (fig. 6).
Fig. 6. Measuring distances using dense point cloud
Testing of the AP TMS layout
After aviation work, the AP TMS layout was installed on one of the identification marks. The layout was installed on the polygon in such a way as to cover all banners located on the ground without the need to move the system, but only rotating the layout around its axis on the rack to maintain a geographical reference to the place of its installation. Banners were installed at the location of the identification marks. The location of the banners in the test area is shown in fig. 7.
Distance measurement with APTMS was carried out using MZRM developed by the authors [23]. MZRM is implemented in the AP TMS layout as follows. The system forms two consecutive video frames. The first video frame contains a certain number of extended (per the desired range of measurement) AVAs with some constant initial gating delay, wherein AVAs are formed due to a
long duration photodetector gating pulse (PGP) and a short SIP duration.
Fig. 7. The location of the banners
The brightness value of pixel I1 with coordinates (x, y) in the first video frame can be represented in the following form:
A( x y) =1 (x y, %d0) •N, q)
I (x, y, Tdo) is the pixel brightness with coordinates (x,y) in AVA with a gating delay Tdo; N is the number of local AVAs in the video frame.
The second video frame is formed from a certain number of local AVAs shifted relative to each other, which is achieved by a discrete change of delay values of the FSI during one frame. The brightness value of pixel I2 with coordinates (x, y) in the second video frame can
be represented as follows: N M
12 (x, y) = m ' Z I [(x, y, (H + (m -1) • ATd )], (2)
m=1
M is the number of different gating time delays; ATd is the step of gating time delay.
To obtain a linear measuring section (dependence of the object image brightness in the system's field of view from the distance to them) it is necessary to perform normalization of total AVA. Normalization is performed by dividing pixel brightness values I2 in the video frame containing the total AVA by pixel brightness values I1 in the video frame containing the first AVA. To obtain a depth map containing a range value in each pixel, the normalized frame is multiplied by calibration coefficients. The MZRM parameters that measured the distance to lead objects are presented in tab. 3.
Tab. 3. MZRM parameters
Parameter Value
Duration of photodetector gating, ns 200
Duration of SIP, ns 30
Number of local AVA 100
Step between local AVA, ns 1.4
According to the timing of the video frames in the MZRM, a first AVA is generated in each odd frame and a total AVA is generated in each even frame. At the same
time, the resulting sequence of video frames containing depth maps will have half the frequency.
Due to the small angle of view of the AP TMS layout, range measurement and "depth maps" were made for fragments of observed objects that fall into the field of view of the AP TMS layout. Depending on the range of the observation objects, the initial gating delay was established, at which the observed objects were in positive contrast in the images in the frames with the first and total AVA. Fig. 8 and 9 show the frames of the first and total AVA for the area with banners No. 2, 3, and 4, respectively.
Fig. 8. Frame of the first AVA
Fig. 9. Frame of the total AVA
Fig. 10 shows the resulting pseudo-color depth map with a distance-color scale, where the values on the scale are given in meters (right, fig. 10).g. 10 shows the resulting pseudo-color depth map with a distance-color scale, where the values on the scale are given in meters (right, fig. 10).
Fig. 10. Depth map of the area with banners
Table. 4 shows the results of measuring distances to objects in a test facility.
Tab. 4. The result of measuring distances to objects in a test facility
Object of interest Measuring tool
AP TMS, m Geodetic instrument, m Dense point cloud, m
Banner 1 11.12 11 10.98
Banner 2 18.5 19.06 19.05
Banner 3 34.78 34.97 34.89
Banner 4 39.62 39.95 39.9
Tab. 5 shows the difference in the results of measuring distances to objects at the polygon using AP TMS, a geodetic instrument, and a digital terrain model.
Conclusions
Tests showed that the accuracy of prospecting data of digital terrain model has centimeter error of measurement of distances to objects (0.01 m - 0.08 m) on the considered route [27]. The method of measuring the range to objects using AP TMS has a decimeter error of measuring distances (0.05 m - 0.56 m), compared to an exemplary geodetic instrument that has valid metrological verification and was considered a reference measuring tool. The obtained errors in measuring the range to objects using the AP TMS layout are presumably associated with non-stationary conditions during testing (significant changes in temperature and humidity) and due to the fact that in practice, after normalizing the total AVA, the obtained measuring area has some non-linearity, which requires the use of a more complex preliminary calibration of the layout AP TMS.
Tab. 5. Distance measurement difference at the test facility
Object of interest AP TMS and geodetic instrument measurement difference, m AP TMS and dense point cloud measurement difference, m Measurement difference of dense points cloud and geodetic instrument, m
Banner 1 +0.12 +0.14 - 0.02
Banner 2 - 0.56 - 0.55 - 0.01
Banner 3 - 0.19 - 0.11 - 0.08
Banner 4 - 0.33 - 0.28 - 0.05
The high geospatial accuracy of the obtained digital terrain model gives the right to claim that it is applicable for testing the accuracy of distance measurements using AP TMS.
Acknowledgments
The study was carried out at the expense of a grant from the Russian Science Foundation № 21-79-10200 in TUSUR.
References
[1] Busck J, Heiselberg H. Gated viewing and high-accuracy three-dimensional laser radar. Appl Opt 2004; 43(24): 4705-4710. DOI: 10.1364/A0.43.004705.
[2] Sun HY, Guo HC, Li YC. Performance analysis of range-gated active imaging system. Proc SPIE 2009; 7382: 73822E. DOI: 10.1117/12.835576.
[3] Laurenzis M, Christnacher F, Monnin D, Zielenski I. 3D range-gated imaging in scattering environments. Proc SPIE 2010; 7684: 768406. DOI: 10.1117/12.849630.
[4] Artamonov SI, Gryaznov NA, Kharlamov VV, Romanov NA, Sosnov EN. Analytical review of the development of laser location systems. Indian J Sci Technol 2017; 10(15): 1-10. DOI: 10.17485/ijst/2017/v10i15/106332.
[5] David O, Kopeika NS, Weizer B. Range gated active night vision system for automobiles. Appl Opt 2006; 45(28): 7248-7254. DOI: 10.1364/AO.45.007248.
[6] Mariani P, Quincoces I, Haugholt KH, Chardard Y, Visser AW, Yates C, Piccinno G, Reali G, Risholm P, Thielemann JT. Range-gated imaging system for underwater monitoring in ocean environment. Sustainability 2019; 11: 162. DOI: 10.3390/su11010162.
[7] Kapustin VV, Movchan AK, Kuryachiy MI. Vision area parameters analysis for active-pulse television-computing systems. 2017 Int Siberian Conf on Control and Communications (SIBCON) 2017; 1-5. DOI: 10.1109/SIBCON.2017.7998432.
[8] Kapustin V, Movchan A, Kuryachiy M, Chaldina E. Active-pulse television measuring systems images space-time filtration by range. J Phys Conf Ser 2020; 1488: 012032. DOI: 10.1088/1742-6596/1488/1/012032.
[9] Wang X, Zhou Y, Fan S, Liu Y, Liu H. Echo broadening effect in range-gated active imaging technique. Proc SPIE 2009; 7382: 738211. DOI: 10.1117/12.836025.
[10] Gruber T, Bijelic M, Ritter W, Dietmayer KC. (2019). Gated imaging for autonomous driving in adverse weather. Source: (https://vision4allseasons.files.wordpress.com/2019/06/abs tract_gated.pdf).
[11] Andersson P. Long-range three-dimensional imaging using range-gated laser radar images. Opt Eng 2006; 45(3): 034301. DOI: 10.1117/1.2183668.
[12] Laurenzis M, Christnacher F, Monnin D. Long-range three-dimensional active imaging with superresolution depth mapping. Opt Lett 2007; 32(21): 3146-3148. DOI: 10.1364/OL.32.003146.
[13] Xinwei W, Youfu L, Yan Z. Triangular-range-intensity profile spatial-correlation method for 3D super-resolution range-gated imaging. Appl Opt 2013; 52(30): 7399-7406. DOI: 10.1364/AO.52.007399.
[14] Xiuda Z, Huimin Y, Yanbing J. Pulse-shape-free method for long-range three-dimensional active imaging with high
linear accuracy. Opt Lett 2008; 33(11): 1219-1221. DOI: 10.1364/OL.33.001219.
[15] Jin C, Sun X, Zhao Y, Zhang Y, Liu L. Gain-modulated three-dimensional active imaging with depth-independent depth accuracy. Opt Lett 2009; 34(22): 3550-3552. DOI: 10.1364/OL.34.003550.
[16] Gruber T, Kokhova M, Ritter W, Haala N, Dietmayer K. Learning super-resolved depth from active gated imaging. 2018 21st Int Conf on Intelligent Transportation Systems (ITSC) 2018: 3051-3058. DOI: 10.1109/ITSC.2018.8569590.
[17] Zhang C, Kovacs JM. The application of small unmanned aerial systems for precision agriculture: A review. Precis Agric 2012; 13: 693-712. DOI: 10.1007/s11119-012-9274-5.
[18] Zakhlebin AS, Kalibekov A, Smoking MI. Construction of geo-linked orthophotoplane of the terrain from images from the television camera of helicopter-type UAVs [In Russian]. Electronic Means and Control Systems: Mater Dock Int Scien-tific-Prakt Conf, Tomsk, 2020; 1-2: 187-189.
[19] Stroner M, Urban R, Reindl T, Seidl J, Broucek J. Evaluation of the georeferencing accuracy of a photogrammetric model using a quadrocopter with onboard GNSS RTK. Sensors 2020; 20(8): 2318. DOI: 10.3390/s20082318.
[20] Hugenholtz C, Brown O, Walker J, Barchyn T, Nesbit P, Kucharczyk M, Myshak S. Spatial accuracy of UAV-derived orthoimagery and topography: comparing photo-grammetric models processed with direct geo-referencing and ground control points. Geomatica 2016; 70(1): 21-30. DOI: 10.5623/cig2016-102.
[21] Altwaijry H, Trulls E, Hays J, Fua P, Belongie S. Learning to match aerial images with deep attentive architectures. 2016 IEEE Conf on Computer Vision and Pattern Recognition (CVPR) 2016: 3539-3547. DOI: 10.1109/CVPR.2016.385.
[22] Zakhlebin AS. Method of terrain orthofotoplanes construction using unmanned quadrocopter equipped with navigation geodetic receiver [In Russian]. Reports of TUSUR 2021; 3(24): 44-49.
[23] Zakhlebin AS. Aerial photography processing method for building a georeferenced orthophotomap of the area using the television camera of the DJI Phantom 4 PRO unmanned aerial vehicle [In Russian]. AS HS RF Reports 2021; 4(53): 26-35.
[24] Coppa U, Guarnieri A, Pirotti F, Vettore A. Accuracy enhancement of unmanned helicopter positioning with low cost system. Appl Geomat 2009; 1: 85-95. DOI: 10.1007/s12518-009-0009-x.
[25] Haarbrink RB, Eisenbeiss H. Accurate DSM production from unmanned helicopter systems. Int Arch Photogramm Remote Sens Spat Inf Sci 2008; XXXVII(B1): 1259-1264. DOI: 10.3929/ETHZ-B-000011976.
[26] Korshunov RA, Noskov VV, Pogorelov VV. Non-central reverse photogrammetric notch [In Russian]. News of Higher Educational Institutions. Geodesy and Aerial Photography 2013; 5: 67-71.
[27] Chaldina ES, Movchan AK, Kapustin VV, Kuryachiy MI. Multi-area range measurement method using active-pulse television measuring systems. 2020 21st Int Conf of Young Specialists on Micro/Nanotechnologies and Electron Devices (EDM) 2020: 293-297. DOI: 10.1109/EDM49804.2020.9153500.
[28] GOST R 59328-2021. Aerial photography topographic. Technical requirements [In Russian]. Moscow: "Standartinform" Publisher; 2021.
Authors' information
Vyacheslav Valerievich Kapustin, born in 1985, in 2010 he graduated from Tomsk State University of Control Systems and Radio Electronics with a degree in Household Electronic Equipment. Candidate of Technical Sciences (Ph.D.), Associate Professor of Television and Management department of TUSUR, Head of the Laboratory of Television Automation of TUSUR. Scientific interests: television systems, digital image processing, computer vision. E-mail: [email protected].
Alexander Sergeevich Zakhlebin, born in 1993, in 2017 he graduated from Tomsk State University of Control Systems and Radio Electronics with a degree in Radio Engineering, Assistant of Television and Control department of TUSUR, junior research associate in television automation laboratory of TUSUR. Scientific interests: aerial photography, photogrammetric image processing. E-mail: [email protected].
Andrey Kirillovich Movchan, born in 1993, in 2017 he graduated from the master's degree at Tomsk State University of Control Systems and Radio Electronics with a degree in Radio Engineering, an assistant of Television and Control department of TUSUR, junior research associate in television automation laboratory of TUSUR. Scientific interests, digital signal processing, machine vision. E-mail: [email protected].
Mikhail Ivanovich Kuryachiy, born in 1952, in 1974 he graduated from the Tomsk Institute of Automated Control Systems and Radio Electronics with a degree in Radio Engineering. Candidate of Technical Sciences, Associate Professor of Television and Management department of TUSUR, senior research associate in television automation laboratory of TUSUR. Scientific interests: digital signal and image processing, measuring television. Email: [email protected].
Mikhail Vladimirovich Krutikov, born in 1953, in 1977 he graduated from the Tomsk Institute of Automated Control Systems and Radio Electronics with a degree in Radioelectronic Devices, head of the laboratory of the Research Institute of Radio Engineering Systems TUSUR. Scientific interests: radio navigation systems, active and passive radar. E-mail: [email protected] .
Received February 24, 2022. The final version - June 02, 2022.
