CC BY
10
Photospectral Data Obtaining with the Unmanned Aerial Spectrometry Vehicle
A.A. Lamaka, A.V. Gutarau, N.G. Shcherbakou, P.V. Ivuts
A.N. Sevchenko Institute of Applied Physical Problems of Belarusian State University, Kurchatov str., 7, Minsk 220045, Belarus
Received 10.10.2022
Accepted for publication 23.01.2023
Abstract
Study of the Earth's surface objects reflectance characteristics with unmanned aerial vehicles is one of the most actual remote sensing trends. Aim of this work was to develop a method for obtaining of photospectral data using unmanned aerial spectrometry vehicle.
An adaptation of the cameras spatial resolution evaluating technique based on a specialized target photographic fixation was proposed. A method for synchronizing of the camera and spectrometer of the videospectral device was also proposed. It was based on an experiment with spectra and screen images recording. Different colors were sequentially displayed on the screen. The percentage contribution of each of colors to the "mixed" spectra was calculated. So the out-of-sync time estimation became possible. In addition the work proposed the method for combining images and spectra with their merging into photospectral images. The method allows to consider the aircraft displacement when linking the spectrometer field of view to the RGB image. The way for photospectral images combining based on the images key points detectors was also proposed.
Spatial resolutions for 3 aerial vehicle cameras were obtained. The study showed that the spatial resolution decrease of Zenmuse H20T caused by the device carrier movement with a speed of up to 5 m/s can be ignored. The videospectral device camera and spectrometer out-of-sync time was evaluated. An automatic merging of a set of images using key points detection was made. The spectrometry areas were linked to the panoramic image. The reflectance coefficients were obtained for each of the areas in the range of 350-900 nm. The areas to image linking accuracy was 84.9± 11.6 %.
A discrepancy between the angular spatial resolution values got experimentally and theoretically was revealed as a result of the cameras spatial resolution evaluating. This indicates the importance of the imaging equipment spatial resolution experimental evaluation. The videospectral device spectrometer and observation camera out-of-sync time evaluation made it possible to correct the data recording time. This led to the timing error standard deviation reduction from 142 ms to 15 ms. The way for the unmanned aerial spectrometry vehicle data obtaining in a photospectral representation was developed. The proposed methods and techniques can be used in similar unmanned systems.
Keywords: UAV, spectrometer, spatial resolution, software synchronization, image connection. DOI: 10.21122/2220-9506-2023-14-1-7-17
Адрес для переписки:
Ломако А.А.
Институт прикладных физических проблем имени А.Н. Севченко Белорусского государственного университета, ул. Курчатова, 7, г. Минск 220045, Беларусь e-mail: alekseylomako@gmail.com Для цитирования:
A.A. Lamaka, A.V. Gutarau, N.G. Shcherbakou, P.V. Ivuts.
Photospectral Data Obtaining with the Unmanned Aerial
Spectrometry Vehicle.
Приборы и методы измерений.
2023. - Т. 14, № 1. - С. 7-17.
DOI: 10.21122/2220-9506-2023-14-1-7-17
Address for correspondence:
Lamaka A.A.
A.N. Sevchenko Institute of Applied Physical Problems of Belarusian State University,
Kurchatov str., 7, Minsk 220045, Belarus e-mail: alekseylomako@gmail.com
For citation:
A.A. Lamaka, A.V. Gutarau, N.G. Shcherbakou, P.V. Ivuts.
Photospectral Data Obtaining with the Unmanned Aerial
Spectrometry Vehicle.
Devices and Methods of Measurements.
2023, vol. 14, no. 1, pp. 7-17.
DOI: 10.21122/2220-9506-2023-14-1-7-17
Получение фотоспектральных данных с использованием беспилотного комплекса авиационного спектрометрирования
А.А. Ломако, А.В. Гуторов, Н.Г. Щербаков, П.В. Ивуть
Институт прикладных физических проблем имени А.Н. Севченко Белорусского государственного университета, ул. Курчатова, 7, г. Минск 220045, Беларусь
Поступила 10.10.2022 Принята к печати 23.01.2023
Исследование отражательных характеристик объектов на поверхности Земли с использованием беспилотных летательных аппаратов является одним из развивающихся направлений в дистанционном зондировании Земли. Целью работы являлась разработка способа получения фотоспектральных данных с использованием беспилотного комплекса авиационного спектрометрирования.
Предложена адаптация методики определения пространственной разрешающей способности камер на основе фотофиксации специализированной миры. Также предложен способ синхронизации камеры и спектрометра видеоспектрального комплекса, основанный на проведении эксперимента по регистрации спектров и изображений экрана, где последовательно отображаются различные цвета. Предложен способ комбинирования изображений и спектров с объединением их в единое фотоспектральное изображение. Способ позволяет учитывать смещение летательного аппарата при привязке поля зрения спектрометра к изображению. Предложен способ объединения фотоспектральных изображений, основывающийся на сшивке изображений по особым точкам.
Получены угловые разрешающие способности для 3 камер из состава беспилотного комплекса. Показано, что при движении беспилотного комплекса со скоростью до 5 м/с ухудшение разрешающей способности камеры 7вптшв Н20Т, вызванное движением носителя аппаратуры, можно не учитывать. Определено время рассинхронизации работы камеры и спектрометра из состава видеоспектрального комплекса. Проведена автоматическая сшивка ряда изображений по особым точкам с привязкой к сшитому изображению областей спектрометрирования (относительная точность привязки к изображению - 84,9± 11,6 %). Для каждой из областей спектрометрирования получены коэффициенты спектральной яркости в диапазоне 350-900 нм.
В исследовании выявлено расхождение экспериментальных и теоретических значений углового пространственного разрешения, что свидетельствует о важности проведения экспериментальной оценки пространственного разрешения съёмочной аппаратуры. Определение времени рассинхронизации спектрометра и обзорной камеры видеоспектрального комплекса позволило осуществить коррекцию времени регистрации данных, что привело к уменьшению среднеквадратичной величины ошибки синхронизации со 142 мс до 15 мс. Разработан способ получения данных БЕКАС в фотоспектральном представлении.
Ключевые слова: БПЛА, спектрометр, пространственное разрешение, программная синхронизация, сшивка изображений.
DOI: 10.21122/2220-9506-2023-14-1-7-17
Адрес для переписки:
Lamaka A.A.
A.N. Sevchenko Institute of Applied Physical Problems of Belarusian State University,
Kurchatov str., 7, Minsk 220045, Belarus e-mail: alekseylomako@gmail.com
Для цитирования:
A.A. Lamaka, A.V. Gutarau, N.G. Shcherbakou, P.V. Ivuts.
Photospectral Data Obtaining with the Unmanned Aerial
Spectrometry Vehicle.
Приборы и методы измерений.
2023. - Т. 14, № 1. - С. 7-17.
DOI: 10.21122/2220-9506-2023-14-1-7-17
Address for correspondence:
Lamaka A.A.
A.N. Sevchenko Institute of Applied Physical Problems of Belarusian State University,
Kurchatov str., 7, Minsk 220045, Belarus e-mail: alekseylomako@gmail.com
For citation:
A.A. Lamaka, A.V. Gutarau, N.G. Shcherbakou, P.V. Ivuts.
Photospectral Data Obtaining with the Unmanned Aerial
Spectrometry Vehicle.
Devices and Methods of Measurements.
2023, vol. 14, no. 1, pp. 7-17.
DOI: 10.21122/2220-9506-2023-14-1-7-17
Introduction
Surface images recorded using unmanned aerial vehicles (UAVs) are widely used in the modern world, along with data obtained using space-based remote sensing equipment [1].
Remote sensing using UAVs has its advantages compared to satellite systems: high spatial resolution, high speed of information acquisition, ease of use, no cloud cover influence, low cost [2]. UAV data is currently widely used in agriculture and forestry [3-5]. As a result, UAV data acquisition and processing techniques are developing rapidly.
The devices with multispectral cameras are the most widely used among UAVs [1]. Such systems have a set of undeniable advantages, for example, the possibility of obtaining geographically referenced images of vegetation indices for vast areas. At the same time, multispectral systems allow recording data for subsequent analysis only for a limited number of vegetation indices. It is caused, firstly, by limited number of spectral bands and, secondly, by vast bandwidths.
The development of videospectral systems allowing to record high spectral resolution spectra combined with the underlying surface images is one of the approaches to expand the scope of remote sensing data obtained using UAVs. The unmanned aerial spectrometry vehicle (UASV) became such a system [6]. The final stage in the vehicle development was the technical characteristics analysis: the resolution of the spectrometer and observation camera, the system modules angle of view, the system modules level of synchronization, etc.
To date, there are no state standards for evaluating the angular spatial resolution per pixel of aerial photosystems. At the same time, there is a technique describing a way of the spatial resolution evaluating based on a flight experiment results [7]. However, this technique does not describe the requirements for the target intended to evaluate the spatial resolution and allows to find only the linear resolution of the system without assessing the uncertainties. As a result, it needs to be supplemented and adapted for use in systems like the UASV.
There are no studies describing modules synchronism level in systems like the UASV. Very little attention is paid to videospectral systems due to the difficulty of obtaining data from such systems in a representation convenient for analysis (for example, a geotagged RGB-image with a well-defined area
on it corresponding to a surface reflectance spectrum). To obtain such a representation, it is necessary to apply data preprocessing methods, including modules synchronization, georeferencing, correction of distortion effects in images. In addition, there is a task of images combining with the use of computer vision libraries. This task becomes more difficult when spectrometer and camera exposition times are considerably different.
There are several tasks that this study solved: elaboration and applying of the observation camera spatial resolution evaluating methodology based on the flight experiment results; development of the method for the data of the spectrometry vehicle spectrometer and observation camera synchronizing; development of the method for obtaining the UASV data in a presentation convenient for analysis. Thus, the aim of the study was to develop the method for obtaining a photospectral data using the UASV.
Devices and methods
The UASV in the configuration used is based on the UAVDJIMatrice 300 RTK (Real Time Kinematic). The positioning accuracy of it, when the RTK system is on, is 1 sm±1 mm/km in horizontal axes and 1.5 sm± 1 mm/km in vertical axes (hovering accuracy in P-GPS mode is ±0.5 m)1. The videospectral device (VSD) and the Zenmuse H20T quad-sensor camera on a stabilizing gimbal2 can be installed on the UAV simultaneously or separately (Figure 1).
The VSD (Figure 2) includes the single-board computer (SBC) Raspberry Pi 4 Model B3. It controls in automatic mode the observation camera (Raspberry Pi Camera) and the spectrometer (connected through the microcontroller STM32F405RGT64) mounted coaxially. The observation camera allows to register RGB-images with resolution up to 2592* 1944 pixels in photography mode. The spectrometer optical scheme is built according to the Rowland scheme [8] based on a concave diffraction
1 UAV DJI Matrice 300 RTK: https://www.dji.com/ matrice-300
2 DJI Zenmuse H20T: https://www.dji.com/zenmuse-h20-series
3 Raspberry Pi 4: https://www.raspberrypi.com/ products/raspberry-pi-4-model-b
4 Microcontrollers & Microprocessors: Highperformance foundation line (STM32F405RG): https:// www.st.com/en/microcontrollers-microprocessors/ stm32f405rg.html
grating with an additional flat mirror to reduce the dimensions. The spectral range of the spectrometer is 350-900 nm, the spectral resolution is 4 nm.
The spectrometer angular field of view (FOV) is 0.7^1.2 degrees, the observation camera FOV is 15^27 degrees.
a b
Figure 1 - The unmanned aerial spectrometry vehicle general form (a), Zenmuse H20T camera with a stabilizing gimbal (b)
The VSD also includes a power supply module with 10 Ah capacity. The body of the VSD is made using three-dimensional printing with three types of filaments for various parts.
The data registered by the VSD are pre-processed and stored on the flash drive connected to the SBC. The UASV output data is a video se-
quence of data divided by directories (30 frames per second, 1920x 1080 pixels) and data registered with the spectrometer (the recording frequency depends on the exposure of the spectrometer and is about 10 spectra/s). Geographic location data during measurements is recorded and stored on the flash drive connected directly to the UAV.
Figure 2 - 3D image of the videospectral device construction. Left view (a), bottom view (b): 1 - single-board computer; 2 - USB flash memory; 3 - spectrometer lens; 4 - camera lens; 5 - spectrometer block; 6 - battery; 7 - power switch; 8 - stop shooting button
The observation camera spatial resolution evaluation
The information from the article on the aerial photosystems linear resolution per pixel estimating methodology based on the flight tests [7] were taken as a basis to determine the spatial resolution of the UASV cameras. It was complemented with recommendations described in the state standards for the
resolution evaluating methods and photogrammetry techniques5.
The first step for holding an experiment to evaluate the spatial resolution in flight conditions
5 Photogrammetry. Requirements for creating oriented aerial images for building stereo models of built-up areas (GOST R 58854-2020, in Russian);
Method for determining the photographic resolving power (GOST 2819-84, in Russian)
is to determine the theoretical linear resolution per pixel using the formula:
L0 = H •S, 0 f
(1)
where L0 is theoretical linear resolution per pixel; H is shooting height; f is camera lens focal length; S is pixel size in the camera matrix.
It is necessary to make a target for experimental evaluation of the camera spatial resolution based on the theoretical values calculated with the formula (1) for all cameras needed. The minimum distance between alternating white strokes has to be equal to the calculated value L0 for a given flight altitude, and the maximum distance between alternating white strokes has to be equal to the expected experimental L0 value for a given flight altitude. It is important to note that formula (1) is a rough below estimation of the camera spatial resolution. For more accurate calculations, in addition to the focal length of the camera lens, the shooting height and the physical size of the matrix pixel it is needed to consider the area, diameter and transmission of the input lens, the specific detectivity of the photodetector, as well as the speed of the UAV during the experiments [9].
The UASV incorporates several cameras, which resolutions are needed to be evaluated experimentally: the VSD observation camera and the Zenmuse H20T camera including observation, wide-angle and radiometric thermal cameras. The Zenmu-se H20T observation camera is a zoom camera, so it stands out from the general list. As a result, it was not included it in the current analysis. It is necessary to calculate the L0 value for heights of 50 and 200 m using the formula (1) for all the cameras (minimum and maximum flight altitude to evaluate the spatial resolution in this study). After that, it is necessary to choose extremum L0 values. These values will be the basis for the line width border values in the target.
The cameras parameters, as well as the theoretical linear resolutions for various flight altitudes, are shown in Table 1 (L050 is the theoretical linear resolution per pixel for 50 m altitude, L0200 is the theoretical linear resolution per pixel for 200 m altitude).
In general, the target should consist of a set of line groups with different frequencies. Each group consists of five light parallel lines on a dark background. The spacing width between the lines must be equal to the line width in the group. The line length to width ratio is constant and equal to 10. Wherein the width of the lines in the target should decrease
from the previous group to the next in a geometric progression with a 0.91 denominator.
Table 1
Cameras parameters and theoretical linear resolutions for different unmanned aerial spectro-metry vehicle flight altitudes
Camera f, mm S, mkm L 50 0, mm T 200 L0 , mm
VSD Overview Camera 8.00 1.40 8.75 35.00
Zenmuse H20T (Wide) 4.50 2.38 26.51 105.70
Zenmuse H20T (Thermal) 13.50 9.80 36.30 145.18
A chart target based on the data from Table 1 was designed. The minimum and maximum distances between white lines was respectively 32 mm and 100 mm. As a result, the target consists of 13 groups of lines. The width of lines depending on the group number is presented in Table 2. The general view of the chart target is shown in Figure 3.
Table 2
The chart target lines width depending on the group number
Group number (N) 1 2 3 4 5 6
Line width (l), mm 100 91 82 75 69 62
N 7 8 9 10 11 12 13
l, mm 57 52 47 43 39 35 32
Figure 3 - General view of the chart target for the unmanned aerial spectrometry vehicle cameras spatial resolution evaluating
The cameras spatial resolution evaluating experiment have to be carried out in the absence of shading the measurement area surface with any objects [7]. Spatial resolution measurement is performed on at least 20 target images for each altitude. Four independent decryption operators are recommended
for image analysis. The work of the operators is to recognize groups of lines with all lines differ along their entire length.
Next, the experimental angular resolution of the cameras is determined by the formula:
l 180
9 = arctg—--,
H n
(2)
where 9 is the experimental angular resolution for a group of lines; l is the target line width from the group of lines recognized by the operator; H is the UAV flight altitude.
It is necessary to recognize the tightest groups of lines in each of the images for a given flight altitude, and then calculate the average value and the standard deviation (SD) of the experimental angular resolution per pixel over the entire set of images from a given camera at a given flight altitude. At the same time, since the flight speed of the UAV affects the resolution ([9]), in this study measurements were carried out both in static mode (when the UAV was at one point during the shooting) and in dynamic mode (when the UAV was moving during the shooting).
This study realizes statistical processing of the results on the basis of state standards in the field of laboratory measurements evaluation6. At the first stage, it was estimated that the measurement results fell into the critical range at a 95 % confidence level for the number of independent estimates equal to the number of decryption operators. Then the average or median value (depending on the scope of all operators results) for all processing results for each group was calculated. If several independent groups of measurements were obtained to evaluate the spatial resolution at a given altitude (for example, with different lighting conditions), the Cochran criterion statistics were calculated. As a result, measurement groups with outlier characteristics were excluded. After that, the values of angular resolutions obtained for different flight altitudes were averaged. The number of independent degrees of freedom in
Accuracy (trueness and precision) of measurements methods and results: Part 2 - Basic method for the determination of repeatability and reproducibility of a standard measurement method (ISO 5725-2:1994/ Cor.1:2002); Part 6 - Use in practice of accuracy values (ISO 5725-6:1994/Cor.1:2001);
Uncertainty of measurement: Part 3 - Guide to the expression of uncertainty in measurement (ISO/IEC Guide 98 - 3:2008)
the evaluation of extended uncertainties was chosen equal to the total number of decrypted images for all groups of measurements for a particular camera. The coverage ratio was evaluated based on this.
The unmanned aerial spectrometry vehicle modules data synchronization
The task of synchronizing the spectrometer and the VSD observation camera was the next task needed to solve for the UASV thematic data processing possibility. This was necessary for accurate binding of the spectrometry area to the RGB image both recorded by the VSD. The highest level of synchronization could be achieved with hardware methods, but they were not available with current UASV realization. Therefore, a software method of equipment synchronization was chosen in this work.
The VSD registers the observation camera video with a 30 Hz frequency and spectra with a 10 Hz frequency in flight mode. Subsequently, the registered data is processed: the observation camera data is matched to the spectrometer data. Each data set corresponds to the registration time related with the SBC time. However, both data sources have their own delays. The out-of-sync time is determined here as the time difference between the own delays of two data sources.
In addition, it is important to check the constancy of such a time difference over a long period of measurements («30 min).
The new additional software module designed for various colors photo images broadcasting on a screen was created to solve the problem. In this case it was possible to make a videospectral data registration using the VSD in laboratory conditions. A certain screen changing colors frequency could be set. Then it was possible to evaluate the out-of-sync time of the VSD observation camera and the VSD spectrometer.
Eight colors in RGB color gradation were selected for the experiment: (0,0,0); (255, 255, 255); (255,0,0); (0, 255, 255); (0, 255,0); (255,0, 255); (0,0, 255); (255, 255,0). The spectrometer exposure time ta was fixed and was equal to 100 ms. The color change period on the screen T was selected to be 5 s to register spectrum corresponding to a signal of one color (without mixing due to time offset) at the first stage of measurements. The period T was fixed at 100 ms at the next stage.
Spectra registered during the measurements were "mixed" from initially selected color pairs due
to time displacement. The percentage contribution of each of the colors to the "mixed" spectra was calculated with the method for evaluating the composition of a spectrum mixture (active set algorithm, [10]). Thus, the out-of-sync time needed could be estimated.
Images and spectra merging
Two RGB images are allocated for each spectrum: a frame of the beginning and a frame of the end of spectrometry. These are two images shifted relative to each other in time by the spectrometer exposure value (with an error caused by the out-of-sync time).
Merging of two images described above with common panoramic image generation can be made after radial and tangential distortion effects correction (for example, with the use of ChArUco method, [11]). This merging task was performed using the method of forming panoramic images based on key points detection [12]. Special algorithms (detectors) were used for this. The numerical characteristics of key points in doing so are determined using other computing algorithms -descriptors, where a descriptor is a kind of mathematical construct (usually a vector) that describes a key point in some way and allows to compare different points with each other [11]. The ORB detector (Oriented FAST and Rotated BRIEF), the FREAK descriptor (Fast Retina Key-point) and the Hamming matching method were used as algorithms for searching and describing key points [11]. A panoramic image merged from two frames is the result of this stage.
The part of image matching to the spectro-metry area can be defined for any image registered with the VSD since the spectrometer FOV is rigidly connected with the observation camera FOV. In this case, for the panoramic image obtained at the previous stage, two regions can be distinguished. The actual spectrometry region bound to the RGB image becomes known after combining such regions (with the inclusion of the entire area between them).
The steps described above were repeated for the data set registered during the experiment. Comparison of videospectral data and UAV location data by timestamps was the last stage of UASV data preprocessing. Thus, each panoramic image was assigned with the coordinate of the UAV for the further photospectral data geotagging possibility.
Results of experimental studies and their discussion
The flight experiment was conducted on the territory of the educational geographical station of Belarusian State University "Western Berezina" to evaluate the spatial resolution of the cameras from the UASV on November 6-11, 2022. Data recording altitudes varied from 50 to 200 m (different cameras require different shooting altitudes due to different theoretical spatial resolutions). The values calculated with the use of Table 1 and experimentally obtained angular spatial resolution values estimation (9) and their SD (a) for various cameras are presented in Table 3.
Table 3 shows that theoretical cameras spatial resolution may differ significantly from the experimental value. Thus, the theoretical resolution differs from the experimental one for the Zenmuse H20T wide-angle camera by 30 %. Meanwhile, the relative extended uncertainty of angular resolution for 95 % confidence probability was 14 %. The theoretical resolution for the Zenmuse H20T thermal imaging camera differs from the experimental one by 20 %. The relative extended uncertainty of angular resolution for 95 % confidence was 7.7 %. Various factors may be the reason for the differences in theoretical and experimentally obtained resolutions: features of the frequency-contrast characteristics, the quality of the lens, the accuracy of its installation, as well as the influence of external factors.
An experiment to evaluate the spatial angular resolution was also conducted while the UAV is moving at a speed of 5 m/s in addition to the static mode experiment. Further sequential measurements were carried out in static and dynamic modes in similar light conditions. The results of the resolution estimates for the Zenmuse H20T cameras (9) and their SD (a) are shown in Table 4.
Table 4 shows that angular spatial resolution relative difference for various measurement modes was 6.9 % for a thermal camera and 1.6 % for a wide-angle camera. Meanwhile, standard deviations of the values were comparable or higher than the resolution difference. This suggests that the spatial resolution deterioration caused by the shooting equipment carrier movement at a speed of up to 5 m/s can be ignored in experiments with the Zenmuse H20T camera.
Table 3
Comparative table of theoretical and experimental angular spatial resolution values for different cameras in static mode
Camera
VSD Observation Camera (1080p)
Zenmuse H20T (Wide) Zenmuse H20T (Thermal)
deg.
o, deg.
deg.
o, deg.
3, deg.
o, deg.
Theoretical resolution
Experimental resolution
0.0100
0.0211
0.0016
0.0303
0.0392
0.0028
0.0416
0.0499
0.0020
Table 4
Angular spatial resolution values in static and dynamic modes experimental results
Zenmuse H20T Zenmuse H20T
Camera (Wide) (Thermal)
6, deg. o, deg. 6, deg. o, deg.
Static mode 0.0473 0.0027 0.0485 0.0016
Dynamic mode 0.0481 0.0046 0.0520 0.0032
An experiment to find out-of-sync time for the VSD observation camera and the VSD spectrometer was made. The measurements of the time-varying test screen were carried out. Spectra of "pure" colors were recorded, as well as "mixed" from pairs of initially selected colors due to time displacement.
The out-of-sync time evaluation is shown by the example of a mixture of two colors pair spectrum: (0,255,255) and (0,255,0). Figure 4 shows spectrum of screen with colors (0,255,255) and (0,255,0) in "pure" (Figure 4a) and "mixed" (Figure 4b) forms. Figure 5 shows a time diagram corresponding to the registration of spectrum shown in Figure 4b. Time moments tx, t2 and t4 shown in this figure are the moments of color change in camera images. Time moments t3 and t5 are the moments of the beginning and the end of spectrometry. The percen-tage of the energy component in the spectrum can be used as a percentage of the particular color type re-gistration time for time shifting evaluation, because the color change period T was equal to the spectro-meter exposure time tc in the experiment. Thus, it is possible to evaluate the out-of-sync time for each spectrum in the recorded data series.
300-
250-
i/3 200-
Я
3 150-
о
ü 100-
<
50-
o-
300
150 120 90 60
и о
< 30
— Color (0,255,255) _ Color (0,255,0)
400 500 600 700 800 900 Wavelength (nm) a
Mixed colors: 23 % - (0,255,255) 77 % - (0,255,0)
300 400 500 600 700 800 900 Wavelength (nm) b
Figure 4 - "Pure" (a) and "mixed" (b) spectra registered in the out-of-sync time evaluating experiment
Figure 5 - Unsynchronized data registration timing diagram
Data registration have been carrying out for 30 min. After analyzing the entire volume of information, it was found out that the out-of-sync time was on average 128 ms. It changed by an average of 15 ms during the entire time of the experiment. Thus, the FOV displacement caused by a synchronization error (Figure 6) was average from 0.71 pixels at a UAV speed (V) of 1 m/s (with a SD g] equal to 0.56 pixels) to 3.56 pixels at a UAV speed of 5 m/s (with a SD Gt equal to 2.79 pixels). The FOV displacement Ay (with a SD Gy, ) in the direction perpendicular to the average velocity vector may be caused by the UAV roll angles changes (the vibration isolation used in the UASV design at the VSD installation site is not enough to compensate for the UAV rolls), as well as wind load. This calculation assumes that the displacement velocity in the direction perpendicular to the average velocity vector is equal to the modulus of the average specified UAV flight speed. In this case, with the flight altitude of 100 m, the UAV speed of 1 m/s and the spectrometer exposure of 100 ms, the FOV position relative accuracy considering synchronization errors was:
S - 2 AS* - 2ASy KS + 2 AS? + 2ASy ,
± k
1 --
S - 2AS* - 2ASy
G_G_
S + 2AS* + 2ASy .
G G J
(3)
= 0.937 ± 0.099,
where S is the area of the spectrometer's FOV (in pixels) after its binding to the RGB image; ASf,y is the spectrometer FOV area change caused by displacement due to synchronization errors; AS*'y is the spectrometer FOV area change caused by
the uncertainty of the FOV displacement; k is coverage factor for 95 % confidence probability used in the extended measurement uncertainty calculation.
Figure 6 - Uncertainty of binding the spectrometer field of view to the RGB image during the UAV movement: 1 - spectrometer field of view in the data registration start moment; 2 - spectrometer field of view in the data registration end moment; w - spectrometer field of view size in the UAV flight direction; Aw - spectrometer field of view bias caused by UAV movement
Videospectral data registered in flight experiment needed to be preprocessed. The video sequence data were converted first to pairs of images, and then to the form of geographically linked panoramic images. The direction and magnitude of the offset between the two images allow determining the position of the spectrometer FOV in the panoramic image (Figure 7), since the spectro-metry area is linked to each of the images with an accuracy determined only by the VSD synchronization. It is also possible to generate a panoramic image with several spectrometry regions attached to it after merging together several images (Figure 8).
=
Figure 7 - Combining images with the spectrometry areas linking: 1 - spectrometrication starting frame; 2 - spectro-metrication ending frame
0.5 -
0.4 -
0.3 -
0.2 -
С 1} Pi
0.1 -
0 -
— 1 — 2
— 3
— 4
— 5
— 6
merging images accounting the shooting equipment distortion correction, is 2.29 pixels (with a SD am equal to 1 pixel). It is 1.53 pixels (with a SD of 0.48 pixels) while determining the displacement magnitude in the region close to the spectrometry area. These values include the displacement both along the axis collinear to the average velocity vector and along the axis perpendicular to this vector. Then, by analogy with (3), at the flight altitude of 100 m, the UAV speed of 1 m/s and the spectrometer exposure of 100 ms, the FOV position relative accuracy considering merging errors was:
( S - 2 AS ^ ± к ( (1 - S - 2 AS* )
m a
КS + 2ASm j К S + 2ASa J
= 0.910 ± 0.058,
(4)
400 500 600 700 800 900 Wavelength (nm) b
Figure 8 - Videospectral device data combining: connected image (a) and reflectance spectra corresponding selected areas (b)
Ground-based registration of solar radiation reflection spectra from the MS-10 milk glass was carried out using the FSR-02 photospectroradiom-eter with the "Calibrovka" hardware-software system [13]. It was made synchronously with flight measurements in the flight area and was needed to obtain the spectral brightness coefficients based on the spectrometry results. The FSR-02 photospec-troradiometer has spectral resolution and spectral range similar to the VSD spectrometer. The data of both spectrometers was recalculated into the energy brightness spectral density values using their radio-metric calibrations. After that, the spectral brightness coefficients spectra of the underlying surfaces were calculated.
Methods based on the use of key points detectors and descriptors cope well with the task of merging images into a panoramic image. The average deviation of the offset between key points Axm when
where ASm is changes in the spectrometer's FOV area caused by displacement due to merging errors.
The dependence function of the general uncertainty value of the spectrometer FOV binding to the RGB image was taken as the sum of independent variables: a variable considering merging errors and a variable considering synchronization errors. In this case, basing on (3) and (4), the total relative accuracy of the spectrometer FOV to the RGB image binding was:
s = 0.849 ± 0.116. Conclusion
The adaptation of existing techniques and recommendations for the spatial resolution of UAV cameras determining was carried out in the course of flight experiments with the unmanned aerial spectrometry vehicle. It consisted in considering several standards for determining the characteristics of photosystems, in the transition from determining the linear resolution per pixel to determining the angular resolution. It also allowed evaluating the accuracy of measurement results. The discrepancy between the experimentally obtained values and the theoretical values of the angular spatial resolution calculated on the cameras factory characteristics basis was revealed as a result of determining the unmanned aerial spectrometry vehicle cameras spatial resolution. This suggests the need for an experimental assessment of the shooting equipment spatial resolution to identify the real characteristics of the cameras. The proposed adaptation of the spatial resolution estimation technique can be used in systems similar to the unmanned aerial spectrometry vehicle.
a
e m =
The spectrometer and the videospectral device observation camera out-of-sync time was determined using the developed method of software synchronization. The out-of-sync time was 128 ms. This made it possible to correct the time of data registration, which led to a decrease in the standard deviation value of the synchronization error from 142 ms to 15 ms. The proposed way of out-of-sync time estimation for various sensors of videospectral systems can be used in systems similar to the unmanned aerial spectrometry vehicle.
A method for obtaining the unmanned aerial spectrometry vehicle data in the photospectral representation was developed in the course of the study. It provides the spectrometer field of view to RGB images binding with a relative accuracy of 84.9± 11.6 %. Combining a set of photospectral images into a single cross-linked image allows the formation of sparse hyperspectral images. Thus, the unmanned aerial spectrometry vehicle is a promising system for getting remote sensing data for a wide range of the Earth surface studying and monitoring tasks.
References
1. Lu H., Fan T., Ghimire P., Deng L. Experimental Evaluation and Consistency Comparison of UAV Mul-tispectral Minisensors. Remote Sens, 2020, no. 12(16), pp. 2542. DOI: 10.3390/rs12162542
2. Iizuka K., Itoh M., Shiodera S., Matsubara T., Do-har M., Watanabe K. Advantages of unmanned aerial vehicle (UAV) photogrammetry for landscape analysis compared with satellite data: A case study of postmining sites in Indonesia. Cogent Geosci, 2018, vol. 4, pp. 1498180. DOI: 10.1080/23312041.2018.1498180
3. Dash J.P., Watt M.S., Pearse G.D., Heaphy M., Dungey H.S. Assessing very high-resolution UAV imagery for monitoring forest health during a simulated disease outbreak. ISPRS J. Photogramm, 2017, vol. 131, pp. 1-14. DOI: 10.1016/j.isprsjprs.2017.07.007
4. Bendig J., Yu K., Aasen H., Bolten A., Bennertz S., Broscheit J., Gnyp M.L., Bareth G. Combining UAV-based plant height from crop surface models, visible, and
near infrared vegetation indices for biomass monitoring in barley. Int. J. Appl. Earth Obs, 2015, vol. 39, pp. 79-87. DOI: 10.1016/jjag.2015.02.012
5. Candiago S., RemondinoF., De Giglio M., Dub-bini M., Gattelli M. Evaluating Multispectral Images and Vegetation Indices for Precision Farming Applications from UAV Images. Remote Sens, 2015, vol. 7, pp. 40264047. DOI: 10.3390/rs70404026
6. Gutarau A.V., Lamaka A.A., Belyaev B.I., SosenkoV.A., Ivut P.V. Unmanned Aerial Spectrometry Vehicle. The 8th Belarussian Space Congress: materials, Minsk, 2022, October 25-27, vol. 1, pp. 129-132 (in Russian).
7. Molchanov A.S. Methodology of evaluation of linear permit per pixel of aerophotosystems of military purpose when conducting flight tests. Izvestia vuzov. Geodesy andAerophotosurveying, 2018, vol. 62(4), pp. 390396 (in Russian).
DOI 10.30533/0536-101X-2018-62-4-390-396
8. Peisahson I.V. Optics of Spectral Instruments. Ed. 2nd, add. and reworked. Instruments with concave diffraction gratings. Leningrad: Mashinostroenie Publ., 1975, Ch. 6, pp. 222-227.
9. Katkovsky L.V. Calculation of objects thermal imaging parameters from unmanned aerial vehicles. Doklady BGUIR, 2020, vol. 18(2), pp. 53-61 (in Russian). DOI: 10.35596/1729-7648-2020-18-2-53-61
10. Box and linearly constrained optimization [Electronic resource]. ALGLIB - Optimization (nonlinear and quadratic). Avialable at: https://www.alglib.net/op-timization/boundandlinearlyconstrained.php. (accessed: 01.09.2022).
11. Kaehler A., Bradski G. Learning OpenCV 3. Published by O'Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA, 2016, pp. 511-583.
12. Lamaka A.A. Considering camera distortion panoramic images forming method for unmanned aerial vehicle multispectral data. Journal of the Belarusian State University. Physics, 2022, vol. 2, pp. 60-69 (in Russian). DOI: 10.33581/2520-2243-2022-2-60-69
13. Katkouski L.V. Hardware-software system "Ca-librovka" for ground-based spectrometry of the underlying surface and atmosphere. The 7th Belarussian Space Congress: materials, Minsk, 2017, October 25-27, vol. 2, pp. 36-40 (in Russian).
