CC BY
12
Vol. 22, No. 05, 2019
Ovil Aviation High Technologies
UDC 629.7.058
DOI: 10.26467/2079-0619-2019-22-5-107-116
ELECTROOPTICAL COMPLEX FOR TERRAIN ON-TIME SURVEY
S.B. STUKALOV1, D.S. STUKALOV1, V.I. KONDRIKOV1, R.S. GAVRYUSHIN1
Moscow State Technical University of Civil Aviation, Moscow, Russia
The article deals with a number of approaches to the use of electrooptical systems and unmanned aerial vehicles (UAV) to solve the tasks of live terrain surface screening, air incident spots and objects survey. The paper provides the grounds for the UAV airframe chosen structure and the design of the electrooptical complex. The light weight, high -wing monoplane airframe structure is suggested. Using the CNC production center, a UAV with high aerodynamic performance was developed. The UAV was fitted with the optoelectronic system. Structural design for the airborne and ground equipment of the survey complex was developed. The structural design comprises the forward-looking and side airborne surveillance cameras and also downward-looking cameras of visible and infrared bands to provide the day and night vision. The downward-looking cameras of visible and infrared bands, integrated into the UAV fuselage belly are used for the surface monitoring and survey. The flight test on the use of electrooptic system comprised the flight on the route and automatic return using the satellite system navigation. The test showed that navigation data correction was required. The positioning accumulative error elimination is suggested to be done via automated image registration with the contour extraction and landmark reference. The image processing results using Canny algorithm were presented. The recommendations on the algorithm practical application and speed of response are given. To solve the problem of terrain on-time surveillance and monitoring the flight routes were subjected to research. Based on the experiments performed, the recommendations were given on the electrooptic complex application and flight route plotting for the tasks of on-time search and rescue.
Key words: electrooptical complex, unmanned aerial vehicles (UAVs), flight experiment, correction of navigation parameters, image processing.
INTRODUCTION
At present, Unmanned Aerial Vehicles (UAVs) which are able to fly automatically on route with precise navigation, are increasingly used for the purpose of ground and terrain monitoring [1, 2] One of the main tasks of UAV flight is the ground monitoring and terrain survey aimed at on-time search of the air incident spots and objects. For such cases the UAVs possess the advantage of quick position and coordinate finding, and also of the opportunity to fly and perform tasks both in daytime and at night.
THE CHOICE AND DESIGN OF UAV AIRFRAME AND ELECROOPTIC ON-TIME
SURVEY COMLEX STRUCTURE
For the tasks mentioned above, the performance required may be provided by the high-wing low-weight airframe structure. The high wing airframe provides a wide range of operational alignments, the enhanced downward-looking visibility, better terrain observation and landing characteristics. A wide wing-span choice provides better aerodynamic performance.
The authors applied AutoCAD software to design and draft the UAV airframe. It enabled to adjust the optimal fuselage design with the cowl compartment for video equipment and transmitters (figure 1).
Ovil Aviation High Technologies
Vol. 22, No. 05, 2019
Fig. 1. The result of the UAV airframe design with the use of AutoCAD
The light-weight structure of the wings, fuselage and tail unit was obtained using the foam plastic cutting and CNC production center lamination. The UAV powerplant consisted of the electrical engine AZ2814 980kv with brushless speed controller Turnigy Plush 60A c BEC 3A.
The flow charts were designed for the electrooptic system cameras switching sequence, and devices for airborne (figure 2) and ground equipment were developed (figure 3).
Fig. 2. Flow chart of airborne equipment of the complex
The airborne equipment comprises the video data transmitter, flight controller, camera selector switch, GPS sensor, antennae, electrooptical cameras № 1 and № 2 for flight management and surveillance cameras of visible and infrared bands.
Vol. 22, No. 05, 2019
Civil Aviation High Technologies
Fig. 3. Flow chart of control devices of the ground equipment of the complex
The complex ground equipment comprises the video data receiver, monitor, remote control, Head Tracker (the cameraman head turn control) with the output from video receiver to the FPV (First Person View) goggles. For navigation and precise UAV positioning the GPS sensor is installed to monitor the surveillance complex position. The developed electrooptic complex (figure 4 a) features video cameras with enhanced vision spectral bandwidth (figure 4 b):
Camera 1 - forward looking, connected to the flight controller system and providing the forward visibility
Camera 2 - side looking with the turn system and Head Tracker
Such system allows to find position day and night and adjust the UAV flight en route according to the reference set.
The surveillance equipment of the UAV are the two downward-looking cameras of infrared and visible bands mounted at the fuselage belly (figure 4 a).
These cameras tackle the task of the surface monitoring in the infrared and visible bands.
Fig. 4. Unmanned aerial vehicle with airborne and ground equipment (a) and on-board cameras
No. 1 and No. 2 (b)
Ovil Aviation High Technologies
Vol. 22, No. 05, 2019
THE RESULTS OF ELECTROOPTIC COMPLEX FLIGHT TESTS
The experiments were performed to solve the problems of on-time terrain and surface surveillance using the electrooptical complex at UAV flight. The flight test on the use of electrooptic system comprised the flight on the route and automatic return using the satellite system navigation (figure 5).
The terminal point of the route
The initial point of entering the route
"Village Shabanova" reference landmark
GPS:
Latitude: 55.89 Longitude: 36.57
"Lake
Alekseevskoye" reference landmark
GPS:
Latitude: 55.88 Longitude: 36.57
The intermediate point of the route
Fig. 5. The flight path of the complex with the return to the starting point
The shots from the forward-looking camera, which fixed the landmarks were obtained within the flight test (figure 6). The images of the terrain allow to distinguish the visible landmarks and to carry out the surveillance tasks in near-real time.
b
Fig. 6. The results of the registration of images of the terrain by an optical-electronic system when approaching the first landmark (a) and upon approaching the second landmark (b)
a
Vol. 22, No. 05, 2019
Civil Aviation High Technologies
Along with this, as the images obtained show, there is a deviation from the plotted route when the GPS sensor data are being used. The UAV was supposed to fly precisely above the landmark number 1. However, this landmark was passed with the left deviation of 10-20 m (figure 6 a). During a flight with numerous landmarks the navigational error becomes significant and may result in task failure.
In order to eliminate the positioning accumulative error, it is suggested to make amendments on the electrooptical complex flight route. This problem can be solved applying automation to the results of the terrain images registration [4].
The technology assisting in the data processing automation may be the set of algorithm applying methods for image processing, analyses, contour extraction1'2 [3-5, 7-10]. The possible variant of practical application may be Canny algorithm, which is a multi-stage algorithm of image boundary processing and forming [6]. The basic algorithm stages are: image smoothing, search of gradients, non-maximum suppression, threshold filtration, hysteresis tracking. For the purpose of contour extraction for further automatic coupling to the navigational parameters, the real vision images of terrain in flight were processed using Mathcad software environment (figure 7, 8).
Sverti(X.K} := a rmvs(X)
b cols(X}
R
2
XI «Н Etack(submatrisfX,0,R - l,0,b - 1),X) XI stackfXl. submatrix(X, a - R. a - 1ДЬ- 1)) for j e 0..b - I for isR.. a - 1 + R
Zf- submatiix(Xl.i- R.i + R,j J)
N
Fig. 7. Processing of the received images by MathCad operators at the stage of smoothing the image according to the Canny algorithm
1 Premium avionics for general aviation airplanes. Official website Avidyne. Available at: http://www.avidyne.com/ (accessed 15.05.2019).
2 Universal avionic systems. Official website Avidyne. Available at: http://www.avidyne.com/ (accessed 15.05.2019).
Ovil Aviation High Technologies
Vol. 22, No. 05, 2019
Fig. 8. Processing of the received images by MathCad operators at the Canny algorithm's
double-threshold filtering stage
The results of image processing, following the given laws are shown in Figure 9.
.. Ug-jg' о 013
0&(jH>1
> iffttj «ФО СЫЕЮ Q 4 «SGCJ
л ч
(pft а* да,
Q GbW
sog® „, чоА?™®».*»
b
Fig. 9. The results of filtering and selection of contours of images of the terrain by an optical-electronic system when approaching the first landmark (a) and when approaching the second landmark (b)
a
Vol. 22, No. 05, 2019
Ovil Aviation High Technologies
The results obtained:
1. Demonstrate the need to correct and back-up the navigational data obtained from satellite systems for UAV flights.
2. Determine the approaches for automatic extraction of the most probable position of reference points, certain objects and contour extraction using Canny algorithm.
3. Help to extract contours and specific points for the particular terrain by changing the filtration threshold for automatic coupling to the navigational parameters and their correction.
4. Eliminate a number of Caddy algorithm stages. For example, for the computer system increased efficiency there is no need in gradient calculation and edge soothing. Noise, non-essential details removal and coupling the image boundaries to form contours is sufficient. The tests proved that such actions increase the processing speed in MathCad more than two times.
5. Such image registration results obtained by the surveillance electrooptical cameras with infrared and visible bands allow to distinguish objects, typical for search situations, and to make the right decisions.
ELECTROOPTICAL COMPLEX PRACTICAL APPLICATION AND CONCLUSIONS
The experiments conducted result in recommendations on the electrooptical complex use and flight route plotting for search and surveillance tasks:
1. For surveillance and search operations at daytime and at night the electrooptical complex should contain the on-board flight management cameras and surveillance cameras of multispectral bands.
2. The optimal fuselage design to house the electrooptical complex is the high-wing low weight airframe.
3. The route turning points should be clearly distinguishable in flight (river knees, road intersections, elaborate buildings etc.).
4. The first turning point (the initial point en route) should be placed next to the flight origin point.
5. The desired flight track, if possible, must avoid powerlines and other facilities with high electromagnetic field (position-radar stations, transmit/receive antennae etc.)
To monitor the terrain in near-real time for the purpose of flight incident surveillance the flight must follow the checklist of special maneuvers (figure 10).
Turn point
>--О
\
Л
Airborne electrooptical
camera field of view
injection point
Track
i t
v.-V
Ю-->
Fig. 10. Typical variant of the flight route of the complex
1. Leg - the surveillance of linear objects (bridges, straight parts of the roads, rivers);
2. Circuit path - patrolling in the given area, aerial inspection;
Civil Aviation High Technologies
Vol. 22, No. 05, 2019
3. Circle - patrolling in the given area, aerial inspection;
4. Figure eight pattern - patrolling in the given area, aerial inspection;
5. Bank-to-bank reversal - non-linear objects surveillance (non-linear parts of the road, river, etc.);
6. Area - the surveillance of the given area.
The terrain image processing and analyses, colour inversion and image upgrade may be performed applying Canny algorithm. The results of software processing show that such processing should be done using the ground equipment, and it should take significantly more time compared to the current flight image capture time. Based on the image processing results it is possible to make timely decisions on the search tasks on the ground.
REFERENCES
1. Stukalov, S.B. (2016). Perspektivnyye napravleniya postroyeniya tekhnicheskikh sistem videniya vozdushnykh sudov [Perspective directions for the construction of technical aircraft vision systems]. Sbornik materialov I mezhdunarodnoy zaochnoy nauchno-prakticheskoy konferentsii uchrezhdeniya obrazovaniya «Belorusskaya gosudarstvennaya akademiya aviatsii» [Collection of materials of the 1-st international correspondence scientific-practical conference of the educational institution "Belarusian State Academy of Aviation". Minsk, December 20-23, 2016]. Minsk: BGAA, pp. 64-69. (in Belarus)
2. Stukalov, S.B. and Skvarskiy, N.A. (2017). Problemy primeneniya bortovykh sistem vizualizatsii vertoletov pri posadke v usloviyakh ogranichennoy vidimosti [Problems of use of onboard visualization systems of helicopters during landing in conditions of limited visibility]. Sbornik materialov II mezhdunarodnoy zaochnoy nauchno-prakticheskoy konferentsii uchrezhdeniya obrazovaniya «Belorusskaya gosudarstvennaya akademiya aviatsii» [The collection of materials of the 2-nd international correspondence scientific-practical conference of the educational institution "Belarusian State Aviation Academy"]. Minsk: BGAA, pp. 101-103. (in Belarus)
3. Gorbunov, A.L. (2014). Stereoscopic augmented reality in visual interface for flight control. Aerospace science and technology, vol. 38, pp. 116-123.
4. Eremeyev, V.V., Kuznetsov, A.E. and Fedotkin, D.I. (1999). Informatsionnyye tekhno-logii koordinatnoy privyazki izobrazheniy zemnoy poverkhnosti po elektronnym kartam [Information technologies of coordinate referencing of images of the earth's surface using electronic maps]. Vychislitelnyye mashiny, kompleksy i seti [Computing machines, complexes and networks]. Mezhvuzovskiy sbornik nauchnykh trudov [Interuniversity collection of scientific papers]. Ryazan: State Radio Engineering Academy, pp. 21-25. (in Russian)
5. Bogdanova, V. (2010). Image enhancement using retinex algorithms and epitomic representation. Bulgarian Academy of sciences. Cybernetics and information technologies. Sofia, vol. 10, no. 3, pp. 10-19.
6. Gurov,V.S., Kolodko, G.N. and Loginov, A.A. (2016). Obrabotka izobrazheniy v aviatsionnykh sistemakh tekhnicheskogo zreniya in Kostyashkina L.N., Nikiforova M.B. (Ed.). [Image processing in aviation vision systems]. Moscow: Fizmatlit, 240 p.
7. Alpatov, B.A., Babayan, P.V., Kostyashkin, L.N. and Romanov, Yu.N. (2011). Image processing and analysis in vehicle-borne optical-electronic systems. Tekhnicheskoye zreniye v sistemakh upravleniya mobilnymi obyektami [Computer vision in control systems of the mobile objects]. Trudy nauchno-tekhnicheskoy konferentsii-seminara [Proceedings of the conferenceworkshop]. Moscow: KDU, iss. 4, pp. 57-62. (in Russian)
8. Gonzalez, R.C. and Woods, R.E. (2007). Digital image processing. 3rd Edition. Prentice-Hall, 976 p.
9. Kostyashkin, L.N., Babaev, S.I., Loginov, A.A. and Pavlov, O.V. (2011). Aircraft control enhanced vision systems technologies. Tekhnicheskoe zrenie v sistemakh upravleniya
Vol. 22, No. 05, 2019
Civil Aviation High Technologies
mobilnymi obyektami [Computer vision in control systems of the mobile objects]. Trudy nauchno-tekhnicheskoy konferentsii-seminara [Proceedings of the conference-workshop]. Moscow: KDU, iss. 4, pp. 45-56. (in Russian)
10. Vizilter, Yu.V. and Zheltov, S.Yu. (2011). Problems of Technical Vision in Aviation Systems. Tekhnicheskoe zrenie v sistemakh upravleniya mobilnymi obyektami [Computer vision in control systems of the mobile objects]. Trudy nauchno-tekhnicheskoy konferentsii-seminara [Proceedings of the conference-workshop]. Moscow: KDU, iss. 4, pp. 11-44. (in Russian)
INFORMATION ABOUT THE AUTHORS
Sergey B. Stukalov, Candidate of Technical Sciences, Associate Professor, Professor of the Air Transport Radio-Electronic Equipment Maintenance Chair of Moscow State Technical University of Civil Aviation, stukalov@mstuca.aero.
Dmitry S. Stukalov, Postgraduate Student of Moscow State Technical University of Civil Aviation, d.s.stukalov@mail.ru.
Vadim I. Kondrikov, Postgraduate Student of Moscow State Technical University of Civil Aviation, i.korostelina@mstuca.ru.
Roman S. Gavryushin, student of Moscow State Technical University of Civil Aviation, motojacksonhawk@gmail.com.
ОПТИКО-ЭЛЕКТРОННЫЙ КОМПЛЕКС ОПЕРАТИВНОГО ИССЛЕДОВАНИЯ ПОВЕРХНОСТИ МЕСТНОСТИ
С.Б. Стукалов1, Д.С. Стукалов1, В.И. Кондриков1, Р.С. Гаврюшин1
1 Московский государственный технический университет гражданской авиации,
г. Москва, Россия
В статье рассматриваются подходы по использованию оптико-электронных систем в беспилотных летательных аппаратах для задач оперативного обследования поверхности местности, мест и объектов летных происшествий. Представлено обоснование выбора схемы планера беспилотного летательного аппарата и конструкции оптико-электронного комплекса. Предложено использование схемы планера типа высокоплан с облегченной конструкцией. Для проведения исследований разработан и создан при помощи станка с числовым программным управлением беспилотный летательный аппарат с высокими показателями аэродинамического качества, оборудованный оптико-электронной системой. Разработаны конструкции бортового и наземного оборудования исследовательского оборудования комплекса. Предложено использование полетных курсовой и обзорной боковой оптико-электронных камер, а также исследовательских оптико-электронных камер нижней полусферы видимого и инфракрасного диапазонов, что обеспечивает видение в дневное и ночное время. Для выполнения просмотра и исследования поверхности местности используются две камеры обзора нижней полусферы инфракрасного и видимого диапазонов, встроенные снизу в фюзеляж беспилотного летательного аппарата. Проведены летные эксперименты по применению оптико-электронного комплекса с полетом по маршруту по системе спутниковой навигации и автоматическим возвратом. Установлено, что для точного выхода на ориентиры маршрута требуется коррекция навигационных данных. Устранение накапливаемой погрешности позиционирования предлагается выполнить автоматизированно, путем регистрации изображений местности оптико-электронной системой с выделением их контуров и ориентиров. Представлены результаты обработки изображений по алгоритму Canny, полученных в ходе летных экспериментов. Даны рекомендации по практическому использованию алгоритма обработки изображений с учетом требуемого быстродействия. Для решения задач оперативного просмотра и исследования местности проведено исследование различных маршрутов полета. На основании проведенных экспериментов даны рекомендации по применению комплекса и построению маршрута полета в исследовательских задачах оперативного поискового характера.
Ключевые слова: оптико-электронный комплекс, беспилотные летательные аппараты (БПЛА), летный эксперимент, коррекция навигационных параметров, обработка изображений.
Civil Aviation High Technologies
Vol. 22, No. 05, 2019
СПИСОК ЛИТЕРАТУРЫ
1. Стукалов С.Б. Перспективные направления построения технических систем видения воздушных судов // Сборник материалов I международной заочной научно-практической конференции учреждения образования «Белорусская государственная академия авиации». 20-23 декабря. Минск: БГАА, 2016. С. 64-68.
2. Стукалов С.Б., Скварский Н.А. Проблемы применения бортовых систем визуализации вертолетов при посадке в условиях ограниченной видимости // Сборник материалов II международной заочной научно-практической конференции учреждения образования «Белорусская государственная академия авиации». 9-10 ноября. Минск: БГАА, 2017. С.101-103.
3. Gorbunov A.L. Stereoscopic augmented reality in visual interface for flight control // Aerospace science and technology. 2014. Vol. 38. Pp. 116-123.
4. Еремеев В.В., Кузнецов А.Е., Федоткин Д.И. Информационные технологии координатной привязки изображений земной поверхности по электронным картам // Вычислительные машины, комплексы и сети: межвузовский сборник научных трудов. Рязань: Рязанская государственная радиотехническая академия. 1999. С. 21-25.
5. Bogdanova V. Image enhancement using retinex algoritms epitomic representation // Bulgarian Academy of sciences, Cybernetics and information technologies. Sofia. 2010. Vol. 10, No. 3. Pp. 10-19.
6. Гуров В.С. Обработка изображений в авиационных системах технического зрения / Под ред. Л.Н. Костяшкина, М.Б. Никифорова. М.: Физматлит, 2016. 260 с.
7. Алпатов Б.А. Обработка и анализ изображений в бортовых оптико-электронных системах / Б.А. Алпатов, П.В. Бабаян, Л.Н. Костяшкин, Ю.Н. Романов // Техническое зрение в системах управления мобильными объектами: Труды научно-технической конференции-семинара. М.: КДУ, 2011. Вып. 4. С. 57-62.
8. Gonzalez, R.C. and Woods, R.E. Digital image processing (2002), 3rd Edition. Prentice-Hall, 2007. 976 p.
9. Костяшкин Л.Н. Технологии систем улучшенного синтезированного зрения для управления летательными аппаратами / Л.Н. Костяшкин, С.И. Бабаев, А.А. Логинов, О.В. Павлов // Техническое зрение в системах управления мобильными объектами: Труды научно-технической конференции-семинара. М.: КДУ, 2011. Вып. 4. С.45-56.
10. Визильтер Ю.В., Желтов С.Ю. Проблемы технического зрения в современных авиационных системах // Техническое зрение в системах управления мобильными объекта-ми-2010: Труды научно-технической конференции-семинара. М.: КДУ, 2011. Вып. 4. С. 11-44.
СВЕДЕНИЯ ОБ АВТОРАХ
Стукалов Сергей Борисович, кандидат технических наук, доцент, профессор кафедры технической эксплуатации радиоэлектронного оборудования воздушного транспорта МГТУ ГА, stukalov@mstuca.aero.
Стукалов Дмитрий Сергеевич, аспирант МГТУ ГА, d.s.stukalov@ mail.ru.
Кондриков Вадим Игоревич, аспирант МГТУ ГА, i.korostelina@mstuca.ru.
Гаврюшин Роман Сергеевич, студент МГТУ ГА, motojacksonhawk@gmail.com.
Поступила в редакцию 17.05.2019 Received 17.05.2019
Принята в печать 24.09.2019 Accepted for publication 24.09.2019
