CC BY
12
56_ОБРАБОТКА ИНФОРМАЦИИ И ПРИНЯТИЕ РЕШЕНИЙ
UDC 004.942 DOI: 10.21122/2309-4923-2022-4-56-59
RYBAK V. A1, AMRO RABIA2
SOFTWARE TOOL FOR PROCESSING AND SELECTING OBJECTS IN INFRARED
IMAGES
1Belarusian State University of Informatics and Radioelectronics 2Belarusian National Technical University
The article describes the scientific problem ofprocessing images obtained in the infrared range. Methods for filtering such images in order to reduce noise are presented. Methods for optimal smoothing are described. A software tool has been developed that allows you to select the desired objects on the processed images.
Keywords: pattern recognition; infrared imaging; limited visibility.
Introduction
At present, the processing of thermal imaging (thermal, thermographic) images is an important area of application of modern computer technology. It finds application in various fields of science and technology: processing information from satellites that scan the earth>s surface in order to compile maps of the area, using several frequency ranges, including both infrared and visible ranges, which allows you to achieve better results; data processing in systems for detecting and targeting objects and their identification; control of compliance with the temperature parameters of technological processes and thermal modes of operation of various electronic systems; processing of human biological parameters in medical research.
The task of any (including thermal imaging) imaging system is to create a sharp, clean image, free from noise and distortion. This always presents certain problems. First, every real imaging system has some limited capabilities; the impulse response of a real system has a finite duration, which leads to an inevitable decrease in resolution. If it is necessary to highlight important details in the image, the size of which is close to the duration of the impulse response, then it is necessary to increase the resolution. So, for example, from the thermal imaging system (thermal imagers, IR receivers) of aircraft and helicopters, images of sufficiently good quality come, but the operators studying them always try to see objects (for example, tanks, people) distorted due to limited resolution ka-mer. Secondly, images can be corrupted due to a certain set of shooting circumstances. You can take every precaution to get high-quality images, but some of them will be spoiled, either due to the movement of the subject or camera, or due to poor focus, etc. Among the low-quality images are always so important or so rare that it is worth trying to improve their quality. The elimination of distortions refers to the tasks of restoration (sharpening) of images [1-3].
The development of effective representations and models of images and methods for their digital processing is given much attention in the works of foreign and domestic scientists and specialists. Nevertheless, the methods of digital image processing developed today, used in television, video technology, as applied to thermal imaging systems, need to be significantly improved, which is associated with a relatively large heterogeneity of parameters and characteristics of thermal imagers, non-linearity of their characteristics, high noise level and low image contrast. At the same time, there are
practically no data when compressing thermal imaging images distorted by multiplicative noise. The need to correct and weaken the influence of these factors on the thermal image, compression during their transmission over communication channels determines the relevance of the topic of the graduation project.
The aim of the work is to improve existing and develop new methods, algorithms and techniques for digital processing, which improve the quality of a thermal image transmitted over communication channels.
Analysis of modern methods of digital processing and correction of thermal imaging images
The main way to implement the thermal imaging method is to create hardware that converts the temperature distribution, or infrared radiation, into a visible image. Realization of the capabilities of the thermal imaging method, which provides both the registration of various objects and the identification of their internal defects, and the effective solution of the problem of «night vision», the detection of hidden or camouflaged objects, or the implementation of search activities in adverse weather conditions, led to the creation of a wide range of thermal imaging equipment: portable, mobile, stationary. Thus, thermal imaging devices are capable of providing a long range of vision at any time of the day, through any camouflage, and even with a slightly reduced atmospheric transparency: in fog, rain, snowfall, dust and smoke. It is known that water vapor and carbon dioxide intensely absorb infrared spectrum waves, and this noticeably affects the sensitivity of devices. There are many receivers capable of receiving thermal waves that provide the necessary technical characteristics both at normal temperature and at deep cooling - from temperatures of 220 ... 240 K to cryogenic temperatures (below 120 K), i.e. 30...80 K. An example of the operation of a thermal imager is given in Figure 1.
Figure 1 - An example of an image generated by thermal imaging systems
DATA PROCESSING AND DECISION-MAKING
57
In recent years, the development of thermal imaging technology has proceeded mainly along the path of using uncooled multi-element MPI, the physical characteristics of which are very high and are practically not inferior to cooling systems. Modern thermal imaging systems (TPS) have small weight and size characteristics and power consumption, provide silent operation and high quality thermal imaging, a wide dynamic range when operating in the broadcast television standard, digital processing in real time, communication with a computer, etc.
Most solid state IR array receivers use cadmium mercury telluride GdHgTe (KPT, cadmium mercury tellurium). KPT is sensitive not only in the region of 8 ... 14 microns, but also in all practically important IR ranges. Therefore, the work is aimed at modeling and improving the operation of devices in the IR range of 8 ... 14 ^m, based on the use of MCT receivers. By the nature of the reception of radiant energy, the method is integral, i.e. the total radiation flux is perceived in the entire working spectral interval. This is the difference between most thermal imagers and spectral research methods, in which radiation with a certain wavelength or in a very narrow spectral range is recorded [4]. Thermal solid-state radiation receivers use the effect of changing the electrical properties of the material (resistance, inductance, capacitance) when its temperature changes due to heating when external thermal radiation is absorbed.
In thermal imaging detection of small-sized target objects, the problem of ensuring the highest possible signal-to-noise ratio is solved through the use of optimal filtering, accompanied by the destruction of an integral image of objects of finite sizes. The main problem in thermal direction finding is associated with ensuring the required probability of correct detection of small targets at the maximum range in the presence of a combination of internal noise and natural background and organized anthropogenic interference.
Developing methods for digital processing and correction of thermal vision images
Computer image processing is possible after converting the image signal from continuous form to digital form. The efficiency of processing depends on the adequacy of the model describing the image, which is necessary for the development of processing algorithms. In this case, it is necessary to take into account the influence of the transmitting and receiving systems and the communication channel on the signal. The image model represents a system of functions that describe the essential characteristics of the image: a brightness function that reflects the change in brightness in the image plane, spatial spectra and spectral intensities of images, and autocorrelation functions.
When designing and creating thermal imaging systems, an important role is played by modeling the structure of thermal imaging images. The brightness of thermal imaging images depends both on the temperature distribution over the surface of the observed object, and on the emissivity and orientation of the viewed elements of its surface - its shape. In addition, the quality of a thermal imaging image depends on the transfer characteristics
and all links of the thermal imager. In modeling the structure of thermal imaging images, we will take the process of generating video signals of the receiver, the power of which is proportional to the thermal radiation flux of the object for the entire thermal imaging frame, which contains L lines with N elements per line. Video signal level U(N, L) of the frame decomposition element.
Since the position of the object is not known in advance, the thermal imaging system solves the difficult problem of detecting a random signal. In the process of solving it, the system must choose between two hypotheses:
- there is only noise;
- in addition to noise, there is a thermal imaging
facility.
Traditionally, such a problem is solved by searching for a thermal imaging object according to its previously known characteristics. Currently, for thermal imagers, the amplitude method for detecting a thermal imaging object in a thermal imaging image is used. The image obtained on the display is given in accordance with the video signal received from the output of the matrix radiation receivers. Each pixel is assigned a limited number of discrete brightness levels - quantization levels. The more quantization levels, the higher the image quality. The threshold method is used to detect an object. The choice of the threshold is made taking into account the fact that the brightness of the emission noise of the object differ in amplitude. The response threshold is selected automatically according to the average brightness level of the background noise. That is, the areas of the image, where the brightness is above the set threshold, are considered a thermal imaging object, the rest are considered noise.
In fact, object detection begins with preliminary mathematical processing and ends with the division of image array elements into two classes by comparing the obtained results with an adaptive threshold. Those elements of the array that have the properties of noise are assigned zero values, and the remaining ones are filled with ones. As a result of these actions, a binary decision array is created. The binary image displayed on the display makes it possible to unambiguously make a decision about the presence of a thermal imaging object. Thus, an array FN,M is finally formed, which can be used to create a binary image on the display. By the numbers of rows and columns of pixels in which single values are located, the spatial position of the object in the image is determined.
The essence of the developed method for detecting an object against a background of noise is associated with a nonlinear operation, which increases the likelihood of making a decision about the presence of the desired thermal imaging object in the field of view of the thermal imager. The result of a non-linear operation, raising to the power of 5 the ratio of the element>s brightness value to the average value of the frame>s brightness, will be obtaining an array of nonlinear filtering. Each element of the array contains values that can be less than or greater than 1. Those elements whose values are less than 1 correspond to noise. Cells with a value greater than 1 may contain an element of a thermal imaging object. It is possible to detect a cell
58
ОБРАБОТКА ИНФОРМАЦИИ И ПРИНЯТИЕ РЕШЕНИЙ
corresponding in magnitude to the radiation of a thermal imaging object only by comparing the values with the adaptive decision threshold. This ends the preliminary mathematical processing of the digital image array and thresholding begins.
Based on the formulated requirements for the task, a decision was made on the need to develop a nonlinear low-frequency filter based on the "core-signal" principle, the core of which will be formed taking into account the features of low-contrast noisy images and depend on additional parameters that allow influencing
the filtering result.
The simplest image noise reduction algorithms are linear. This means that each pixel of the processed image is obtained by a linear combination of several pixels of the original image. A linear combination of any quantities is the sum of these quantities, each of which is multiplied by its own constant factor. In this case, for each pixel, neighboring pixels are analyzed, which are located in a certain rectangular window around this pixel (Figure 2a).
Figure 2 - Noise suppression by the linear averaging method: a - the usual averaging window; b - window with a weighted sum of adjacent pixels
Instead of the arithmetic mean of neighboring pixels, you can take their weighted sum, where the weight coefficient of each neighboring pixel depends either on the distance in pixels from it to the central pixel, or on the difference in their values. These algorithms are simple, but they do not always give good results. In this paper, a modification of this method is considered. In it, as the value of the central pixel, you can also take the weighted sum of neighboring pixels, only take the neighbors not in a row, but after one or two pixels (Figure 2.b). With this approach, it is possible to suppress low-frequency noise, which is more noticeable to the eye than high-frequency noise. Similarly, this method can be applied in the time domain, only the averaging will be done already between adjacent frames, and the window will be taken accordingly in time (that is, each pixel will be averaged over pixels located in the same position in adjacent frames).
To prevent the occurrence of halos around moving objects in a thermal imaging image, algorithms for determining the movement of objects are built into
the time filters. In this case, two options are possible: simple motion detection (pixels in moving blocks simply remain unchanged, and noise along moving objects is not suppressed) or building a compensated previous and/ or next frame and mixing the current one with it. In the latter case, motion compensation must be performed qualitatively, otherwise there will be artifacts in place of incorrectly found blocks [5].
Interface of the developed program
An easy-to-use graphical application has been developed for pattern recognition in infrared images. It was customary to choose people and animals as recognizable images. The downloaded image appears on the left side of the main window. Further, when you click the «Detect Objects» button, an image appears on the right side of the screen with recognized and selected objects and assigned them to one of the Animal or Person classes. The number of recognized objects is written above the Detect Objects button (Figure 3).
Figure 3 - The result of the program
DATA PROCESSING AND DECISION-MAKING
59
Conclusion
In the presented work, various methods for processing thermal images, both known and their new modifications for a specific task, as well as new methods that are applied to specific images, are investigated. In general, the content of the work is applied, so most of the theoretical results are supported by computational experiments, the results of which not only served as an illustration or test of the theory, but often gave impetus and were the source material for further research. Based on the results of the conducted research, the following
conclusions can be drawn.
- Methods for selecting the contour of an object with preservation of the boundaries of the regions and recursive restoration of the internal structure are proposed.
- A method for selecting and recognizing image objects has been created.
The scope of the developed tools is the search and identification of objects in conditions of insufficient visibility (the presence of people at night in the forest, in the mountains, in fog, etc.)
ЛИТЕРАТУРА
1. Мирошников ММ Теоретические основы оштолектронных устройств. - Л.: Машиностроение, 1983. 696 с.
2. Иванов В.П., Курт В.И., Овсянников В.А., Филиппов В.Л. Моделирование и оценка современных тепловизион-ных приборов. Казань, ФНПЦ НПО ГИПО, 2006. 594 с.
3. Тарасов В.В., Якушенков Ю.Г. Инфракрасные системы «смотрящего» типа. - М.: Логос, 2004. 444 с.
4. Калинкина Д.А., Ватолин Д.С. Проблема подавления шумов на изображениях и видео и различные подходы к ее решению. Компьютерная графика и мультимедиа. Проблема. 3(2)/2005. Интернет-ресурс: http://cgm.computergraphics.ru/ content/view/74.
5. Фриден Б. Улучшение и восстановление изображений. Суббота. «Обработка изображений и цифровая фильтрация». — М.: «Мир», 1979. — 320 с.
REFERENCES
1. Miroshnikov MM Theoretical foundations of optoelectronic devices. - L.: Mashinostroenie, 1983. 696 p.
2. Ivanov VP., Kurt V.I., Ovsyannikov V.A., Filippov V.L. Modeling and evaluation of modern thermal imaging devices. Kazan, FNPTs NPO GIPO, 2006. 594 p.
3. Tarasov V.V., Yakushenkov Yu.G. Infrared systems of «looking» type. - M.: Logos, 2004. 444 p.
4. Kalinkina D.A., Vatolin D.S. The problem of noise suppression in images and videos and various approaches to its solution. Computer graphics and multimedia. Issue. 3(2)/2005. Internet resource: http://cgm.computergraphics.ru/content/view/74.
5. Frieden B. Improving and restoring images. Sat. «Image processing and digital filtering». - M .: «Mir», 1979.-320 p.
Рыбак Виктор Александрович, проректор по учебной работе Белорусского государственного университета информатики и радиоэлектроники, кандидат технических наук, доцент.
Rybak Viktor Aleksandrovich, vice-Rector for Academic Affairs of the Belarasian State University of Informatics and Radioelectronics, Candidate of Technical Sciences, Associate Professor.
E-mail: V.Rybak@bsuir.by
Амро Рабиа, аспирантка Белорусского национального технического университета. Amro Rabia, post-graduate student of the Belarusian National Technical University.
E-mai: mido@bntu.by
