CC BY
16TECHNICAL SCIENCES
COMPLEX ALGORITHM OF DIGITAL IMAGE SEGMENTATION AND CLASSIFICATION OF
VISUAL OBJECTS
Nedashkivskyi B.
lecturer The Open International University of Human Development 'Ukraine'
https://orcid. org/0000-0002-9886-2674
ABSTRACT
The article presents a complex algorithm for digital image segmentation and classification of visual objects. The classification of segmentation methods is given and the principles of clustering are revealed. It is noted that the main method of data clustering can be divided into hierarchical and sectional clustering, edge segmentation classification is edge-based segmentation, this type of segmentation usually uses edge detection or edge concept. The principles of implementation of the segmentation algorithm by region / region are determined. It is emphasized that the basis of the segmentation algorithm by region / area is the assumption that neighboring pixels within one area have a similar value, the general procedure is to compare one pixel with neighbors. It is emphasized that if the similarity criterion is met, the pixel can be set as belonging to the cluster as one or more of its neighbors, the choice of similarity criterion is important, and the results are affected by the presence of noise. The types of segmentation algorithm by region / region are described - the algorithm of growing region and block segmentation. It is noted that the first performs image segmentation by examining adjacent pixels of a set of points and determines whether pixels could be classified into a cluster of the main point or not, and the second is analogous to the first with the difference that there is no need to select point criteria, points are generated automatically. The stages of algorithms for implementing each of the described approaches to segmentation and their advantages and disadvantages are presented. A complex algorithm of digital image segmentation is formed, which is presented in the form of a block diagram. It is emphasized that an alternative approach to improving the accuracy of recognition is to use information about the structure of relationships between categories of regions / objects. The directions of further researches which are based on software development of complex algorithm are outlined.
Keywords: complex algorithm, segmentation, digital format, image, classification, visual object, graphic
data.
Problem statement. Digital visual information processing now encompasses various types of infocom-munication applications and expands their range of action. This includes both traditional applications (voice, industrial, security television) and relatively new applications (videoconferencing, technical vision, digital cinema, high and ultra-high-definition TV, 3D TV).
According to the modern classification proposed by V.A. Laver and A.N. Levchuk, digital image processing includes the following main areas [1]:
- image correction, their "dissection", i.e., splitting them into parts by digital means, modifying these parts and reassembling them;
- assessment of image parameters in order to control the quality of image transmission and reception;
- transformation and coding of images for storage and transmission via communication channels;
- computer graphics, as well as visualization of information, i.e., presentation of data arrays in the form of various images, is very effective, as it facilitates solution of many tasks, complicated precisely by their ab-stractness.
If there is no possibility to process the whole image it is subjected to segmentation. Image segmentation consists in the classification or clustering of the image into several parts (areas) according to the image features, for example, pixel value or frequency response. Today there are many image segmentation algorithms that are widely used in science and everyday life. According to their segmentation method, it is possible to
classify them by regional segmentation, data clustering and segmentation by base point.
Analysis of recent research and publications. Digital image processing is a field of research which covers various areas of life of modern society. The principles of digital image recognition, the stages of its processing and classification have been studied over the years by many foreign and domestic scientists.
In [2] a review and analysis of existing methods of localization on digital image was carried out. Ya.I. Shedlovskay [3] disclosed the principles of decoding and carried out the analysis of multidimensional photo-grammetric images of high spatial sparsity.
T.V. Kovalenko [4] proposed the description of methods and information technology for segmentation of texture areas of images in airborne monitoring systems.
In the field of recognition of aerospace images, it is worth noting the work of V.V. Pustovarov [5]. The author has developed a unique information system of decision support for the recognition of buildings on satellite and aerial images. The study describes in detail the principles of aerial photo segmentation.
Among the foreign authors it is worth noting such works as: Chen J., Deng Y., Bai G., Su G. [6], Ga-nakwar D.G., Kadam V.K. [7], Taigman Y., Yang M., Ranzato M., L. Wolf. [8], Menezes J., Poojary N. [9], Paudyal P., Battisti F., Carli M. [10], Feng Z., Huber P., Kittler J., Christmas W., Wu X.J. [11], Shiga M., Muto S. [12], Azawi N., Gauch J. [13], Ren Y., Tang L. [14] and others.
However, given the described scientific advances on the topic, the research questions of the complex algorithm for segmentation of digital images and classification of visual objects remain open and require detailed elaboration.
Task definition. Give a comprehensive algorithm for digital image segmentation and visual object classification.
Presentation of the basic material of the study.
Segmentation of digital images is used in many applications, given the scope of applications, the availability of different types of segmentation algorithms is indisputable. Regional segmentation includes region growing algorithms, JSEG and fast scanning algorithm. All of them extend each region pixel by pixel based on their pixel value or quantized value, so that each cluster has a high positional relationship. For data clustering, their concept is based on the whole image and takes into account the distance between each data. A characteristic of data clustering is that each pixel in the cluster is certainly not a linkage. The basic method of data clustering can be divided into hierarchical and sectional clustering. Edge-based segmentation is an edge-based segmentation. This type of segmentation typically applies edge detection or the notion of an edge. The watershed algorithm is typical, but it is always a problem of redundant segmentation, so it is proposed to use markers to improve the watershed algorithm by smoothing and selecting markers.
Regional methods mainly rely on the assumption that neighboring pixels within the same area have a similar value. The general procedure is to compare one pixel with its neighbors. If the similarity criterion is satisfied, the pixel can be established as belonging to a cluster as one or more of its neighbors. The choice of the similarity criterion is weighty, and the results are affected by the presence of noise.
The growing region/area (GR) algorithm is one of the simplest region/area-based segmentation methods. It performs image segmentation by examining neighboring pixels of a set of points known as principal points, and determines whether the pixels could be classified to the principal point cluster or not [3]. The algorithm procedure is as follows:
Step 1. We start with a number of starting points, which have been grouped into clusters, which are called c1¡cz C„ ■ And the positions of the starting points are
set as p1,pZl...,p3-
Step 2. To calculate the difference between the pixel values of the starting point p¡ and the neighboring
points, if the difference is less than the threshold (criterion) that we define, the neighboring point can be clas-sifiedinto where ¿ = 1,2,...,n
Step 3. Calculate the boundary q and set these boundary points as new points p_ In addition, the average pixel values c, should be listed according to.
Step 4: Repeat Steps 2 and 3 until all the pixels in the illustration are distributed into the appropriate cluster.
The threshold value is set by the user, and it is usually based on intensity, gray level or white level values. The regions are chosen to be as uniform as possible.
There is no doubt that each of the GR segmentation regions has high color similarity and no fragmentation problems. However, it still has two disadvantages, the initial 3 seed points and the laboriousness. The initial point problem means that different sets of initial points cause different segmentation results. This problem reduces the stability of segmentation results from the same image. The other problem is labor intensive because the GR takes a long time to compute, and this is a serious problem of the algorithm.
The Block Segmentation (BS) algorithm is a derivative of the growing region. Their difference is that no explicit point selection is needed. During the segmentation process, points can be generated automatically. Thus, this method can perform fully automatic segmentation with the added advantage of reliability since it is region-based segmentation. The WB steps are as follows.
Step 1: The initialization process is a cluster Ci,
containing one image pixel, and the working state of the process consists of a set of identified clusters,
Step 2. We define the set of all unsigned pixels that borders at least one of these clusters as:
S= \x£ [Jc, A3fc:jVMnCt
l, J—1 1
where - current neighboring pixels of the pointLet s ~ be the measure of the difference
where g(X) denotes the pixel value of the point x, and i - is the cluster index, such that ¿v(x) intersects the
Step 3. Choose a point z e 5 and a set q. where
j e [l,n] such that
<S(z,C;) — min
If c.) less than a predetermined threshold £, the pixel is clustered in c_. Otherwise, it is necessary to select the most significant such cluster q in this way C = arg m iii{<5 (z, Cj)}
If s(Z: c) < t- it is possible to distribute the pixel to
c. If neither of the two conditions holds, it is obvious
that the pixel is essentially of all the clusters detected so far, such that a new cluster cn + 1 is generated and
initialized by the point z.
Step 4. After a pixel has been assigned to a cluster, the average value of the cluster pixel should be updated.
Step 5: Repeat steps 2-4 until all pixels are assigned to the cluster.
The main purpose of partitioning and merging regions is to distinguish the homogeneity of the image [5]. His concept is based on quadrangles, which means that each tree node has four descendants, and the root of the tree corresponds to the whole image. In addition,
each node represents the division of a node into four sons' nodes.
Let a represent the whole region of the image and define a predicate P. The goal is that if P(P) = false -divide the image p into quadrants. If for any quadrant p = false- divide that quadrant into subquadrants. Before that for any region /¡., p(Rj)= true- After the
splitting process, the fusion process is to merge two adjacent regions R_ i if P(R. u = true-
Implementation algorithm:
Step 1. Partitioning steps: For any area that
P(rj = false we partition it into four non-intersecting quadrants.
Step 2: Merge steps If further splitting is not possible, merge any neighboring areas R i for which
p{rt u rk) - true-
Step 3: Stop only if no further merging is possible.
Pros:
- Images can be progressively partitioned according to the desired resolution, as the number of partition levels is determined initially by the user.
- Criteria such as average value or variance value of a pixel segment are set by the user.
- Merge criteria may differ from the split criteria.
Cons:
The possibility of forming block segments. The block segment problem can be reduced by splitting to a higher level, but the trade-off is that computation time will increase.
The unsupervised segmentation algorithm for color texture areas in images and video (NSOKTZV) consists of segmenting images and video recordings into homogeneous areas of color texture.
The concept of the NSOKTZV algorithm is to divide the segmentation process into two parts, color quantization and spatial segmentation. Color quantization quantifies the colors of an image into several representative classes that can differentiate image regions. The quantization process is performed in color space without regard to the spatial distribution of colors. The corresponding color class labels replace the original pixel values and then create a map of the image classes. In the second part, spatial segmentation is performed on the class map instead of corresponding color pixels. The advantage of this separation is that, accordingly, color similarity analysis and color distribution are more traceable than their complete completion at the same time. This quantization method quantifies a set of image pixels to a single color, called a color class. The colors of the image pixels are then replaced by the corresponding color class labels, and the newly established label image is called a class map. A class map can be seen as a special kind of texture composition.
Let z - the set of all y data points on the map of
the class and z _ (X|y), where z e 7 and (x - the pixel position of the image. The average m is
Let^ classify into classes c-Zj.i - l,...,c- Letm.
- the average value of data points jy. of the class zr Allows
5 - the total variance of points belonging to the same class. We define
The value j is related to the degree of distribution
of color classes. The more even the distribution of color
classes, the lower the value j. n the other hand, if the
image consists of several homogeneous color areas and the color classes are more separated from each other, the value j is high. The motivation for the definition <
comes from Fisher's multiclass linear discriminant [8], but for an arbitrary nonlinear class distribution.
After implementing the operational method j two
values need to be applied in the NSOKTZV algorithm:
the average J and the current value j The average value
j is used as a criterion for evaluating the effectiveness of the segmentation area. The average value j is defined as
where ¡k - j. assessed for the region - the
number of points (pixels) in the area k - total number
of points on the class map. For a fixed number of regions, "better" segmentation usually has a lower value j. The low value J represents each segmented region
containing several evenly spaced color class labels. shows two examples of a segmented class map and their value J.
The operation of the local value ; bases takes place
on the local window. The size of the local window determines the size of the areas of the image that can be detected. Small windows are useful for localizing intensity/color edges, while large windows are useful for detecting texture boundaries. Also, the higher the local value j, the more likely the corresponding pixel is near
an area. The NSOKTZV algorithm uses several scales to segment the image. The main window at the smallest scale is a 9x9 window without corners. The larger window size (larger scale) is obtained by doubling the size of the straight scale.
According to the characteristics of the values ¡. the
modified method of region growing can be applied to image segmentation.
Figure 1 - Comprehensive algorithm for digital image segmentation
The algorithm starts segmentation on a large scale. It then repeats the same process on new segmented regions at the next lower scale. After the final segmentation on the small scale is completed, the region merge operation follows the region growth to obtain the final segmentation result. A block diagram of the steps is shown in Fig. 1.
Data clustering is one of the methods widely used in image segmentation and classification. The basic concept of data clustering is to use the centroid to represent each cluster and based on the similarity to the centroid of the cluster for classification. According to the characteristics of the clustering algorithm we can roughly divide into "hierarchical" and "sectional" clustering. Except for these two classes, the average shift algorithm is also part of data clustering, and its concept is based on density estimation.
Conclusions and prospects for further research. The paper presents a complex algorithm for digital image segmentation and classification of visual objects. The essence of the basic segmentation algorithms is defined and it is proved that the best results for tasks of this type are demonstrated by two classes of algorithms. The first one is based on using Fisher vectors to describe images and linear machine of reference vectors as a classifier. The second class of algorithms uses pixel intensities directly to describe images, using the correlation method as the classifier. An alternative approach to improve recognition accuracy is the use of information on the structure of links between categories of regions/objects.
Prospects for further research are based on the development of software to implement a comprehensive
algorithm for segmentation of digital images and classification of visual objects using an object-oriented programming language.
References
1. Laver V.A., Levchuk A.N. Image processing textbook / V.A. Laver, A.N. Lev. - Uzhgorod: Publishing house LLC "Autodor - SHARK", 2021. - 51 p.
2. Kolomiets S. P. Analysis of existing methods of localization on a digital image // International Scientific Journal "Internauka". — 2018. — №17. https://doi.org/10.25313/2520-2057-2018-17-4209
3. Shedlovskaya Ya.I. Decoding and analysis of multidimensional photogrammetric images of high spatial sparsity. - On the rights of manuscript. Dissertation for the degree of Candidate of Technical Sciences in the specialty 05.01.01 "Applied geometry, engineering graphics". - Bogdan Khmelnitsky Melitopol State Pedagogical University, Melitopol, 2021. - 184 p.
4. Kovalenko T.V. Methods and Information Technologies for Segmentation of Image Texture Areas in Air Monitoring Systems. - Manuscript. Dissertation for the degree of Candidate of Technical Sciences in the specialty 05.13.06 "Information technologies". - State
University of Telecommunications. Kyiv, 2018. - 233 p.
5. Pustovarov V.V. Information technology for developing a decision support system for the recognition of buildings on space and aerial photographs. -Qualification scientific work on the rights of manuscript. Dissertation for the degree of Candidate of Technical Sciences (Ph.D.) on the specialty 05.13.06 "Information Technology". - Cherkassy State Technological University, Cherkassy, 2021. - 161 p.
6. Chen J., Deng Y., Bai G., Su G. Face Image quality assessment based on learning to rank // IEEE Signal Processing Letters, 2015. V. 22, № 1. P. 90-94.
7. Ganakwar D.G., Kadam V.K. (2019). Face Detection Using Boosted Cascade of Simple Feature. 2019 International Conference on Recent Advances in Energy-efficient Computing and Communication (ICRAECC).
doi:10.1109/icraecc43874.2019.8994977.
8. Taigman Y., Yang M., Ranzato M., L. Wolf. Deepface: Closing the gap to human-level performance in face verification // Computer Vision and Pattern Recognition, 2014. P. 1701-1708.
9. Menezes J., Poojary N. (2019). Hyperspectral image Data Classification with Refined Spectral-Spatial features based on Stacked Autoencoder approach. Recent Patents on Engineering, 13. doi: 10.2174/187221211366619091114 1616.
10. Paudyal P., Battisti F., Carli M. A study on the effects of quality of service parameters on perceived video quality // 5-th European Workshop on Visual Information Processing, 2014. P. 1-6.
11. Feng Z., Huber P., Kittler J., Christmas W., Wu X.J. Random cascadedregression copse for robust facial landmark detection // IEEE Signal Processing Letters, 2015. V. 22, № 1. P. 76-80.
12. Shiga M., Muto S. (2019). Non-negative Matrix Factorization and Its Extensions for Spectral Image Data Analysis. E-Journal of Surface Science and Nan-otechnology. 17 (1). 148-154. doi:10.1380/ejssnt.2019.148.
13. Azawi N., Gauch J. (2019). Ransac Based Motion Compensated Restoration for Colonoscopy Images. Signal & Image Processing: An International Journal. 10 (4). 9-16. doi:10.5121/sipij.2019.10402.
14. Ren Y., Tang L. (2019). A nonconvex and nonsmooth anisotropic total variation model for image noise and blur removal. Multimedia Tools and Applications, 79 (1-2), 1445-1473. doi:10.1007/s11042-019-08179-8.
