CC BY
27UDC 004.932 10.23947/2587-8999-2017-2-185-193
Building detection in remote sensing images using a digital surface model*
A.V. Dunaeva, F.A. Kornilov**
N.N. Krasovskii Institute of Mathematics and Mechanics of the Ural Branch of the Russian Academy of Sciences, Yekaterinburg, Russia
Ural Federal University named after the first President of Russia B.N.Yeltsin, Yekaterinburg, Russia
The paper considers an approach to detection of buildings and structures in the satellite imagery. The proposed method performs the extraction of high objects in a digital surface model and then improves the recognition accuracy using the segmentation of spectral information. The results of the quality comparison of the proposed approach with using different image segmentation algorithms are presented.
Keywords: satellite imagery; segmentation; building extraction; digital surface model.
Introduction. The analysis of satellite images is an extremely time consuming process, which makes it urgent to develop methods of automatically extracting and interpreting the information required in a particular task. One of such tasks is the detection of buildings and structures in satellite images of the Earth's surface that occurs in such fields as cartography, navigation of aircrafts and others. There is a wide variety of approaches for solving it, for example based on the analysis of textures and image contours [1], or using image recognition algorithms [2, 3].
Variability of the shape and appearance of structures does not allow solving the problem of their detection in satellite images with the required accuracy at the present time. However, the use of the heights of objects in the scene can lead to a significant improvement of the recognition quality. In this paper, we investigate the method of combining spectral and height information, which uses the texture segmentation of images to refine the boundaries of objects found from the height map. The proposed algorithm was tested on the orthorectified four-channel satellite image of the Earth's surface (red, blue, green, near-infrared channels), as well as height data in each pixel (height map) obtained from a pair of satellite images using a stereo matching algorithm [4]. Figures 1, 2 show the examples of the initial data.
This paper is organized as follows. Section 1 is devoted to a review of image segmentation algorithms. Section 2 provides the description of the method for extracting high objects from a height map. This information is subsequently used to classify regions obtained by segmentation algorithms.
* The research is done with the support from the comprehensive program of the FSR UrB, Project No. 15-16-1-14.
** E-mail: maryanova.av@yandex.ru, kornilovfa@imm.uran.ru.
Section 3 compares the quality of the building detection method using the segmentation algorithms described in section 1.
1. Image segmentation algorithms. Segmentation is used to pre-process images in many computer vision problems, since it allows to analyze homogeneous and holistic regions for a certain characteristic instead of separate pixels of an image. The characteristics used to split the image can be very different: color, texture, etc. The variety of approaches to solving the segmentation problem can be divided into the following groups: clustering (Mean Shift [5]), algorithms on graphs (Normalized Cuts [6], SWA [7]), region splitting and merging (gPb-owt-ucm [8], MCG [9]), algorithms using artificial neural networks (COB [10]).
To compare the quality of the segmentation algorithms, there are image databases, the most popular of which are BSDS500 [8] and PASCAL VOC [11]. It should be noted that such databases include images containing several large objects in the foreground, and the quality of the algorithms on images containing a large number of small objects is not investigated. Satellite images of the urban landscape can serve as an example of such images. For this reason, it is important to study the quality of segmentation algorithms in such cases.
In [12], algorithms [5-8] were compared and the best results on satellite images were shown by the SWA and gPb-owt-ucm algorithms (Figures 3 and 4). In this paper, the MCG and COB algorithms, which demonstrated the best result on image databases, were also considered (Figures 5 and 6). However, the COB algorithm showed low quality of segmentation on satellite images. The work of the COB algorithm is based on the processing of the data obtained from the artificial neural network of the special architecture, so the low quality of its result can be related to the need for retraining of the neural network on satellite images as its training was done on the BSDS500 and PASCAL VOC sets.
Fig. 1. The fragment of the satellite image
Fig. 2. The height map
Fig. 3. The
segmentation produced by SWA
Fig. 4. The
segmentation produced by gPb-owt-ucm
Fig. 5. The
segmentation produced by MCG
Fig. 6. The Fig. 7. The result of Fig.8. The result of
segmentation the proposed the high region
produced by COB segmentation detection algorithm
algorithm
The considered algorithms demonstrate the high quality of segmentation, but they do not take into account information about object heights. To test the influence of height data on the result of satellite image segmentation, we developed an algorithm based on the idea of combining pixels in the region on the basis of the closeness of their brightnesses and heights. Figure 7 shows the segmentation produced by the algorithm.
The algorithm receives a four-channel image and a height map. The following sequence of steps is performed.
1. Median filtering of the image.
2. Preliminary selection of a region. Looping through all pixels in the image is performed. Beginning with the considered pixel, we execute a recursive search for neighboring pixels with a close height and brightness (in 4-dimensional space). Further, mode values of the height and each of the four channels are calculated separately on the found region.
3. Selection of the region. In the obtained region, we select a point having a height exactly equal to the calculated mode of height and the closest brightness to the modes of each channel. Such a decision is due to the fact that the modes cannot coincide with the spectral and height data. Then, from the found point, we construct a new region by repeating the operations from step 2, and fill the new region with the color of the computed mode values.
4. Repeat steps 2-3. Due to the noisiness of the input data, the regions obtained by this algorithm will have a small area. The repetition of the segmentation procedure will allow us to merge the regions into larger ones.
5. Small region removal. As a result of the brightness or height discontinuity, regions with a negligible area can remain. Such regions are combined with neighboring ones in such a way that the region to which the points are added has a larger area, and its height and brightness are as close to the points being added as possible. This procedure is performed for regions in order of increasing area.
After segmenting the satellite image (using this algorithm or one of the above), it is necessary to determine whether the obtained regions belong to buildings and structures. For this purpose, in the next paragraph, a description of the algorithm for extracting high regions from the height map will be given.
2. Extraction of high regions. 1. Search for height differences. For each pixel p of the
height map with the height value h, we consider its one-dimensional neighborhoods O/(p) and
p) of the radius d in horizontal and vertical directions. For the pixels pi eOd (p), we calculate
Ah;. - h - h+1 ,i e [-d,d -1] where the index 0 corresponds to the point p, which is the center of the
neighborhood. And if Ah0 -max Aht either Ah0 - min Aht, or |Aho| > T, then the point p is marked as
having a significant height difference. We do not consider the points at which the minimum and maximum are simultaneously reached, since they are the errors in the height map.
2. Obtaining high regions. On the points that have a significant height difference, we construct line segments, which will form the target regions. For the construction of these line segments, we search horizontally and vertically in forward and backward directions on the height map for points with a height difference upward. Then we begin to draw a line segment from each selected point that goes:
• to the point with the height difference upward, the value of which is added to the value of the
difference at the starting point. Then the construction of the line segment continues;
• to the point at which the height value is less or close to the height value at the beginning of
the line segment;
• before accumulating the maximum possible length of the line segment.
Performing this procedure in the forward and backward directions (from left to right and vice versa, from top to bottom and vice versa) allows us to find even those regions of buildings in which one of the sides can be blurred due to the influence of noise.
Then, the points of segments that selected only horizontally or vertically are discarded. The remaining points of segments form regions. We select only those of them that contain at least one point with a significant height difference. This operation allows to reject the false intersections of vertical and horizontal segments.
3. Refinement of the form of the found regions. We consider connected regions in the height map, all points of which have the same height. If more than half of the points of the region were marked as high in the previous step, the remaining points of the region are also marked as high. Thus, the boundaries of the detected regions are smoothed and false alarms associated with slopes of hills and other sharp differences of the terrain height are rejected. The regions with small area are removed from consideration.
Fig. 9. The original image (left) and the vegetation extracted using the NDVI (right). In the presented fragment there are false alarms on the buildings.
4. Extraction of buildings among the found high regions. Not only buildings and structures, but also high vegetation (trees) are related to the detected high objects. The normalized difference vegetation index (NDVI) [13], which is calculated from the red and infrared channels of a satellite image, effectively separates vegetation from other objects in satellite images. However, in addition to vegetation, this index also extracts buildings that will lead to skipping objects when the building detection algorithm is running. Figure 9 illustrates examples of false alarms of the NDVI. To solve this problem, it is necessary to perform additional color filtering of the vegetation extracted using the NDVI.
The result of the high region extraction in the height map is presented in Figure 8.
3. Building detection. As mentioned earlier, our aim was to detect buildings and structures. Performing the segmentation procedure allows to split a satellite image into homogeneous regions, however, buildings can be covered by several regions. The detection of high objects in the height map allows us to extract the object entirely, but with low accuracy of boundary localization. It is suggested to combine the outputs of high region extraction and segmentation algorithms as follows: if more than 75% of the pixels of the segmented region are marked as high, then the whole region is considered high. This makes it possible to increase the accuracy of object localization and reduce the number of false alarms (on regions of trees closely adjacent to buildings). Figure 10 shows the results of the proposed approach using different segmentation algorithms.
To assess the quality of the algorithm, the extracted regions of buildings are compared with the manual markup. The following criteria are used for this:
1. Intersection over Union (also known as the Jaccard Index), which sensitive to the accuracy of
localization of the detected objects:
IoU =
Sob] n R
Sobl u R
where SoJ. - the found regions of buildings, and R - the manual markup.
2. The percentage of found buildings .
\Sobj nR \Sob, n rI
3. Precision = —.-¡—1 and recall = J———1.
So^l R
The results of the comparison can be observed in Table. 1.
Table 1.
Comparison of building detection quality using different segmentation algorithms
Segmentation algorithm IoU Nobj Precision Recall
High region extraction without segmentation 0,60 0,94 0,66 0,86
Proposed segmentation algorithm 0,62 0,90 0,72 0,83
SWA 0,54 0,65 0,75 0,65
gPb-owt-ucm 0,42 0,34 0,76 0,47
MCG 0,52 0,22 0,79 0,62
COB 0,51 0,35 0,81 0,59
The proposed segmentation algorithm, which differ only in the use of the height data, demonstrates better quality of the object localization (the highest value of IoU) than more sophisticated, but less specialized segmentation algorithms. Therefore, it almost certain that using height data will improve the quality of these algorithms.
MCG COB
Fig. 10. The results of building detection using different segmentation algorithms. For the building detection the outputs of segmentation and high region extraction algorithms were combined. The detected buildings are displayed in black, the manual markup in red, the regions obtained by the
high region extraction algorithm in blue.
Conclusion. In this paper, we proposed the approach for building detection based on the processing of a height map and a segmentation of satellite images. The findings of this study suggest that the use of height data allows to significantly improve the quality of segmentation algorithms on satellite images. Further work is planned to modify the considered segmentation algorithms to take into account height data. It is worth noting that the result of building detection also depends on the stereo matching and vegetation detection algorithms and applying more advanced methods will lead to a reduction in the number of missing objects.
References
1. Singhal S., Radhika S. Automatic Detection of Buildings from Aerial Images Using Color Invariant Features and Canny Edge Detection. International Journal of Engineering Trends and Technology (IJETT) 11 (8). pp. 393-396, 2014.
2. Ghafarian S., Ghafarian S. Automatic building detection based on supervised classification using high resolution Google Earth images. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 40 (3). pp 101-106. 2014.
3. A. van Etten. Building Extraction with YOLT2 and SpaceNet Data. URL: https://medium.com/the-downlinq/building-extraction-with-yolt2-and-spacenet-data-a926f9ffac4f
4. V.B. Kostousov, D.S. Perevalov, F.A. Kornilov. Digital terrain model generation from sattelite stereoscopic data // Materials of the XXX conference of the memory of the outstanding designer of gyroscopic instruments N.N. Ostryakova. - St. Petersburg: Concern CSRI Elektropribor, JSC. 2016. pp. 382-388.
5. D. Comaniciu, P. Meer. Mean shift: A robust approach to feature space analysis // IEEE TPAMI. May 2002. Vol. 24. No. 5. pp. 603-619.
6. Cour T., Benezit F., and Shi, J. Spectral segmentation with multiscale graphs decomposition // IEEE TPAMI. August 2000. Vol. 22. No. 8. pp. 888-905.
7. M. Galun, E. Sharon, R. Basri, A. Brandt. Texture segmentation by multiscale aggregation of filter responses and shape elements // ICCV. Vol. 2. 2003.pp. 716-723.
8. P. Arbelaez, M. Maire, C. Fowlkes, J. Malik. Contour Detection and Hierarchical Image Segmentation // IEEE TPAMI, Vol. 33. No. 5. 2011. pp. 898-916.
9. Arbelaez P., Pont-Tuset J., Barron J.T., Marques F., Malik J. Multiscale Combinatorial Grouping // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014. pp. 328335.
10. K.K. Maninis, J. Pont-Tuset, P. Arbelaez, L. Van Gool. Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks // IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). Issue 99. 2017. pp. 1-14.
11. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, A. Zisserman. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results // URL: http://www.pascalnetwork.org/challenges/VOC/voc2012/workshop/index.html
12. A.V. Maryanova. The investigation of the quality of image segmentation algorithms depending on the size of objects in the image // Proceedings of the 46th International Youth School-Conference «Actual problems of mathematics and its applications», Ekaterinburg. 2015. pp. 129-134.
13. V.V. Kozoderov, T.V. Kondranin. Methods for assessing the state of soil and vegetation cover from optical remote sensing systems. Moscow, MIPT. 2008. 222 p.
Authors:
Dunaeva Alexandra Valerievna, mathematician, Department of Applied Management Problems, N.N. Krasovskii Institute of Mathematics and Mechanics of the Ural Branch of the Russian Academy of Sciences (S. Kovalevskaya street 16,Yekaterinburg, Russia, 620990); assistant, Department of High-Performance Computer Technologies, Ural Federal University named after the first President of Russia B.N. Yeltsin (Mira street 19, Yekaterinburg, Russia, 620002)
Kornilov Fedor Andreevich, researcher, Department of Applied Problems of Management, N.N. Krasovskii Institute of Mathematics and Mechanics of the Ural Branch of the Russian Academy of Sciences (S. Kovalevskaya street 16, Yekaterinburg, Russia, 620990), candidate of Physico-Mathematical Sciences
УДК 004.932
Обнаружение строений на спутниковых снимках с использованием карты высот * А.В. Дунаева, Ф.А. Корнилов**
ФГБУН Институт математики и механики им. Н.Н. Красовского УрО РАН,
ФГБОУ ВО Уральский федеральный университет им. первого Президента России Б.Н. Ельцина, Екатеринбург, Россия
Работа посвящена задаче обнаружения строений и сооружений на спутниковых снимках земной поверхности. Предлагается схема алгоритма, основанного на выделении высотных объектов местности по карте высот и уточнении их границ на основе сегментации спектральной информации, что позволяет повысить точность распознавания. Приводятся результаты сравнения качества работы предложенного подхода с использованием различных алгоритмов сегментации изображений.
Ключевые слова: спутниковые снимки; сегментация; обнаружение строений; карта
высот.
Авторы:
Дунаева Александра Валерьевна, математик, отдел прикладных проблем управления, ФГБУН Институт математики и механики им. Н.Н. Красовского УрО РАН (620990, Российская Федерация, г. Екатеринбург, ул. Софьи Ковалевской, д. 16); ассистент, кафедра высокопроизводительных компьютерных технологий, Институт естественных наук и математики, ФГБОУ ВО Уральский федеральный университет им. первого Президента России Б.Н. Ельцина (620002, Российская Федерация, Екатеринбург, ул. Мира, 19)
Корнилов Фёдор Андреевич, научный сотрудник отдела прикладных проблем управления, ФГБУН Институт математики и механики им. Н.Н. Красовского УрО РАН (620990, Российская Федерация, г. Екатеринбург, ул. Софьи Ковалевской, д. 16) кандидат физико-математических наук
* Работа выполнена при частичной поддержке комплексной программы ФНИ УрО РАН (проект 15-16-1-14).
** E-mail: maryanova.av@yandex.ru, kornilovfa@imm.uran.ru.
