Automatic detection of channels in seismic images via deep learning neural networks Текст научной статьи по специальности «Компьютерные и информационные науки»

Automatic detection of channels

■ ■■■ ■ ■ ■ ■

in seismic images via deep learning neural networks

Fedor V. Krasnov

Expert, Science & Technology Centre Gazprom Neft PJSC

Address: 75—79 liter D, Moika River Embankment, Saint Petersburg, 190000, Russian Federation E-mail: [email protected]

Alexander V. Butorin

Expert, Science & Technology Centre Gazprom Neft PJSC

Address: 75—79 liter D, Moika River Embankment, Saint Petersburg, 190000, Russian Federation E-mail: [email protected]

Alexander N. Sitnikov

Deputy Chief Executive Officer for Geology and Field Development, Science & Technology Centre Gazprom Neft PJSC

Address: 75—79 liter D, Moika River Embankment, Saint Petersburg, 190000, Russian Federation E-mail: [email protected]

Abstract

The business goal of interpreting seismic data has always been addressed by the high-level experts engaged. The authors applied a computer vision approach to interpret seismic data. The expert task of interpreting seismic data has become partially automated via machine learning techniques utilized to classify the images used by the authors. The methods of transformation of seismic traces through spectral decomposition were used to obtain the data set. In the previous works of the authors, methods of spectral decomposition via continuous wavelet transformation were created, and this also laid the foundation of this study.

Use of artificial neural networks of deep learning has enabled the authors to meet the goal of image classification. In this regard, it is important to note that the business policy related to information dissemination imposed certain limitations on the computing capacity used and the number of the data labeled. The solution found for the use of trained artificial neural networks and image augmentation helped us to successfully cope with the goal, in spite of the limitations.

The results obtained allow us to identify geological units with a test accuracy of 90% rendering to the Fl-score measure. This has enabled the Scientific and Technical Center of Gazprom Neft to implement automated procedures in the existing business processes in order to significantly reduce the time needed to process seismic data.

The authors consider the possibility of "digitizing" and preserving the knowledge of the highestlevel experts in interpreting seismic data, as well as the possibility of using contactless examination to locate geological units in the seismic data within the Gazprom Neft group of companies to be a socially efficient outcome of this study.

Key words: supervised machine learning, small datasets, spectral decomposition, seismic, geological units, computer vision.

Citation: Krasnov F.V., Butorin A.V., Sitnikov A.N. (2018) Automatic detection of channels in seismic images via deep learning neural networks. Business Informatics, no. 2 (44), pp. 7—16. DOI: 10.17323/1998-0663.2018.2.7.16.

Introduction

The subject of this study is automation of analysis of geological and geophysical data in order to optimize decision making. This subject is being actively addressed within the Digital Oil Field (DOF) strategy [1]. Field development is the longest and the most expensive stage of field exploitation, starting from planning for the first well drilling and lasting for decades while a field is under development. Petroleum engineers have to come up with the most efficient solutions both in the preparatory stage and in the later stages in order to produce maximum quantities at the lowest cost. The DOF strategy applied to oil prospecting and production encompasses most of the major activities: exploration, drilling, development, production and construction of field facilities.

Digital technologies are used in geological exploration to analyze geological and geophysical data on an explored medium: results of drilling, well surveys and geophysical works. All of this information represents a significantly large data stream the analysis of which enables predicting the geological structure of a field and building geological and hydrody-namic models for further modeling of exploration and finding the most efficient production techniques.

The data from surface geophysical surveys produce significant volumes of information on a geological medium on the entire field area. In this sphere, the seismic survey is of special significance. It enables us to examine a medium through modes of elastic waves propagation in it. This approach requires artificial

wave injection in the near-surface zone, after which the resulting elastic waves interact with the medium and return to the surface where they are being recorded.

The subsequent processing of the recorded elastic waves results in a seismic image of the geological medium formed by the wave signals reflected by elastic contrasts. One of the formats of presenting such information is a seismic cube (for aerial surveys) where the X—Y— time coordinates show amplitudes of reflected signals for the geological medium from its surface to several kilometers depths (Figure 1). Further analysis concerns studying distribution of amplitudes of the reflected waves which depend on the medium properties on the given area and within a target interval. The information so obtained gets integrated with the well data and serves as a basis for evaluating the studied material's geological properties, i.e. it forms a geological model of a field.

Fig. 1. Example of a seismic wave field (Y axis - time, Z and X axes - distances, color - the reflected wave's amplitude)

Fig. 2. Comparing false color images of seismic data (left) and the present day river geometry (right)

Most of the current fields are in the later stages of development as regards major production strata, which calls for searching for so-called 'overlooked intervals' — prospective undeveloped targets which may contain hydrocarbons in the rock. The task of searching for such targets requires analysis of the geological properties of the entire geological cross-section as yet undeveloped. In this regard, target-oriented interpretation is widely practiced, i.e. searching for unambiguous geological objects featured by reflected waves amplitudes behavior. By doing so in order to obtain more meaningful results, researchers rely on various transformations of the original wavefield in order to detect certain particularities. An example of such transformation is spectral decomposition and related false color imaging described in the work [2] (Figure 2).

The process of searching for 'overlooked intervals' includes analyzing areal distribution of field anomalies within an interval in question and in doing so a researcher has to visually identify geological targets by indicative anomalies. In such cases, it is common to use actu-alism principles which establish similarity of present day objects with their similar ones in the geological past. Thus, the search process includes solving the following tasks:

♦ identification of a target interval;

♦ obtaining areal measurements of the interval's anomalies with active time/depth steps;

♦ analysis of the obtained areal measurements to search for geological targets;

♦ joint analysis of the identified targets and the well data.

The major disadvantage of this approach is the significant labor input as well as the potentially subjective interpretation of geological targets. A researcher has to study an enormous number of areal measurements and identify prospective intervals and areas.

As was noted above, the DOF strategy implies automation of processes such as geological surveys to optimize field operations. Therefore, automation of the process of searching for 'overlooked intervals' is a significant business challenge, both in terms of cutting time consumed and obtaining more objective results irrespective of a particular researcher and for identifying potential targets for further drilling and hydrocarbon production.

This article describes the solution of this problem by using machine learning technologies for automatic search and classification of identified anomalies. Furthermore, the algorithm used should provide high precision solutions without requiring a broad sample for training. A typical anomaly is a river geometry represented by channels (streambeds) which are of significant interest from the point of view of having a collector and can be positively identified on actualism principles.

Thus, the article examines the problem of 'translating' researchers' knowledge into a learning algorithm of search and classification. The solution of this problem allows us to increase efficiency of decision making as well as increase exploration depth by avoiding time consuming procedures.

Neural networks are currently widely spread in the gas and oil industry as a tool for addressing various problems. The experts note that this

is facilitated by productivity growth of computing systems and new types of neural networks with a large number of layers, such as Deep Neural Networks (DNN). There can be a few dozen of layers in these networks, but there are examples of DNN where the number of layers exceeds one hundred.

Xception, outlined in this paper, is the one of the examples of DNN [3]. The prediction accuracy based on Xception compared to other DNN is presented in the table (Table 1).

DNN learning is a complex engineering challenge. As a rule, for DNN training a ImageNet database with tagged images is used. ImageNet is an image database organized according to hierarchy (currently only nouns), in which each node of the hierarchy is depicted by hundreds and thousands of images. Currently ImageNet has an average of over five hundred images per node. An example of such nodes could be 'cats' and 'humans'. For DNN training with the ImageNet, a subset of ImageNet classes is selected. However, among the ImageNet classes there are no geological images.

There are over 22 million parameters that need to be optimized in the Xception learning process. A uniquely performing computing cluster was used for Xception training, based on the ImageNet set [5].

Finding a method of transferring the geologist's expertise in identifying geological objects into a digital algorithm is the key issue that is

being addressed in this paper. The authors put forward the following research hypothesis:

"Machine learning algorithms based on deep neural networks allow us to make highly accurate identifications of geological units from the seismic data processed via spectral decomposition methods using supervised learning in small datasets."

This research consists of an introduction, description of established methods and methods created by the authors in order to prove the hypothesis being put forward, the results of digital experiments and a conclusion.

1. Methods

Funding a solution for image classification with the help of supervised learning in small datasets is the linchpin of this paper. The scheme of the method offered by the authors is depicted as a research framework in Figure 3.

Furthermore, each element of the research framework is considered in details from the point of view of data processing and building algorithms.

1.1. Seismic cubes

From the data point of view, seismic cubes are represented in the SEG-Y format. SEG-Y is an open data format containing acoustic traces. The size of a SEG-Y file can vary between 1 GB and 100 GB. One such file con-

Table 1.

Maximum accuracy of different DNN on ImageNet dataset [4]

Model Top-5 Accuracy Parameters Depth

Xception 0.945 22,910,480 126

VGG16 0.901 138,357,544 23

VGG19 0.910 143,667,240 26

ResNet50 0.929 25,636,712 168

InceptionResNetV2 0.953 55,873,736 572

No channel

Channel

■¡¿3

Images classification

Deep learning neural network optimization

Seismic cubes Feature design Dataset creation

2L

L □

c «W Hi

v. S ■»..

Fig. 3. Research framework

tains information about a certain deposit area -the area can be dozens of square kilometers vast. One cube can approximately contain a couple of million traces. Each trace has a sampling step to a time limit of 1—4 ms. Therefore, the total three-dimensional cube's network can be represented by an approximate size (3000, 3000, 2000). Each seismic cube is presented in the Python NumPy format after conversion.

1.2. Feature design

The search for geological units is being done in a certain seismic time range (Z axes). The secant surfaces are used in order to set the higher and lower boundaries of this range, called reflecting horizons (RH). Identifying optimal RH is a separate optimization task, so the selected range is divided into a couple of conforming RH in order to do this. At this point it is unclear in which of the selected RH will the geological unit show itself.

Continuous wavelet transformation (CWT) is performed in order to go from the coordinates and depth space into the coordinates and frequencies space for the chosen range limited by RH.

Each section of the frequency cube which is cut by the reflecting horizon represents a two-dimensional map of frequency amplitudes after this. The RGB blending mechanism looked at in this work is used in order to visualize such maps [6]. The number of RGB maps obtained can be calculated using a formula:

NumOfRGBmaps = ReflHorDim -C™%Dim

(1)

where NumOfRGBmaps — the number of RGB maps;

ReflHorDim — the number of conforming horizons in the chosen range;

^FrvquencyDim — the number of combinations from a number of chosen frequencies (Fre-quencyDim) by the number of colors Color-Dim (for the considered case ColorDim = 3 (RGB)).

The two-dimensional RGB maps obtained are the initial data for searching for geological units.

1.3. Dataset creation

The main objective of creating a dataset is to prepare the RGB maps obtained for the process of machine learning. An RGB map may or may not contain geological units. In order to mark a dataset, the given RGB map has to be manually classified.

It will be possible to meet the challenge of binary classification via the marked dataset. We want to point out that the dimensions of an RGB map are equal to the considered area of deposit. As discussed before, that is (3000, 3000) on average. In order to effectively utilize algorithms of computer visioning on the RGB map, it was decided to divide it into images sized (224, 224). Therefore, each map got around 169 images (mini RGB maps).

A full marking of the set of data obtained is around 16 million mini RGB maps. That is why the decision was made to mark a small subset of mini RGB maps consisting of 50 mini

RGB maps with channels and 50 mini RGB maps without channels.

1.4. Training deep neural network

We considered the following architectures of deep neural networks:

❖ VGG16 — a network that comprises 16 layers by Visual Geometry Group from Oxford University for identifying objects on images [7];

❖ Inception v3 — a neural network by Google for identifying units on images. Consists of 159 layers [8];

ResNet50 — a neural network by Microsoft utilizing residual learning. Used for identifying objects on images. Consists of 168 layers [9];

Xception is a modification of Inception network consisting of 126 layers [3].

Guided by a comparison of the accuracies of architectures above, we have chosen the most modern one, which is the Xception.

"No labeled data, no CPUs to train own DNN" — these were the conditions in which the authors resolved the given problem. This format of problem-solving using limited computational and manpower resources is of practical interest. That is why we made the decision to use the parameters obtained during training Xception with the ImageNet dataset.

DNN Xception learning with the ImageNet dataset is designed to classify 1000 classes of domestic objects from ImageNet. There are no geological objects in the ImageNet dataset. That is why the authors of this research utilized the knowledge transfer method. The linchpin of the knowledge transfer method is to use only a part of the trained DNN layers, develop our own layers and re-train DNN with the data from the needed problem domain. In the research considered, these were the images from mini RGB maps.

The authors added one pooling and five full connected layers (Figure 4) with dimensions (1024, 512, 256, 64, 1).

global_average_pooling2d_8: GlobalAveragePooling2D input: (None, 5, 5, 2048)

output: (None, 2048)

dense_39: Dense input: (None, 1024)

output: (None, 512)

1

dense_38: Dense input: (None, 512)

output: (None, 256)

1 r

dense_39: Dense input: (None, 256)

output: (None, 264)

dense_40: Dense input: (None, 64)

output: (None, 1)

Fig. 4. Five additional dense layers for DNN

Fig. 5. An example of a close-up of one mini RGB map

An image close-up method, upgraded based on the research, was utilized by the authors in order to manually increase the dataset [10]. A certain set of transformations was applied to each marked image. The number of different images in the dataset was increased 12 times as a result of that. An example of a close-up of one mini RGB map is shown in Figure 5.

2. Experiment results

The following optimization algorithms were used for training the model:

♦ Stochastic Gradient Descent (SGD) optimizer;

♦ RMSProp optimizer;

♦ Adam optimizer.

During the optimization process the function of crisscross entropy was used as a loss function

ht{y>P):

Llog{y,p) = -{ylog{p) + {i-yyiog(l-p)), (2)

where the true class y e {0, 1}, and p is the probability that y = 1.

Model training was done in two modes.

Training mode I. The training is done only with the layers added by the authors. The parameters obtained during model training with ImageNet data are not changed. The learning curves for different optimizers with already lined up parameters (learning rate and other specific parameters to each optimizer) are shown in Figure 6.

Crossentropy 2.5 2.0

1.5

1.6 0.5 0

0 5 10 15 20 25 30 35 40

Epochs

Fig. 6. Learning curves for different optimizers in Training Mode I

Crossentropy

5 -4 -3 -2 -1 -0 -

0 5 10 15 20 25 30 35 40

Epochs

Fig. 7. Learning curves for different optimizers in Training Mode II

Training mode II. Training of all parameters is done. Parameters obtained during model training with ImageNet data are used initially. The learning curves for different optimizers with already lined up parameters (learning rate and other specific parameters to each optimizer) are shown in Figure 7.

Finally, the authors stopped at the optimized Adam algorithm [11] in Mode II with the optimizer's parameters learning rate = 3.16e — 4; beta1 = 0.9; beta2 = 0.999. The training process took 18 hours on average. We note that in Mode I the training process took one hour on average. This difference in training time is explained by the difference in parameter quantity. The Adam optimizer showed a faster convergence and productivity.

An Accuracy metric was used to check the training quality of the model. Values of the Accuracy metric depending on the learning step for the different parameters of learning rate of the chosen Adam optimizer in Mode II are shown in Figure 8.

An analysis of the Precision vs. Recall metrics was made in order to choose the probability threshold of making a positive decision in relation to class. In terms of the task at hand, we were mostly interested in identifying channels on RGB maps. The small number of cases of identifying False Positive is not critical. The final values of accuracy metrics are shown in the Table 2.

A classification of RGB map images for one deposit was made with the help of the trained model. Two RGB maps placed into differ-

Accuracy

1.0 0.9 0.8 0.7 0.6 0.5 0.4

- learning rate = 3.16e - - 04

----learning rate = 5.62e - - 04

.......... learning rate = 1.00e - 03

— learning rate = 1.78e - 03

- learning rate = 3.16e - 03

0 5 10 15 20 25 30 35 40

Epochs

Fig. 8. Values of the Accuracy metric depending on the learning step for different learning rate parameters of the chosen Adam optimizer in Training Mode II

ent classes based on the model's decision are shown in Figure 9.

Conclusion

The progress of the deep neural networks society's modern level of openness allows small research organizations to solve their production tasks with the help of the most advanced algorithms. Open access publication of not only of the source code but also the parameters of trained deep neural networks leads the interpretation of geological image classification to a new level.

Classification report

Table 2.

Precision Recall F1-score Support

No channel 0.89 0.92 0.91 26

Channel 0.91 0.88 0.89 24

Average / Total 0.90 0.90 0.90 50

Fig. 9. Two RGB maps. The left one is classified as containing channels, the right one does not. The 224, 224 square is the area based on which the classification decision was made

In the current research, the authors received results with the Accuracy metric, comparable with the declared ones for this architecture of deep neural network. The average value of F1-score metric was 90% accurate.

The authors have successfully utilized their spectral decomposition methods developed in previous works. In the current paper, these methods were used in order to create a dataset.

The image augmentation method has proved itself to be a flexible and universal instrument for widening the dataset. This method can be used in many areas where the opportunity for manual classification of a dataset for learning is limited.

The challenge of classification of geological units which has been met by the authors contains novelty and can be applied in science

research organizations of the gas and oil industry.

The automated processes obtained by the authors allow us to increase the accuracy of geological unit identification, which is especially important in the case of reserves that are hard to retrieve.

The methods of interpretation of false color images of seismic fields that are currently in use are based on heuristic approaches. This research's novelty is in maintaining the existing heuristic approaches with the help of deep neural networks training.

Methods of identifying geological units with the help of deep neural networks are examples of the "supervised learning" method. This is a greenlight in a difficult process of adapting the concept of "Digital oilfield" by accumulating knowledge from experts for multiple use. ■

References

1. Sitnikov A., Pustovskikh A., Asmandiyarov R., Gilmanov R., Sheremeev A., Zulkarniev R. (2015) Digital information systems creation for optimization of complex geotechnical jobs programs formation process for JSC Gazprom Neft oilfields. Proceedings of SPE Russian Petroleum Technology Conference, Moscow, Russia, 26—28 October 2015, vol. 1, pp. 642—662.

2. Butorin A.V., Krasnov F.V. (2016) Approaches to the analysis of spectral decomposition for the purpose of detailed geological interpretation. Proceedings of SPE Russian Petroleum Technology Conference, Moscow, Russia, 24—26 October 2016, vol. 1, pp. 275—289.

3. Chollet F. (2017) Xception: Deep learning with depthwise separable convolutions. arXiv:1610.02357v3 [cs. CV]. Available at: https://arxiv.org/pdf/1610.02357.pdf (accessed 01 February 2018).

4. He K., Zhang X., Ren S., Sun J. (2016) Identity mappings in deep residual networks. Proceedings of the 14th European Conference on Computer Vision (ECCV2016). Amsterdam, The Netherlands, 11—14 October 2016, part IV, Lecture Notes in Computer Science, vol. 9908. Springer, Cham,

pp. 630-645.

5. Abadi M., Agarwal A., Barham P., et al. (2016) TensorFlow: Large-scale machine learning on heterogeneous distributed systems. Proceedings of the 12th USENIXSymposium on Operating Systems Design and Implementation (OSDI'16). Savannah, GA, USA, 2—4November 2016, pp. 265-283.

6. Li Y.-F., Cheng J.-Y., Zhu S.-J., Wang C. (2009) Seismic multi-attribute analysis based on RGB color blending technology. Journal of China Coal Society, no. 11, pp. 018-034.

7. Simonyan K., Zisserman A. (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556v6 [cs. CV]. Available at: https://arxiv.org/pdf/1409.1556.pdf (accessed 01 February 2018).

8. Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. (2016) Rethinking the inception architecture for computer vision. Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016). Las Vegas, NV, USA, 26 June - 1 July 2016, pp. 2818-2826.

9. He K., Zhang X., Ren S., Sun J. (2016) Deep residual learning for image recognition. Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016). Las Vegas, NV, USA, 26 June - 1 July 2016, pp. 770-778.

10. Krizhevsky A., Sutskever I., Hinton G.E. (2012) ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, pp. 1097-1105.

11. Kingma D., Ba J. (2017) Adam: A method for stochastic optimization. arXiv:1412.6980v9 [cs. LG]. Available at: https://arxiv.org/pdf/1412.6980.pdf (accessed 01 February 2018).