CC BY
0
Synthesis of stochastic algorithms for image registration by the criterion of maximum mutual information
A.G. Tashlinskii1, G.L. Safina2, R.M. Ibragimov1 1 Ulyanovsk State Technical University, 432027, Russia, Ulyanovsk, Severnyi Venets 32;
2 National Research Moscow State University of Civil Engineering, 129337, Russia, Moscow, Yaroslavskoe shosse 26
Abstract
We discuss a synthesis of stochastic algorithms, obtaining expressions for gradients of Shannon, Renyi and Tsallis mutual information on the basis of the mathematical apparatus of stochastic gradient adaptation of algorithms for estimating image registration parameters. To obtain the expressions, derivatives of the image entropy with respect to the estimated parameters are used. The entropies are calculated using a Parzen window method. A comparative study of the synthesized algorithms in terms of stability and accuracy of the registration parameter estimates, including in conditions of additive noise, is carried out.
Keywords: image, estimation, parameter, binding, stochastic procedure, mutual information.
Citation: Tashlinskii AG, Safina GL, Ibragimov RM. Synthesis of stochastic algorithm for image registration by the criterion of maximum mutual information. Computer Optics 2024; 48(1): 109-117. DOI: 10.18287/2412-6179-CO-1332.
Introduction
The widespread use of video information puts forward ever-increasing requirements, both for technical means and for image processing methods aimed at improving the perception, analysis, recognition and interpretation of images for decision making. A key operation in solving applied problems of image and video data processing in various fields [1, 2] is image registration. It involves finding a function or parameters of a given function that establishes correspondences between conjugated points of two or more images. In this case, these can be both frames of the same video sequence, reflecting the dynamics of the scene, and images formed by different devices, received at different times, in different spectral ranges. The registration result can be either the final product or a link in solving another problem. For example, when combining images according to the registration results, they are reduced to common coordinates by a spatial geometric transformation, before being complexed; when identifying the object of interest its parameters are found.
Mutual geometric deformations of the registered images can be either global, i.e. inherent in the entire area of the processed image, or local, i.e. restricted to particular areas. When processing images, both types of deformation need to be described using some mathematical models. Thus, the images of the same scene taken at different camera angles are described by a projective model [3], the estimation of the parameters of which requires significant computational costs. At the same time, the terrain can add local distortions. To describe local deformations, simpler models, for example, an Euclidean one, are often sufficient.
As already noted, the problem of image registration arises in a variety of fields: in medicine, for example, when combining images of positron emission, magnetic
resonance and computed tomography [4, 5], monitoring by video information of the dynamics of changes in elements of industrial facilities [6], constructing trajectories of unmanned aerial and underwater vehicles [7], monitoring the Earth's surface [8], mapping the soil and vegetation cover by remote sensing [9], detection of changes [10], detection and identification of objects of interest [11, 12], increasing informativeness [13], and many others.
Taking into account the variety of problems arising from various restrictions on the source data and the mathematical apparatus used, image registration methods have been intensively developing for several decades and are becoming increasingly widespread. Many effective approaches focused on various applied problems have been proposed. Conventionally, these approaches can be divided into two large categories: image registration based on key features [14, 15] and based on intensities [16]. In situations where the linked images have a different structure (for example, obtained from different spatial coordinates and/or at different times) or nature (formed in different spectral ranges), the identification of key features is often difficult and the binding parameters are unreliable. The intensity-based registration is based on an estimate of the numerical value of some given measure of similarity of the studied images and does not require searching for any features.
The difficulties that arise when registering multispec-tral and multi-temporal images are due to the fact that, in addition to different camera angles and shooting scales, such images often have significant mutual nonlinear brightness distortions and pulse interference of a priori unknown type. At the same time, traditional methods, for example, SIFT [17] and SURF [18], using reference marks [19], sequential analysis [20] in difficult conditions have low accuracy and reliability. Good resistance to conditions of a priori uncertainty is provided by procedures for stochastic non-identification registration of images [21, 22]
based on information-theoretic measures [23]. In the synthesis of such procedures, information-theoretic similarity measures, in particular, mutual information (MI) of images [24], are chosen as objective functions.
1. The problem statement
When registering several images with a view of reducing the error, each subsequent image is referenced to some image conditionally taken as reference. Then, in relation to it, the other images will have spatial deformations, whose parameters need to be determined. If another image is selected as the reference image, then the images are still bound in pairs. Therefore, we will consider the binding of two images: a reference Zr and a deformed one Zd, assuming that the conjugate points of these images can be found using a given model of geometric deformations with a set of parameters a. The determination of optimal binding parameters using the mathematical apparatus of non-identification stochastic adaptation [25] is reduced to a recurrent search for the extremum of a multidimensional objective function Q(a, Z r, Zd ) in the parameter space. The parameter vector corresponding to the ex-tremum is considered to be optimal.
Stochastic algorithms for estimating image registration parameters a are based on a relay stochastic gradient adaptation procedure, which can be written as:
= a,+ At Vq (a,_!, Zr, Zd),
(1)
where Vq() is the stochastic gradient of the objective function Q (at_1, Zrt, Zd); A , is the gain matrix that determines the rate of change of the estimates a,; Zr is the image Z r, resampled at the t-th iteration according to the estimates at_1 obtained at the previous iteration; t = 1, T is the iteration number.
The speed of (1) is determined primarily by the computational costs of finding the stochastic gradient Vq(). To reduce them, it is advisable to use at each iteration only a small part of the pixels zrjt e Z[ c Zr and zd e Zf c Zd of images Zr and Zd, which we will call the local sample Z, = Zr, u Zf, where j,eQZ are the coordinates of the pixels included in the local sample Z,. The rule for choosing these coordinates can be different, in particular, it can depend on the iteration number or the local sample pixel coordinates can be chosen randomly with equal probability. In general, the optimization problem [26] of determining the image area from which pixels are selected as a function of the iteration number and the autocorrelation function of image brightness requires a separate study for each type of objective function.
The stability of the calculated estimates of the registration parameters to the conditions of a priori uncertainty of the image parameters and brightness noise is provided by a relay stochastic procedure in which the sign function is used to limit the variability of the estimates:
at = at_i + atsign vq (a,_i, z, ).
(2)
The objective of this study is to synthesize relay stochastic algorithms for estimating image registration parameters when choosing the objective function of the estimation quality. Such functions are Shannon, Renyi and Tsallis MI. In this case, the stochastic gradient of MI should be obtained on the basis of partial derivatives of MI entropies. Solving this problem is important because it provides a basis for analyzing the effectiveness of other, more simplified approaches. In particular, when using the histogram method [27] or estimating the stochastic gradient of objective function by finite differences [28]. Another problem is to experimentally estimate the efficiency of the obtained algorithms under noisy conditions.
2. The synthesis of image registration algorithms based on mutual information 2.1. Stochastic gradients of mutual information of the studied images
The numerical values of Shannon MI [29] at the t-th iteration can be found by a well-known expression using Shannon entropy:
Qs, = His (Zr,) + His (Zd) _ HHs (Zr, Zd )
(3)
where HHS (Zr) and HHS (Zdt) are Shannon entropy estimates of images Zr and Zd; HHS (, Zd) is the estimate of the joint Shannon entropy of these images.
The estimates of Shannon entropies from a local sample of images can be found by the formulas:
HS (Z* ) = _E p (Z*t )log2 p (zii )
jeQZ
HHS (Zr, Zd ) =
=_EE p(j)p(j)log2 p(j)P(zjik),
j. ,jk e^Z
(4)
(5)
where p (zd,), p (z¡,) and p (, zj^) are values of probability density function (PDF) and joint PDF at pixel brightness zrjti, zj^ e Z,.
Then, in accordance with (3), the stochastic gradient of Shannon MI is determined by the expression:
Vq St =
dHs (Zr) dHs (Zr, Zd)
9a
9a
Tsallis MI [30] is determined as
Qt, = HT (zr)+HHT (zd)_HT ((, zd) + + (1 _ qT )HT (z )HT (zd ),
where
Ht (Z*) = (qT _ 1)1 E p(z*t)((pqT 1 (z*i)),
jeQZ
HT (Zr, Zd ) =
=( _ 1)_1 EE (1 _pqT1 (zjti, zjik))
(6)
(7)
(8)
(9)
ji ,jk eQZ
are estimates of the Tsallis entropy of images Z[ and Zd and the joint entropy; qr is the order of Tsallis entropy.
Accordingly, the stochastic gradient of the Tsallis MI (7) is
vq Tt —
дит (z; ) dw .4
—0-> (l + H(Zd)(1 -qT ))-
dHT (z;, Zd ) 3a
Renyi MI [31] can be calculated as
HR (Z ; )+HR (zd )
(10)
Qr. — -
HR (Z ;, zd )
(11)
where
HR (Z* ) —(1 - qR )-1log2 X PqR ), (12)
je!2z
Hr (Z;, Zd) — (1 -qR )-1 XX log2 pqR ( j, zdk) (13)
ji ,jk EÛZ
are Renyi entropy estimates; qR is the order of Renyi.
Taking into account (11), the stochastic gradient of the Renyi MI can be calculated by the expression:
Vq r, — H R (Z ;, Zd )
3H R (Z ; )
3a
(( R (Z ; )+H R (Zd )) 3H R (Z;, Zd )
(14)
H R (Z ;, zd )
3a
Thus, finding the stochastic gradients of the Shannon, Tsallis, and Renyi MI is reduced to determining the PDF and joint PDF estimates of images Zr and Zd at each iteration, as well as the derivatives of the entropies of the corresponding MI with respect to the estimated parameters.
2.2. Estimates of probability density_functions of the reference and deformed images
As already noted, when finding the numerical values of MI at each iteration of the procedure (2), it is required to estimate the single PDF and joint PDF of images Z rt and Zd. At the same time, estimates of both PDF must be found from a local sample Zt, which has a small volume (units, tens of pixels), which leads to increased requirements for the accuracy of PDF approximation. PDF estimates can be found either using the histogram method [27] or using the Parzen window method [32]. Studies show that the Parzen window method is more informative for the problem under consideration, which also assumes a simple software implementation.
The idea of the method is that the PDF estimate p(z) is found as a normalized superposition of elementary symmetric distributions w(z)j, centered on the pixel brightness zjd , jteQz, that are in the local sample:
p> (z) = ( E w (z)| , (15)
( Jeqz ]t
where ( is the local sample size (cardinality of the set Qz).
In particular, if the elementary distribution is Gaussian, then
(z ) — X~fexP
Ц jeQW 2naG
(z - zjt)
2л
(16)
where ctg is the variance of the elementary distribution, which is chosen from some criterion and significantly affects the quality of entropy estimates. Instead of a Gaussian distribution, any other suitable differentiable function can be used, for example, the Cauchy PDF. However, the features of the Gaussian function simplify the subsequent analysis, therefore, in this study, it is chosen as an elementary distribution in the Parzen window method.
2.3. Derivatives of the entropy of mutual information with respect to the estimated parameters
To calculate the stochastic gradient of all MI types, it is necessary to find partial derivatives dH (zr, )|3a and dH (zd, Zr |3a with respect to the estimated registration parameters of the single image entropy Zr and joint entropy of images Zr and Zd. This can be done either by numerical methods using finite differences [28], or, as in this work, analytical formulas can be obtained taking into account the accepted model of mutual spatial deformations of images.
There is the known approach to find analytical dependencies of Shannon MI when representing an image model by a random field Z = {zj}. In this approach, finding estimates of the Shannon entropy is based on the fact that it can be expressed as the mathematical expectation of the negative logarithm of the PDF [33]:
Hs (Z) = -E p(Zj )log2 p(Zj) = M[-log2 p(Z)],
Zj eZ]
where M[ ] is the expectation operator. Estimates p(zj) for a local sample Z:cZ with volume (1 are constructed using the Parzen window method. The mathematical expectation (- log2p(Z)) is estimated by the average value p(zj) of another sample z2 from the same random field Z, which can have a volume (2 that is generally different from (1. Then, as a calculated relation for entropy, we obtain:
Hs (Z) « H S (Z) —
1 г 1
—--X log2 — X WG (zi - )
H"2 zk eZ2 z, ez1
(17)
where zt is the brightness of the element ZjeZ:; zk is the brightness of the element ZjeZ2. Then the derivative of (17) is calculated. The usage of two samples (one for estimating the PDF and another for finding the average value of the logarithm of the PDF) leads to inefficient usage of the information contained in them.
In this work, for the analytical determination of entropy derivatives, a different approach was used, in which
only one sample of images Zrt and Zd is used. As functions of entropy for differentiation, expressions (4) and (5) were used for Shannon MI, (8) and (9) for Tsallis MI, and (12) and (13) for Renyi MI. Omitting intermediate calculations, we present the resulting expressions.
With known Zt the partial derivatives for the Shannon entropy can be found by the expression:
dHst (Zr )
3a
l0g2
-1r gigfvi Ai
M^G k EZ( |zr EZ( | 3a
M-1 S v;
A AA + 1
J JJ
(18)
zk are
where Vrik = p[ ( _ zrk); Ai = z. _ zrk; zf and brightness of pixels zrit e Zt.
Similar relationships can also be written for
9HS, (Zd, Zr ))a :
H ( Zd, Zr )
1
3a
( (
l0g2
M^G z[ ,zd eZt |zf,zf eZt
g seivsviAi 30.
M-"1 S V Vf
a aa +1
j jj
(19)
zk are
where Vf = wt (zf _ zf ); Af = zf _ zf ; zf and brightness of pixels zft e Z,.
The partial derivatives for the single and joint Tsallis entropy are determined by the expressions:
( ( 3Ar 3H, (Zr )_ zg. [S lViAi ^^
3a
(1 - qi )m^G
1 -
Mqi
gvi
\qr
-A A
(20)
/J
3HTt (zd, Zr )
g g I Vi Vfi A
3A r
Jr Xld Kr ik ik -3a
3a
(1 - qi )m^G
1 -
Mqi
Mq
g vrkVi
\q.
-A A
/J
(21)
where qi is the order of Tsallis entropy.
The partial derivatives for the single and joint Renyi entropies [3] correspond to the expressions
3HRt (Zr )
g
3a;
v
gVi A^ gvrk
zr eZ ca
a( -1A
3a
1-- 1°G g|gV
qR
zt eZt
\qR.
(22)
3HHRt (Zd,Zr)
3a
(
_ g
(
g vrk V
A(r -1)
z', ,zf eZt
AA
g ViVdkAi
zd eZ 3a
(23)
/ J
1 -fU g g Viv:
HR J z'. ,zf eZt I z; ,zf eZt
\Qr
Note that i ^ k when summing in expressions (18)-(23). In addition, the calculations can be reduced by summing over k from 1 to and over i from k + 1 to
Let us take into account that in expressions (18)-(23) the derivatives 9AJk/9a depend on the accepted model of spatial deformations. Let's represent 9Arik/ 9a in the form
9Aj =9Ai j = 9(zi _zk) j + 9(z-r _zk)j (24)
3a
3j 3a
3j\
3a
3j2
3a
In this case, since the dependence of the brightness zr of pixels ze Ztr on the basic coordinates j of the image Zr. is not known a priori, the derivative 9(z[t _ zk ))9j can only be estimated through finite differences. For example, in the simplest case, assuming the image is two-dimensional j = jji)T, the derivative 9zjt/ 9j1 at the node (j = n, j2 = m) of the pixel grid, assuming the usage of second-order interpolation and a unit grid step, can be estimated as
3j1
zn
j1 _n j2 _m
2
(25)
The derivative 9j / 9a can be found analytically using the adopted model of mutual spatial deformations of images Zd and Zr. In particular, if a special case of the general affine model is used, the similarity model [3] is
ji _ k (h* cos 9 - h*2 sin 9) + hj1 + hj10, j2 _ k (h* sin 9 + h*2 cos 9) + hj2 + hh 0.
(26)
The parameters a = (h/1, h/2, 9, k)t of this model are shifts hj1 and hj2 along the basic image axes, rotation angle 9 and scale factor k. In this case we have
j = j = 1,j = (( cos 9 _ hi sin 9),
9/1 9/2 9k v j1 j2 ;
j _j = j _(sin 9 + h'cos 9),
3j2 3/1 3k
3j1
(27)
99 = k (_h* sin 9 _ h*2 cos 9), 99 = k (_h* cos 9 _ h*2 sin 9),
where h* =(hh _h^); h*2 =(hj2 _ h20); h 0, hj20) are the coordinates of the rotation center.
X
X
z
Thus, all the necessary relations for calculating the stochastic gradient of the Shannon, Renyi and Tsallis MI for the algorithmic implementation of the procedure (2) for estimating the image registration parameters have been obtained.
3. Experimental study of algorithms under noisy conditions
On the base of the obtained expressions, relay stochastic image registration algorithms were synthesized, which were tested on simulated and real images. The simulated images were formed using a wave model [34], the advantage of which is the isotropy of the obtained images, as well as the proximity of the brightness and correlation functions to Gaussian ones. The latter is important for experimental verification of analytical results, since such limitations are often assumed when obtaining them. Satellite images of the optical range were used as real images.
When conducting an experiment (1), a positive-definite diagonal gain matrix with constant learning coefficients was used:
At =
X ,1 0 0 0 0.2 0 0 0
0 X j 2 0 0 0 0.2 0 0
0 0 X9 0 0 0 0.1 0
0 0 0 0 0 0 0.005
(28)
where is measured in degrees. The number of iterations of algorithms is T = 100. The local sample Zt at each iteration is formed equally randomly. The local sample size | for all algorithms is 80 pixels. The sizes of samples Zti and Zt2, ZtmZt2 = Zt are equal. To find the variance ctG of an elementary Gaussian distribution, one of the well-known approaches [33] is used, in which it is determined by several iterations through an estimate ctZ of the variance of a local sample:
2 UÔI -2
= X
je!2z
(z d - zd )2 ^-1
(29)
where zf is the average value of {zjt} .
The objective of the experimental study was a comparative estimation of the effectiveness of algorithms developed on the basis of various types of MI in under noisy conditions. At the same time, in order to be able to analyze the accuracy of the estimates of the registration parameters generated by the algorithms, the deformed image was constructed from the reference one according to the specified deformation parameters and an independent centered Gaussian noise was additionally added to it. The range of the ratio q of image variance to noise variance was studied in the range from 5.5 to 0.5.
The following indicators were used to compare the effectiveness of algorithms synthesized on the basis of different types of MI: noise resistance, error of the vector of estimates of deformation parameters and speed. The noise
resistance of algorithms was defined as the proportion of evaluation failures for a given number of the algorithm implementations. Estimation failure here means the absence of convergence of estimates to the optimal value for a given number of iterations of the algorithm for at least one of the deformation parameters. The error was characterized by the variance of the error of estimates of each of the deformation parameters, as well as the Euclidean mismatch distance [35], which integrally characterizes the vector of parameter estimates. The implementation of the algorithms was estimated by the number of iterations until the convergence of the estimates of the registration parameters was achieved. The hypothesis of achieving parameter convergence to optimal values was considered confirmed if the variance and the average value of the parameter estimation error in the sliding window did not exceed the specified threshold values.
An example of a simulated image is shown in Fig. 1a. Due to the specifics of its formation, its distinctive feature is the absence of relatively high-frequency spatial components, for example, contours. The deformed image obtained using a similarity model with parameters hx=- 3.4, hy = - 4.2, 9 = 5.1, k = 1.07 and also noisy with q = 1 is shown in Fig. 1b.
Cl)
Cj brightness brightness
Fig. 1. Example of original and noisy deformed simulated image and their brightness probability distributions
The brightness values of the original and deformed images are shown respectively in Fig. 1c and Fig. 1d. Due to the linearity of the noise operation, the greater the noise, the closer the brightness distribution is to Gaussian. However, it is worth noting that the distribution mode for the simulated image practically does not change.
The dependence of the proportion of estimation failures on the signal-to-noise ratio for 30 algorithm implementations is shown in Fig. 2a. Here (as well as in Fig. 3, Fig. 4a, Fig. 6 and Fig. 7a) the green curve shows the results of the algorithm synthesized on the basis of Shannon MI, red - Tsallis MI and blue - Renyi MI. It can be seen that Tsallis MI has the least stability among the studied objective functions. For it the failure proportion tends to 100 % already at q = 4. The best stability is shown by the algorithm based on the Renyi MI, in which estimation
failures begin at q < 2. The algorithm based on Shannon MI is some better in this indicator, but significantly inferior to the usage of Renyi MI. The variances of the error in estimating the deformation parameters is shown in Fig. 2b - 2d (Fig. 2b is the dependence of the variance of the error in estimating the shift along one of the coordinate axes on q, Fig. 2c - in estimating the rotation angle, Fig. 2d - in estimating the scale factor), from which it can be seen that the smallest error in estimating all parameters is given by the algorithm based on the Renyi MI, and the largest is based on the Tsallis MI. At the same time, the loss in variance for estimates of all parameters is more than 1.5 times. The algorithm based on Shennon MI loses about 20 %. The vertical dotted line on the graphs means that the estimation failure with a further increase in noise changing. The vertical dotted line on the graphs means the estimation failure with a further increase in noise.
a)
c)
d)
Fig. 2. The proportion of estimation failures and the variance ofparameter estimates for different types of MIfor simulated image
5 3 1
signal-to-noise ratio Fig. 3. The number of iterations of algorithms before the convergence of the registration parameter estimates
The dependence of the number of algorithm iterations that necessary for the convergence of the estimates of the registration parameters on the signal-to-noise ratio is shown in Fig. 3. It can be seen that the difference in the results for the algorithms based on Shannon and Renyi MI does not exceed the experimental error. The algorithm based on Tsallis MI requires 1.3 - 1.5 times more iterations to achieve the convergence domain of the parameters.
a)
cj displacement along axis y ¿¡j displacement along axis i
Fig. 4. The dependence of MI on the displacement for simulated image
a)
c)
d)
brightness
Fig. 5. Example of the original and noisy deformed real images and their brightness probability distributions
The obtained results, reflecting a lower probability of estimation failures and a greater accuracy of estimates of the registration parameters of the algorithm based on the MI, are also confirmed by the analysis of the dependencies of various types of MI on the displacement along one of the coordinate axes (Fig. 4a). The figure shows the curves normalized to the maximum for three types of MI, obtained on the base of the image of Fig. 1a. It can be seen that the decline rate of the numerical value of Renyi MI with increasing shift is less than that for Shannon and Tsallis MI. This, in the conditions of geometric deformations of images and noise, provides a larger amount of information for the algorithm and, accordingly, better results.
Examples of Shannon, Tsallis and Renyi MI curves with different signal-to-noise ratios are shown in Fig. 4b - Fig. 4f. Here (as well as in Fig. 7b) the curve 1 corresponds to the ratio of 5.5, the curve 2 - to the ratio of 3, the curve 3 - to the ratio of 1. It can be seen that at the boundary of the shift range, the difference in values at q = 5.5 and q = 1 for Renyi is less than 1.1 times, whereas for Shannon MI is less than 1.7 times, for Tsallis MI is less than 2.6 times.
An example of a real satellite image of the Volga River water area of the optical range is shown in Fig. 5a. In contrast to the simulated image of Fig. 1a, this image is more high-frequency, having many contour lines. The deformed and noisy image at q = 1 is shown in Fig. 5b. To compare the results, the deformation parameters, as well as the parameters of the algorithms, are selected the same as in the previous example. The brightness values of the original and deformed images are shown respectively in Fig. 5c and Fig. 5d. At the same time, the brightness distribution of the real image is very different from the Gaussian one. The addition of noise significantly changes the distribution, bringing it somewhat closer to normal, which also changes the mode of distribution.
The graphs of estimation failures proportion are shown in Fig. 6a, and the dependences of the variance of the Euclidean mismatch distance of the estimation vector on the signal-to-noise ratio are shown in Fig. 6b. In general, the obtained results confirm the conclusion about the greater efficiency of the algorithm on the basis of the Re-nui MI. However, we note that in the real image, the sta-
0.6
bility of the visualis to additive noise is slightly higher than in the simulated one. In addition, in terms of estimation failure proportion and in terms of the accuracy of the estimates, the usage of Shannon MI gives similar results. This can be explained by the fact that a higher-frequency image provides, on average, less information content of the local sample due to a smaller radius of correlation of the autocorrelation function. At the same time, in this case, with a given size of local sampling, the probabilities of changes in the estimates in procedure (1) in the direction of their improvement are close. However, for the Tsallis MI, this sample size is not enough to compensate for a larger decline in the MI with the same parameters of geometric mismatch of the linked images. This is confirmed by the graph of the dependence of the number of iterations of the algorithms to the convergence of the estimates of the binding parameters (Fig. 6c). It can be seen that the required number of iterations has increased by about half for all algorithms, however, according to the previous algorithm based on the Tsallis MI, it is requared 1.5 times more iterations to achieve convergence of parameters.
i.o
o °-6
c
o
C | 0.2 o.
Jv
0.4
0.2
a)
5 3.5 1.5 signal-to-noise ratio
/
h
J
■•2
5 1
65
55
45
35 3
N
signal-to-noise ratio cj
5 3 1
signal-to-noise ratio
Fig. 6. Parameters of algorithm efficiency for different types of MIfor the real image
a)
displacement along axis y ¿j displacement along axis
Fig. 7. The dependences of MI on the displacement for real images
The normalized dependences of the studied MI on the displacement along one of the coordinate axes, similar to Fig. 4a, are shown in Fig. 7a. It can be seen that for the given real image they have a significantly larger amount of decline than for the simulated one, which indicates a smaller amount of information with the same amount of deformation. This leads to a lower probability of changing the estimates in procedure (2) in the direction of their improvement. As a result, the number of iterations required for convergence of algorithms increases. An example of Shannon MI curves with a different signal-to-noise ratio is shown in Fig. 7b. The colors of the curves correspond to Fig. 4b. At the boundary of the specified displacement range, the difference for Shannon MI is
approximately 1.2 times, for Tsallis MI is approximately 1.3 times, for Renyi MI is approximately 1.1 times. Thus, this experiment also confirmed the high resistance of the Renya MI to additive noise.
Conclusion
The synthesized on the basis of the obtained analytical expressions for the stochastic gradient of mutual information of Shannon, Renyi and Tsallis, the recurrent algorithms for estimating the image registration parameters showed high efficiency. In particular, when the sizes of the studied images were 500x500 pixels, the total number of pixels used in all estimation iterations did not exceed 5000, which is 2 % of their total number. When the size of the local sample increases, the accuracy and stability of the registration estimates generated by the algorithms increase.
Approbation of the algorithms on simulated and real images under noisy conditions showed that the algorithm based on the Tsallis MI showed less efficiency (both in the proportion of estimation failures and in the dispersion of the Euclidean mismatch distance). In general, when estimating the deformations of images with a brightness distribution close to Gaussian, there is less resistance to noise than in higher-frequency images with a complex distribution of pixel brightness. In particular, in the ex-
amples on simulated images for the algorithm based on the Tsallis MI, 100 % of estimation failures were observed at a signal-to-noise ratio of about 2, while on real satellite images, the failures with the same noise amounted to about 30 %. Note also that, compared to algorithms based on Shannon and Renyi MI, the usage of Tsallis MI requires 1.3 - 1.4 times more iterations to achieve convergence of registration estimates.
On the basis of the conducted studies, it can be concluded that under conditions of increased a priori uncertainty, in particular, when registration of multispectral and multitemporal images, stochastic algorithms based on Renyi and Shannon MI can be recommended. At the same time, the algorithm based on the Rynyi MI provides a slightly higher accuracy and convergence rate of the registration estimates, but it also implies a greater computational complexity.
Acknowledgment
The work was done by Leading Research Center "National Center for Quantum Internet" of ITMO University supported by Russian Science Foundation (project No. 24-21-00484) and the grant "Fundamental and Applied Problems of Photonics" No. 621317 of ITMO University.
References
[1] Azam MA, Khan KB, Ahmad M, Mazzara M. Multimodal medical image registration and fusion for quality enhancement. Computers, Materials & Continua 2021; 68(1): 821-840. DOI: 10.32604/cmc.2021.016131.
[2] Yu G, Zhao S. A new feature descriptor for multimodal image registration using phase congruency. Sensors 2020; 20(18): 5105. DOI: 10.3390/s20185105.
[3] Gonzalez RC, Woods E. Digital image processing. London: Pearson; 2018.
[4] Maes F, Vandermeulen D, Suetens P. Medical image registration using mutual information. Proc IEEE 2003; 91(10): 1699-1722. DOI: 10.1109/JPROC.2003.817864.
[5] Fida AD, Gaidel AV, Demin NS, Ilyasova NY, Zamytskiy EA. Automated combination of optical coherence tomography images and fundus images. Computer Optics 2021; 45(5): 721-727. DOI: 10.18287/2412-6179-CO-892.
[6] Dementiev VE, Magdeev RG, Tashlinskii AG. Detecting anomalies in temporal image sequences based on object identification by the stochastic gradient adaptation. 2021 Int Conf on Information Technology and Nanotechnology (ITNT) 2021: 1-5. DOI: 10.1109/ITNT52450.2021.9649175.
[7] Kamaev AN, Karmanov DA. Visual navigation of an autonomous underwater vehicle based on the global search of image correspondences. Computer Optics 2018; 42(3): 457-467. DOI: 10.18287/2412-6179-2018-42-3-457-467.
[8] Frolov VN, Tupikov VA, Pavlova VA, Alexandrov VA. Informational image fusion methods in multichannel optoelectronic systems [In Russian]. Izvetiya Tul'skogo Gosudarstvennogo Universiteta. Technicheskie Nauki 2016; 11(3): 95-104.
[9] Zhang H, Xu R. Exploring the optimal integration levels between SAR and optical data for better urban land cover mapping in the Pearl River Delta. Int J Appl Earth Obs Geoinf 2018; 64: 87-95. DOI: 10.1016/j.jag.2017.08.013.
[10] Wan L, Xiang Y, You H. A post-classification comparison method for SAR and optical images change detection.
IEEE Geosci Remote Sens Lett 2019; 16(7): 1026-1030. DOI: 10.1109/LGRS.2019.2892432.
[11] Magdeev RG, Tashlinskii AG. A comparative analysis of the efficiency of the stochastic gradient approach to the identification of objects in binary images. Pattern Recogn Image Anal 2014; 24(4): 535-541. DOI: 10.1134/S1054661814040130.
[12] Magdeev RG, Tashlinskii AG. Efficiency of object identification for binary images. Computer Optics 2019; 43(2): 277-281. DOI: 10.18287/2412-6179-2019-43-2-277-281.
[13] Marcos D, Hamid R, Tuia D. Geospatial correspondences for multimodal registration. Proc IEEE Conf on Computer Vision and Pattern Recognition 2016: 5091-5100.
[14] Park H, Bland PH, Brock KK, Meyer CR. Adaptive registration using local information measures. Med Image Anal 2004; 8(4): 465-473. DOI: 10.1016/j.media.2004.03.001.
[15] Can A, Stewart C. A feature-based, robust, hierarchical algorithm for registration palm of images of the curved human retina. IEEE Trans Pattern Anal Mach Intell 2002; 24(3): 347-363. DOI: 10.1109/34.990136.
[16] Maintz JBA, Viergever MA. A survey of medical image registration. Med Image Anal 1998; 2(1): 1-36. DOI: 10.1016/s1361-8415(01)80026-8.
[17] Wu J, Cui Zh, Sheng VS, Zhao P, Su D, Gong Sh. A comparative study of SIFT and its variants. Meas Sci Rev 2013; 13(3): 122-131. DOI: 10.2478/msr-2013-0021.
[18] Bay HV, Ess A, Tuytelaars T, Gool LV. SURF: Speeded up robust features. Comput Vis Image Underst 2008; 110(3): 346-359. DOI: 10.1016/j.cviu.2007.09.014.
[19] Alexanin AI, Morozov MA, Fomin EV. The problems of image superimposition with one-pixel accuracy [In Russian]. Sovremennye Problemy Distantsionnogo Zondi-rovaniya Zemli iz Kosmosa 2019; 16(1): 9-16. DOI 10.21046/2070-7401-2019-16-1-9-16.
[20] Zlobin VK, Kolesnikov AN, Kostrov BV. Correlation-extreme methods of combining aerospace images [In Russian]. Vestnik of Ryazan State Radio Engineering University 2011; 37(3): 12-17.
[21] Tashlinskij A.G, Tikhonov VO. Errors analysis technique for pseudogradient measurement of multidimensional processes parameters [In Russian]. Izvestiya Vysshikh Uchebnykh Zavedenij: Radioelektronika 2001; 44(9): 75-80.
[22] Tashlinskii AG, Safina GL, Kovalenko RO, Ibragimov RM. Usage of mutual information as similarity measures for stochastic binding images. 2021 International Conference on Information Technology and Nanotechnology (ITNT) 2021: 1-6. DOI: 10.1109/ITNT52450.2021.9649386.
[23] Tashlinskii A, Ibragimov R, Safina G. Application of Renyi mutual information in stochastic referencing of mul-tispectral and multi-temporal images. 2022 VIII Int Conf on Information Technology and Nanotechnology (ITNT) 2022: 1-6. DOI: 10.1109/ITNT55410.2022.9848648.
[24] Voronov SV, Tashlinskii AG. Efficiency analysis of information theoretic measures in image registration. Pattern Recogn Image Anal 2016; 26(3): 502-505. DOI: 10.1134/S1054661816030226.
[25] Tsypkin YaZ. Information theory of identification [In Russian]. Moscow: "Fizmatlit" Publisher; 1995.
[26] Tashlinskii AG, Safina GL, Voronov SV. Pseudogradient optimization of objective function in estimation of geometric interframe image deformations. Pattern Recogn Image Anal 2012; 22(2): 386-392. DOI: 10.1134/S1054661812020174.
[27] Kovalenko RO, Tashlinskii AG. Optimization of the histogram intervals number which approximate brightness probability distributions in stochastic image alignment based on in-
formation similarity measures. 2022 24th Int Conf on Digital Signal Processing and its Applications (DSPA) 2022: 1-5. DOI: 10.1109/DSPA53304.2022.9805456.
[28] Minkina GL, Samoilov MYu, Tashlinskii AG. Choice of the objective function for pseudogradient measurement of image parameters. Pattern Recogn Image Anal 2007; 17(1): 136-139.
[29] Shannon CE, Weaver W. The mathematical theory of communication. Urbana: University of Illinois Press; 1998.
[30] Cvejic N, Canagarajah CN, Bull DR. Image fusion metric based on mutual information and tsallis entropy. Electron Lett 2006; 42(11): 626-627. DOI: 10.1049/iel:20060693.
[31] Renyi A. On measures of entropy and information. Proc Fourth Berkeley Symposium on Mathematical Statistics Probability 2006; 4.1: 547-561.
[32] Mussa HY, Mitchell JBO, Afzalb AM. The Parzen Window method: In terms of two vectors and one matrix. Pattern Recogn Lett 2015; 63: 30-35. DOI: 10.1016/j.patrec.2015.06.002.
[33] Viola P, Wells WM. Alignment by maximization of mutual information. Proc IEEE Int Conf on Computer Vision 1995: 16-23. DOI: 10.1109/ICCV.1995.466930.
[34] Krasheninnikov VR. Fundamentals of the image processing theory [In Russian]. Ulyanovsk: ULSTU Publisher; 2003.
[35] Tashlinskii, A.G. Optimal euclidean distance of estimates mismatch at stochastic gradient estimation of image interframe geometric deformation parameters [In Russian]. In-formatsionno-Izmeritelnyye i Upravlyayushchiye Sistemy 2018; 11: 33-39.
Authors' information
Alexander Grigorevich Tashlinskii, (b. 1954) graduated from Ulyanovsk Polytechnic Institute in 1977 (presently, Ulyanovsk State Technical University), majoring in Radio Engineering. Doctor of Technical Sciences, Professor, Head of Research Centre for Digital Signal and Image Processing «Signal» in Ulyanovsk State Technical University. Research interests are computer optics, image processing, computer design, and digital photography. E-mail: tag@Mlstu.ru
Galina Leonidovna Safina, (b. 1983) graduated from Ulyanovsk State Technical University in 2005, majoring in Applied Mathematics. PhD in Technical Sciences, Docent, Head of the Department of Fundamental Education in branch of Moscow State University of Civil Engineering in Mytishchi, Associate Professor of the Department of Computer Science and Applied Mathematics in Moscow State University of Civil Engineering. Research interests are image processing, programming, and stochastic gradient identification. E-mail: minkinag@mail.ru
Radik Maratovich Ibragimov, (b. 1995) graduated from Ulyanovsk State Technical University in 2019, majoring in Radio Engineering. Master of Engineering and Technology, post-graduate student, junior researcher of the Department of Radio Engineering in Ulyanovsk State Technical University. Research interests are digital image processing, programming, mathematical modeling, and stochastic gradient identification.
E-mail: ibragimow.it@gmail.com
Received May 04, 2023. The final version - August 11, 2023.
