A solution method for image distortion correction model based on bilinear interpolation Текст научной статьи по специальности «Компьютерные и информационные науки»

A solution method for image distortion correction model based on bilinear interpolation

J. Li1, J. Su1, X. Zeng1

1College of Information Science & Engineering, Hunan International Economics University, Changsha 410205, China

Abstract

In the process of the image generation, because the imaging system itself has differences in terms of nonlinear or cameraman perspective, the generated image will face the geometric distortion. Image distortion in general is also a kind of image degradation, which needs the geometric transform to correct each pixel position of the distorted images, so as to regain the original spatial relationships between pixels and the original grey value relation, and which is also one of important steps of image processing. From the point of view of the digital image processing, the distortion correction is actually a process of image restoration for a degraded image. In image processing, in terms of the image quality improvement and correction technology, namely the image restoration, with the wide expansion of digital image distortion correction processing applied, the processing technology of the image restoration has also become a research hotspot. In view of the image distortion issue, this paper puts forward the image distortion correction algorithm based on two-step and one-dimensional linear gray level interpolation to reduce the computation complexity of the bilinear interpolation method, and divide the distorted image into multiple quadrilaterals, and the area of the quadrilateral is associated with the distortion degree of the image in the given region, and express the region distortion of each quadrilateral with the bilinear model, thus determining parameters of bilinear model according to the position of the quadrilateral vertex in the target image and the distorted image. Experiments show that such algorithm in this paper can meet the requirements of distortion correction of most lenses, which can accurately extract the distorted edge of the image, thus making the corrected image closer to the ideal image.

Keywords: image distortion, bilinear interpolation, correction model.

Citation: Li J, Su J, Zeng X. A solution method for image distortion correction model based on bilinear interpolation. Computer Optics 2019; 43(1): 99-104. DOI: 10.18287/2412-6179-2019-43-1-99-104.

Acknowledgments: This work was supported in part by the Scientific research project in Hunan education department (16C0905, 17C0896).

Introduction

In the process of generation and transmission, it is likely that the image will face distortion, such as color cast, vagueness, geometric distortion and geometric tilt and so on.

Image distortion refers to such deformation as extrusion, stretching, migration and distortion etc. of the relative to the reference system (actual location on the ground or topographic map) and such geometrical position of the image pixels is generated during the imaging process of image pixels, thus making the image geometric location, size, shape, orientation and so on change. Distortion widely exists in various kinds of actual engineering application, especially in the remote sensing and remote sensing field. At present, almost all fields involving the application of scanning and imaging need the distortion correction [1]. The distortion correction by digital image technology is mainly used in two areas: one is the correction of medical endoscope image, which aims to measure the size of the lesion area of the image, the other is the machine vision field, which aims to improve its visual positioning accuracy to correctly complete the actions. Geometric distortion includes barrel distortion, pincushion distortion and geometric tilt etc. In fact, it is often necessary to calibrate the image of another intake mode based on a certain image, in order to correct its geometric distortion, which is called geometric distortion correction of image. Current methods

adopted to correct the image distortion include two main steps: the spatial transform, rearrange the image pixels in the image plane to restore the original spatial relations; gray level interpolation: endow pixels after spatial transform with relative gray value to restore the grey value of the original location [2, 3].

In terms of researches on the image distortion correction, in 1966, the research results of CCD image distortion correction published by Hallert firstly adopt the least square method to realize the data processing [4]. In 1975, Faig considered various factors causing distortion during the CCD imaging, set up relatively sophisticated imaging model and solved by using the nonlinear optimization method [5]. Hideaki firstly proposed the grid template method in 1995 and simulated the mapping relationship between the actual pixels and distorted pixels by using the intersections between straight lines of the template, and thus completing the image distortion correction [6]. Smith proposed the template image and checkerboard template of hexagonal lattice [7]. These methods are all based on a specific template, once the template is sampled, it will not be meaningful unless images taken in the specific environment sampled by such template are corrected. Correction method based on parameters doesn't require the template, and the distortion correction can be implemented only with some specific

parameters (such as the focal length, etc.). Miroslav and so on use the maximum entropy of the linear model during the image collection to calculate parameters of the distortion correction model [8]. Devernay creates a multimode model, and also proposes a new correction algorithm through such model, and at present such algorithm has been applied in many fields [9]. In addition, Juho Kannala further studies the imaging principle of all kinds of lens to put forward the distortion correction algorithm which is able to adapt to a variety of lens [10]. Although these algorithms highly improve the distortion correction algorithm in time and space complexity, which makes great contribution for the distortion correction algorithm in the actual application, the image distortion correction algorithms proposed at present still have many problems. For example, when accessing images, the eccentric distortion will occur and the radius mapping will face the radius discontinuity, thus resulting in the image default case. Therefore, with the study on the existing image distortion correction algorithm, the algorithm is improved based on the analysis of these distortion correction algorithms to improve the correction effect and the efficiency of the existing algorithm, thus having certain practical values.

This paper firstly aims at the problems prone to geometric distortion in accessing images, through the indepth analysis of specific reasons for the image geometric distortion, studies the principle of image geometric distortion and the geometric distortion correction algorithm. The bilinear interpolation is a kind of common gray level interpolation method with high accuracy and without gray scale discontinuity problem when compared with nearest neighbor interpolation, but its calculation process is complicated with big calculation amount, thus affecting the real-time distortion correction. Therefore, this paper proposes the image distortion and correction algorithm with two-step and one-dimensional linear gray level interpolation to reduce the computing complexity of the bilinear interpolation method, and also realize the image geometric distortion correction. Correction experiments prove the effectiveness of the algorithm in this paper, and show such algorithm has higher recovery efficiency and good processing effect when compared with the traditional correction algorithm.

Causes of image distortion and geometric correction cause analysis of image distortion

In the image access process, because the imaging system itself is influenced by such factors as nonlinear, shooting angle and other factors, the image geometric distortion will occur. Geometric distortion is divided into the system distortion and the non-system distortion. System distortion is regular and can be predicted, and the non-system distortion is random. When making the quantitative analysis of the image, the precise geometric correction will be first implemented in the distorted image so as not to affect the accuracy of quantitative analysis.

Distortion is one imaging system aberration, and the ideal imaging system is not only clear in the imaging, but

also satisfies the object image similarity relation. In the case that the system is able to achieve the clear imaging with dissimilar object image, the dissimilar degree is measured by the distortion. Distortion process describes the mapping relationship between the ideal image point position and the actual image point position, namely, (x, y, z) =f(xd,yd, zd), and the key of the distortion correction lies in determining such mapping relationship. The geometric distortion of images collected by the machine vision system is nonlinear, so such nonlinear transform can be represented by the polynomial transform between coordinates:

n n-i

Xd = XX aijX'y1,

i=°7=0 (1)

n n-l

yd =XXbiiX‘yi.

i =0 i=0

Correction accuracy is associated with n, the greater n is, the higher the correction accuracy is, but with the increase of n, the amount of correction calculation will increase sharply [11, 12].

Geometric distortion correction

First study the address mapping relations under rectangular coordinate system by using the matrix coordinate system. Preset f(x, y)] is the original image and [f(u, v)] is the image after the distortion. Average distortion is not just simple changes and tilt of the aspect ratio. What we usually encounter is the geometric distortion of the spatial warping type, here we adopt the quadratic polynomial to simulate, and the distortion can be expressed as,

u = a0 + a x + a2 y + a3 x2 + a4 xy + a5 y2, (2)

v = b0 + bxx + b2y + b3 x2 + b4 xy + b5y2. (3)

As well, once the estimation of the parameter (ai, bi) can be obtained, the distortion function can be known, and the corrected spatial warping mapping can be achieved by the above polynomial transform [13].

(2) and (3) are the binary quadratic equation with 6 parameters. Once six pairs of corresponding points can be respectively taken from the distorted image and the corrected image, the estimation of the parameter (ai, bi) can be achieved by the solution of equations. In theory, the more pairs of the corresponding points are taken, the more accurate the estimation of the parameter (ai, bi) can be achieved is. Preset m is the corresponding point, so it can be expressed by the vector as:

U‘ [U1 ,u2, ..,Um ]V‘ = [V1,V2, .., Vm ]. (4)

1 x1 y1 x2 x y1 y2'

1 x2 y2 x22 x2 y2 y22

A = . (5)

1 xm ym xm xmym ym _

Its coefficient is af = [a0, ab..., am], b = [b0, bb..., bm].

The following Fig. 1 and Fig. 2 are images with distortion.

Fig. 1. Image 1 with distortion

Fig. 2. Image 2 with distortion

Bilinear spatial transform and interpolation method

For one image f with the image pixel coordinates of (x, y), after the geometric distortion, an image g is generated with the image pixel coordinates of (x', y'), and such transform can be expressed as.

x’ = r (x, y),

(6)

y = s( x, y).

Here, r(x, y) and s(x, y) are the spatial transform, which generates the geometric distortion image s(x , y ). For example, if r(x, y) = x / 2, s(x, y) = y / 2, the distorted image is simply the size contraction of f(x, y) by half on two spatial directions.

If r(x, y) and s(x, y) are known in the analytic study, in theory the reverse transform can be used to restore f(x, y) from the distorted image g (x', y). However, in practice, it is impossible to formulate the analytic function r(x, y) and s(x, y) set, and these analytic functions describe the geometric distortion in the whole image plane. The most commonly used way to overcome this difficulty is to express the pixel spatial relocation by the control point method, and these points are the subset of pixels, and their positions in the input (distorted) and output (corrected) image are accurately known. As long as you can find their corresponding relationship, and realize the correction for each pixel, finally the whole image can be corrected. The 4 vertices are corresponding points of junction. Assume that

the geometric deformation of the quadrilateral area is modeled in bilinear equation, namely:

r (x, y) = C1 x + c2 y + c3 xy + c4, s(x, y) = c5x + c6y + c7 xy + c8.

From formula (7), we can infer:

x' = c x + c2 y + c3 xy + c4, y' = c5 x + c6 y + c7 xy + c8.

Because there are total eight known points, and these equations can solve eight coefficients of c, i = 1, 2,...8. These coefficients form the geometric distortion model used to transform all pixels in the quadrilateral area, and such quadrilateral is defined by the derived coefficient. Usually enough points are needed to generate the quadrilateral set of the whole image with each quadrilateral having its own set of coefficients [14].

Bilinear interpolation method realizes the linear interpolation in two directions by using four adjacent pixel grays of points to be solved. The following derived calculation formula has the pixel gray value to be solved.

For (x, y+v), we get

f (x y + v) = [ f (x y +1)-f (x y)] v + f (x y).

For (x +1, y+v), we get f (x +1, y + v ) =

= [f (x+1 y+1)-f (x+1 y)]v + f (x+Г y).

For (x +u, y+v), we get f (x+u, y+v) = [f (x+1, y+v) --f (x, y+v)]u + f (x, y+v) =

= (1 - u)(1 - v) f (i, j) +

+(1 - u) vf (i, j+1)+u(1 - v) f (i +1, j) + +uvf (i +1, j +1).

This f (x +u, y+v) value stands for the gray value of (x, y) of the corrected image.

This method is more complex with large amount of calculation, but there is no gray discontinuity defect [15].

Algorithm realization & experimental test

Two-step & one-dimensional linear gray level interpolation method

Assume the quadrilateral mapped in the distorted image by number i rectangle area Ii(xi, yi) of the lattice target surface as Di (xdi, ydi), so the distortion model for such region is:

Dt (xdi, ydi) = f a + ад- + a2,y,- + aз,■x,■y,■,

b0i + b^i + Й2,У,- + ЬзЛ-у, ).

Based on vertex coordinates (x,y)(i = 1~4) of the rectangular area in the lattice target surface, and vertex coordinates (xdi, ydi)(i = 1~4) of the quadrilateral region in the corresponding distorted image, the coefficient of distortion model can be determined from the form (10) and form (11).

1 x1 y1 x y1 a0i xd1 1 x1 y1 x1 y b0i yd1

1 x2 y 2 x2 y2 an xd 2 1 x2 y 2 x2 y2 bn yd 2

= . (10) =

1 x, y3 x3 y3 a2i xd 3 1 x3 y3 x3 y3 b2i yd 3

1 x4 y 4 x4 y4 _ _ a2i _ xd 4 1 x4 y 4 x4 y4 _ bn _ _yd4 _

(11)

After the geometric correction towards the distorted image by the distortion model, the gray value of each pixel of the output image needs to be determined again. Here we adopt the backward mapping method to determine the gray value of the output image. The so-called backward mapping refers to the successive mapping of the output pixel into the distorted image collected in the visual system, and the corresponding gray level is determined by realizing the bilinear interpolation within 4x4 neighborhood around the mapping point. The specific principle is as follows.

Assume (xby{), (x2,y1), (xby2), (x2,y2) are the four points adjacent to the interpolating point (x, y), in which xi < x < x2, yi < y <y2, the gray value respectively are g1,g2,g3,g4, the bilinear function is defined as,

g (x, y) = c1 x + c2y + c3 xy + c4. (12)

If fixed variable x (or y), the function g(x,y) and variable y(x) forms the linear relationship.

Use the separability of two-dimensional bilinear function, namely the two-dimensional bilinear function can be seen as the product of two one-dimensional linear functions, to realize the two-dimensional bilinear interpolation algorithm by dividing into two-step onedimensional linear interpolation: first achieve the onedimensional linear interpolation along the horizontal direction, and then realize one- dimensional linear interpolation along the vertical direction, namely,

gi2 =b-(g2 -gi) +gi, g34 =Xx(g4 -g3) +g3, g = gi2 + Xy(g34 - gi2 ) .

From form (13) and (14), we can get, g = XxXy(g1 +g4 -g3-g2) +

+Xx(g2 - g1) + (g3 - g1) + Xyg1.

In which Xx = x - x1, Xy = y-y1, and Xxe[0, 1], Xye[0, 1].

Form (15) is same with the result of directly achieving two-dimensional bilinear interpolation, but the two-step

(13)

(14)

(15)

one-dimensional linear gray interpolation greatly improves the computational efficiency, thus increasing the real-time distortion correction.

Analysis of experimental results

Distortion correction is divided into two steps. The first step is to achieve the geometric transform of the pixel coordinate space towards the original image, and the aim is to make the pixels in the correct position, the second step is to again determine the gray value of the new pixels. Because after the above coordinate transform, some pixels may be squeezed together and sometimes spread out, thus making corrected pixels do not fall on the discrete integer coordinate, so it’s necessary to determine the gray value of these pixels. Implement the proposed distortion correction algorithm in Matlab environment programming, after establishing the distortion model, correct the distorted image with such a model. The following Fig. 4 and Fig. 5 show the process to determine the distortion model by using the standard lattice image.

The above Fig.4 and Fig.5 show respectively the collected original image and the results after the distortion correction. In which, Fig.4 (a) and Fig.5 (a) are the collected original distorted images, and the distortion are mainly expressed in the image pixel displacement, thus the distortion and deformation of the objects in the images occur, and such images face the barrel distortion. In the process of image distortion correction, the correction parameters are first obtained. First of all, choose apparent feature points as the control points between distorted image and standard image, and the number is determined by undetermined coefficients. Control points are one-to-one corresponding in the distorted image and standard image. Through the determination of the corresponding relation of these control points, the mapping relationship between distorted image and standard image can be established. The control points are artificially selected, and their coordinates are also read artificially. Secondly, establish the geometrical coordinate correction model based on the measured certain number of control points, because there

exist errors, the control points of the distorted image and control points of the standard image belong to a kind of approximate correspondence rather than complete

correspondence. Finally, according to the mapping relationship, correct coordinates of each pixel of the distorted image.

(а) 'Ey (c) I

Fig. 4. Distorted image 1 correction: image with distortion (a), distortion correction of this algorithm (b), traditional algorithm (c)

(a) \

(b)

(c)\

Fig. 5. Distorted image 2 correction: image with distortion(a), distortion correction of this algorithm (b), traditional algorithm (c)

There are many methods to evaluate the accuracy of calibration. In this paper, the average error and root mean square error are used to compare the correction effect. The statistical results are shown in Tab.1. The average distortion rate error is the residual distortion rate corrected by the algorithm, which reflects the correct degree of image restoration. It can be seen from the table that the errors of the coordinates of the grid points in both horizontal and vertical directions are improved, and this algorithm has a significant decrease in the ratio of the error before the improved algorithm. The root mean square error reflects the difference program between the computed imaging coordinates and the actual imaging coordinates. The more different the difference is, the better the effect of the original image, and the improved algorithm is obviously improved than the traditional algorithm.

In the following Tab. 1,

Mean error =

Zlf (X)- y>

, i = 1,2,3...,n .

Root mean square error

Z

(f (x,) - y )2

—11/2

n

All in all, Fig. 4b and Fig. 5b are the images after the distortion correction, and the images distortion are effectively corrected without emptiness and information leakage, and without color distortion, thus verifying the validity of the proposed algorithm in this paper. And Tab.1 indicate that the algorithm in this paper makes the image restoration more close to the ideal image.

Conclusion

In the process of imaging, the geometrical position of image pixels will face such deformation as extrusion,

stretching, migration and distortion etc, make the image geometric location, size, shape, orientation and so on change. The image geometric distortion can be eliminated by geometric correction. Bilinear interpolation method is a kind of common gray level interpolation method when compared with nearest neighbor interpolation, such algorithm is highly precised without gray discontinuity, but its calculation process is complicated with large amount of calculation, which will influence the real-time distortion correction. This paper, based on the separability of the two-dimensional bilinear function, puts forward a kind of two-step and one-dimensional linear gray level interpolation image distortion correction algorithm.

Table 1. Error comparison of before and after correction

Direction Mean distortion error Coordinate RMSE/ Pixel

Algorithm in this paper Horizontal 3.26% 103.4839

Vertical 2.41% 67.3711

Traditional algorithm Horizontal 5.53% 186.8235

Vertical 4.76% 133.1276

References

[1] Geoffroy O, Saint-Martin D, Olivie DJL, Voldoire A,

Bellon G, Tyteca S. Transient climate response in a two-layer energy-balance model. Part I: Analytical solution and parameter calibration using CMIP5 AOGCM experiments. Journal of Climate 2013; 26(6): 1841-1857. DOI:

10.1175/JCLI-D-12-00195.1.

[2] Chen MZQ, Zhang L, Su H, Chen G. Stabilizing solution and parameter dependence of modified algebraic riccati equation with application to discrete-time network synchronization. IEEE Transactions on Automatic Control 2016; 61(1): 228-233. DOI: 10.1109/TAC.2015.2434011.

[3] Yadav S, Shire SJ, Kalonia DS. Viscosity behavior of high-concentration monoclonal antibody solutions:

Correlation with interaction parameter and electroviscous effects. J Pharm Sci 2012; 101(3): 998-1011. DOI: 10.1002/jps.22831.

[4] Zhou F, Cui Y, Liu L, Gao H. Distortion correction using a

single image based on projective invariability and separate model. Optik 2013; 124(17): 3125-3130. DOI:

10.1016/j.ijleo.2012.09.038.

[5] Hou J, Li H, Zheng Z, Liu X. Distortion correction for

imaging on non-planar surface using freeform lens. Optics Communications 2012; 285(6): 986-991. DOI:

10.1016/j.optcom.2011.12.014.

[6] Dammann P, Kraff O, Wrede KH, Ozkan N, Orzada S, Mueller OM, Sandalcioglu IE, Sure U, Gizewski ER, Ladd ME, Gasser T. Evaluation of hardware-related geometrical distortion in structural MRI at 7 Tesla for image-guided applications in neurosurgery. Academic Radiology 2011; 18(7): 910-916. DOI: 10.1016/j.acra.2011.02.011.

[7] Kaynig V, Fischer B, Muller E, Buhmann JM. Fully

automatic stitching and distortion correction of

transmission electron microscope images. Journal of Structural Biology 2010; 171(2): 163-173. DOI:

10.1016/j.jsb.2010.04.012.

[8] Melo R, Barreto JP, Falcao G. A new solution for camera calibration and real-time image distortion correction in medical endoscopy-initial technical evaluation. IEEE Transactions on Biomedical Engineering 2012; 59(3): 634644. DOI: 10.1109/TBME.2011.2177268.

[9] Siedlecki D, de Castro A, Gambra E, Ortiz S, Borja D, Uhlhorn S, Manns F, Marcos S, Parel JM. Distortion correction of OCT images of the crystalline lens: Gradient

index approach. Optom Vis Sci 2012; 89(5): 709-718. DOI: 10.1097/OPX.0b013e3182508344.

[10] Ding H, Molloi S. Image-based spectral distortion correction for photon-counting x-ray detectors. Med Phys 2012; 39(4): 1864-1876. DOI: 10.1118/1.3693056.

[11] Zhou F, Cui Y, Gao H, Wang Y. Line-based camera calibration with lens distortion correction from a single image. Optics and Lasers in Engineering 2013; 51(12): 1332-1343. DOI: 10.1016/j.optlaseng.2013.05.010.

[12] Hou J, Li H, Zheng Z, Liu X. Distortion correction for

imaging on non-planar surface using freeform lens. Opt Commun 2012; 285(6): 986-991. DOI:

10.1016/j.optcom.2011.12.014.

[13] Yu G, Li K, Liu Y, Chen Z, Wang Z, Yan R, Klose T, Tang L, Jiang W. An algorithm for estimation and correction of anisotropic magnification distortion of cryo-em images without need of pre-calibration. J Struc Biol 2016; 195(2): 207-215. DOI: 10.1016/j.jsb.2016.06.003.

[14] Tan TL, Sim KS, Tso CP, Chong AK. Contrast

enhancement of computed tomography images by adaptive histogram equalization-application for improved ischemic stroke detection. International Journal of Imaging Systems and Technology 2012; 22(3): 153-160. DOI:

10.1002/ima.22016.

[15] Du Y, Chen D. Suppressing gray-scale elements in topology optimization of continua using modified optimality criterion methods. Computer Modeling in Engineering and Sciences 2012; 86(1): 53-70. DOI: 10.3970/cmes.2012.086.053.

Authors’ information

Jun Li (b. 1980) received the B.S degree in Computer Science and Technology from Hunan Normal University in 2003, and received the M.S. degree in Computer Science and Technology from Hunan University. Now, he is a researcher at College of Computer, Hunan University of International Economics, China. Her research interests include image processing and information security.

Jie Su (b. 1981) received the B.S degree in Tourism and Hospitality from Hunan Normal University in 2003, and received the M.S. degree in Ideological and Political Education from Changsha University of Science and Technology. Now, she is a researcher at college of computer, Hunan University of International Economics, China. Her research interests include regional economics and direction of tourism information management.

Xiliang Zeng (b. 1979) received the B.S degree in Computer Science and Technology in National University of Defense Technology in 2003, and received the M.S. degree in Computer Science and Technology from Hunan University. Now, she is a researcher at college of computer, Hunan University of International Economics, China. Her research interests include image processing and information security.

Received January 26, 2018. The final version - December 09, 2018.