CC BY
82
UDC 621.397.3:519.642.3 DOI: 10.17586/0021-3454-2019-62-4-379-386
SPECTRAL METHOD FOR STABLE ESTIMATING
THE DISTORTION PARAMETERS IN INVERSE PROBLEM OF IMAGE RESTORATION
V. S. Sizikov, A. A. Sergienko, D. A. Kondulukova
ITMO University, 197101, Saint Petersburg, Russia E-mail: sizikov2000@mail.ru
The spectral method is developed for estimating the parameters of the point-spread function (PSF) in the problem of restorating the distorted (smeared, defocused) images. The method is based on the analysis of a spectrum, or the Fourier transform (FT) of a distorted image. This method makes it possible to estimate the PSF parameters: the angle 0 and magnitude A of image smearing, as well as the size p of the image defocusing spot. This enhances the image restoration accuracy. The smeared image spectrum is compressed in the smearing direction, and this makes it possible to estimate 0 and A. The defocused image spectrum is also compressed, and more strongly, the larger the defocusing spot. A new estimates are obtained for the smearing parameters 0 and A using the Nyquist frequency and for the defocusing parameter p using the Bessel function. The results of applying this method to image processing are presented. The developed technique can be used to enhance the accuracy of smeared and defocused image restoration via their mathematical processing by solving the integral equations.
Keywords: image distortions (smearing, defocusing), point-spread function, distortion parameters, Fourier spectrum of distorted image, Nyquist frequency, Bessel function,
Introduction. Various image recording devices (IRDs: digital photo cameras, video cameras, telescopes, microscopes, tomographs, etc.) record images of objects (people, animals, texts, astronomical objects, viruses, car numbers, etc.). In this case, the image may be smeared (due to the device shift or the object movement) or defocused (due to an erroneous focus setting, etc.), and also noisy [1—6].
The distorted image can be restored by mathematical and computer processing [1—10]. Such a problem is usually realized by solving integral equations (IEs). Moreover, as shown in [11], there are two approaches.
In the first approach, a set of one-dimensional integral equations (both convolution type and general ones) is used to eliminate the smear, along which the x-axis is directed, and the y-axis is perpendicular to the smear. In this case, to eliminate defocusing, one two-dimensional IE is used.
In the second approach, both for smear elimination and defocusing elimination, we use the two-dimensional Fredholm IE of the first kind of convolution type [3, 5—8, 10—13]:
ro ro
J J h(x-£,y-q)w(£,q)d^dq = g(x,y) + 8g, (1)
—ro —ro
where h is the point-spread function (PSF) or apparatus function, which is usually spatially invariant (a difference function); w and g are the intensity distribution over the true and distorted images, respectively; Sg is the noise; the axes x and are directed horizontally, whiley and n are vertically downward.
In this paper, we will only use the second approach, i.e. restore both smeared and defocused images by solving equation (1). In the first case, when discretizing the problem, the point-spread function h will be a PSF-quasidiagonal matrix in the form of a narrow strip [6, p. 112]. This is
reflected in the m-function fspecial.m in the IPT package of m-functions of the MatLab system, where the computation of h(x,y) is performed with use of fspecial.m [9, 14].
The problem of solving equation (1) is ill-posed (essentially unstable) [3, 5—7, 10]. Therefore, to solve it, we will use stable methods — the Tikhonov regularization [3, 5—7, 9, 10, 12, 13, 15— 19] and the Wiener parametric filtering [3, 5, 6, 9].
The solution of equation (1) by Tikhonov's regularization method and two-dimensional Fourier
transform (FT) is equal to wa (x, y) = F "1(Wa (ra1, ©2)) or:
rot»
Wa(x, y) = — i i Wa(©1, ©2) e"*®1*^ d©1 d©2 , (2)
where F"1 is denotation of the inverse Fourier transform (IFT), a > 0 is the regularization parameter, and Wa (©1, ©2) is the regularized spectrum (two-dimensional FT) of the solution, equal to
Wa (©1, ©2) g'(©1~©22) G (©.2©2>2 , (3)
\H (©1, ©2)| +a (©2 +©2)p
where p > 0 is the regularization order (usually p = 1, 2 or 3), while H(©1, ©2) and G(©1, ©2) are spectrums (FTs) of functions h(x, y) and g(x, y): H(©1, ©2) = F(h(x, y)), G(©1, ©2) = F(g(x, y)),
where F is denotation of the Fourier transform.
The solution of equation (1) by Wiener's parametric filtering method equals [3, 5, 6]:
wK&= -L H^(©1,©2)G(2©1,©2) e"d©1 d©2 , (4)
4^2 ii IH(©l,©2) |2 +K where K > 0 is a parameter (the NSR in power).
The solution of equation (1) by Tikhonov's regularization (TR) and Wiener's parametric filtering (WPF) is realized in the IPT package of the MatLab system in the form of m-functions decon-vreg.m and deconvwnr.m, respectively.
However, even the TR and WPF methods are very sensitive to errors of the distortion parameters — the smearing values A and 0, as well as p of image defocusing, i.e. to inaccuracies in knowledge of the PSF, or to errors of the nucleus of integral equation (1).
As an illustration, Fig. 1a shows the 389*512*3-pixels image liftingbody.png (converted to a grayscale image), smeared at angle 0 = 72° and smear A = 24 pixels. Fig. 1b shows the image restoration result via solving equation (1) by the WPF method with exact smear parameters 0 = 72° and A = 24 pixels. Furthermore, the m-functions of the system MatLab fspecial.m, imfilter.m (direct problem) and deconvwnr.m (inverse problem) are used [9, 14]. And Fig. 1c shows a restorated image with erroneous values of the smear parameters 0 = 74° and A = 26 pixels. Although 9 and A
*
differ insignificantly from the exact 0 and A, the restoration was inadequate (see Fig. lc) .
(a) (li) (c)
Fig. 1
* Note that the parameter K included in the solution (4) was chosen by visual selection. About ways for selecting the parameters K and a, see [3, 5—7, 12, 15—17], etc.
Fig. 2a presents a similar example with a defocused palette (indexed) 400*318-pixels image kids.tif (converted to a grayscale image).
Fig. 2
In this example, the PSF is a uniform disk of radius p = 10 pixels. Fig. 2b shows the image restoration result via solving equation (1) by the WPF method with exact defocusing parameter p = 10 pixels. And in Fig. 2c, a restored image is given with erroneous value of the defocus parameter (5 = 11 pixels. Although (5 differs little from the exact p, the restoration was unsatisfactory (Fig. 2c).
These and other examples [6, 20] indicate that the method is necessary to enhance the accuracy of the estimation of image distortion parameters, in other words, to increase the accuracy of knowledge of the point-spread function. One of these methods is the spectral method for estimating the PSF [1, 2, 6, 20—23].
In this paper, the spectral method is further developed [6, 20—22], in particular, its verification is performed on a number of distorted images.
The spectrum of a distorted image. Consider a smeared image of two objects: liftingbody and an aircraft (Fig. 1a), as well as a defocused image of two kids (Fig. 2a). We direct the x-axis horizontally, and y-axis vertically down. Then the two-dimensional Fourier transform (FT) or the spectrum G(rax, ra y) of the distorted (smeared or defocused) image g (x, y) equals
ro ro
G(rax, ray) = J J g(x,y)/raxx+rayy) dxdy, (5)
—ro —ro
where rax and ray are the Fourier frequencies directed horizontally and vertically, respectively. Fig.
3 and 4 show moduli | G(ra x, ra y) | of the Fourier spectra of smeared image presented in Fig. 1a and
defocused image presented in Fig. 2a. A continuous FT (5) is calculated through a discrete FT by means of the m-functions fft2.m and fftshift.m.
It has already been noted [20, 22] that the spectrum (5) is significantly different for smeared and defocused images, and this allows to determine the type of distortion (smear or defocusing). The spectrum of a smeared image (Fig. 3) is a central quasi-ellipse and a set of almost parallel lines. The defocused image spectrum in the case when the PSF is a homogeneous disc is a set of ellipses (Fig. 4). In addition, the distortion parameters can be estimated from the spectrum.
I «y Fig. 4
Estimating the smear direction and magnitude. In Fig. 3, we draw the axis of the central quasi-ellipse and axis œ perpendicular to it. According to Fig. 3, we estimate the angle between the horizontal and axis œ to be 9 « 76°.2 and the angle y = 90° -0 « 13°.8 (measured angles). Then 9 = 90°-y, where [20]
y = arctg I
(6)
moreover 0 and y are true (sought) angles. Here r = M/N, where M is the number of rows, while N is the number of columns in image g. From Fig. 3, using several measurements, we determine: V = 17°.92 ± 0.15, 9 = 72°.08 ±0.15, which is close to the exact value of smear angle 9 = 72°.
To determine the smear magnitude A, we measure the ratio Qmax/q1, where q1 и Qmax are the first and last zeros of the spectrum | G(q) | along q (see Fig. 3). Then [20]
A = 2 ^^ . (7)
Q1
By several measurements, we obtain on average: Qmax/Q1 = 11.8, A = 23.6±0.6 pixels,
which is close to the exact value of smear A = 24 pixels.
Using the values 9 = 72°.08 and A = 23.6 found from the spectrum (A is rounded to A = 24), we obtain a restorated image via solving equation (1) by Wiener's parametric filtering (WPF) according to (4) with K = 10"4 using m-function deconvwnr.m. In this case, the point-spread function (PSF) was calculated using m-function fspecial.m [9, 14]. The result of restoration is shown in Fig. 1b.
Estimating the defocusing parameter. Fig. 2a shows the defocused image, and Fig. 4 shows the modulus |G(rox,roy)| of its spectrum. The spectrum is a set of ellipses, as shown in [20] with regard to [1, 2, 23—26], moreover
p = 3.84/q1; 7.02/Q2; 10.16/Q3;13.32/Q4;... , (8)
where 3.84; 7.02; 10.16; 13.32; ... are the ziroes of the first-order Bessel function of the first kind, and q1, q2, q3, q4, ... are the Nyquist frequencies (but not in pixels) corresponding to the semiaxis of each ellipse. Upon discretization, the maximum Nyquist frequency is romax = n both along the horizontal and vertical axes in Fig. 4. Then the frequencies Qi, i = 1, 2, 3, 4,. will equal [20, 22]
Qi = Qirel Qmax = Qirel ^ i = 1 2 3 4 ■ - , (9)
where Qi rel = qJ Qmax are dimensionless ratios.
We obtain: Q1rel = 0.122, q1 = 0.383, p = 10.02; Q2rel = 0.226, q2 = 0.710, p = 9.88, etc.
On the average, p = 9.95 ± 0.07, which is close to the exact value of p = 10 pixels.
Only now, using the value p = 9.95 found from the spectrum and rounded to p = 10, we obtain a reconstructed image via solving equation (1) by the Wiener parametric filtering method according
to (4) with K = 3 x10"4 using m-function deconvwnr.m. The restoration result is shown in Fig. 2b. We see a clear image restoration and it is due to the fact that the spectral method makes it possible to determine the defocusing parameter p almost exactly.
We note that in papers [20, 22, 27] the defocusing variant is considered in the case when the PSF is a Gaussian, and the estimates of the parameter a of the Gaussian are used.
Conclusion. This work confirms the effectiveness of the spectral method for determining the image distortion type (smear, defocusing) and determining the distortion parameters. This makes it possible to determine more exactly the point-spread function (PSF) and restore more exactly an image via solving an integral equation by the WPF or TR method.
This work was supported by the grant MFKTU ITMO (Project No. 617026 „Technologies of cyberphysical systems: control, computations, security").
REFERENCES
1. Bates R.H.T., McDonnell M.J. Image Restoration and Reconstruction, Oxford, Oxford U. Press, 1986, 336 p.
2. Gruzman I.S., Kirichuk V.S., Kosykh V.P., Peretyagin G.I., Spektor A.A. Digital Image Processing in Information Systems, Novosibirsk, 2002, 352 p. (in Russ.).
3. Gonzalez R.C., Woods R.E. Digital Image Processing, 2nd ed., Upper Saddle River, Prentice Hall, 2002, 793 p.
4. Jahne B. Digital Image Processing, Berlin-Heidelberg, Springer-Verlag, 2005, 584 p.
5. Sizikov V.S. Inverse Applied Problems andMatLab, St. Petersburg, 2011, 256 p. (in Russ.).
6. Sizikov V.S. Direct and Inverse Problems of Image Restoration, Spectroscopy and Tomography with MatLab, St. Petersburg, 2017, 412 p. (in Russ.).
7. Tikhonov A.N., Goncharsky A.V., Stepanov V.V. Ill-Posed Problems in Natural Science, Moscow, 1987, pp. 185-195. (in Russ.).
8. Soifer V.A., ed., Methods of Computer Image Processing, Moscow, 2001, 784 p. (in Russ.).
9. Gonsales R.C., Woods R.E., Eddins S.L. Digital Image Processing using MATLAB, New Jersey, Prentice Hall, 2004, 609 p.
10. Hansen P.C., Nagy J.G., O'Leary D.P. Deblurring Images: Matrices, Spectra, and Filtering, Philadelphia, SIAM, 2006, 130 p.
11. Sizikov V.S., Sergienko A.A., Rushchenko N.G. Journal of Instrument Engineering, 2019, no. 1(62), pp. 69-77. (in Russ.).
12. Voskoboinikov Yu.E. Optoelectronics, Instrumentation, and Data Processing, 2000, no. 6(43), pp. 489-499. DOI: 10.3103/S8756699007060015.
13. Sizikov V.S., Ékzemplyarov R.A. Journal of Optical Technology, 2013, no. 1(80), pp. 28-34. DOI: 10.1364/J0T.80.000028.
14. D'yakonov V., Abramenkova I. MATLAB. Processing of Signals and Images, St. Petersburg, 2002, 608 p. (in Russian).
15. Voskoboinikov Yu.E., Mukhina I.N. Optoelectronics, Instrumentation, and Data Processing, 2000, no. 3, pp. 41-48.
16. Donatelli M., Estatico C., Martinelli A., Serra-Capizzano S. Inverse Problems, 2006, vol. 22, pp. 2035-2053. DOI: 10.1088/0266-5611/22/6/008.
17. Donatelli M., Huckle T., Mazza M., Sesana D. Numerical Algorithms, 2016, no. 2(72), pp. 341-361. DOI: 10.1007/s11075-015-0047-x.
18. Sidorov D.N. Methods for Analysis of Integral Dynamic Models: Theory and Applications, Irkutsk, 2013, 293 p. (in Russian).
19. Sidorov D. Integral Dynamical Models: Singularities, Signals and Control, Singapore-London,World Sci. Publ., 2014, 243 p.
20. Sizikov V.S. Journal of Optical Technology, 2017, no. 2(84), pp. 95-101. DOI: 10.1364/JOT.84.000095.
21. Sizikov V.S. Journal of Optical Technology, 2015, no. 10(82), pp. 655-658. DOI: 10.1364/JOT.82.000655.
22. Sizikov V.S., Stepanov A.V., Mezhenin A.V., Burlov D.I., Éksemplyarov R.A. Journal of Optical Technology, 2018, no. 4(85), pp. 203-210. DOI: 10.1364/JOT.85.000203.
23. Vasilenko G.I., Taratorin A.M. Image Restoration, Moscow, 1986, 304 p. (in Russ.).
24. Korn G.A., Korn T.M. Mathematical Handbook for Scientists and Engineers, NY, McGraw-Hill, 1961.
25. Bronshtein I.N., Semendyaev K.A. Mathematical Handbook for Engineers and Students of Technical Colledges, 13th ed., Moscow, 1986, 544 p. (in Russian).
26. Bracewell R.N. The Hartley Transform, NY, Oxford University Press, 1986, 168 p.
27. Antonova T.V. Numerical Analysis and Applications, 2015, no. 2(8), pp. 89-100. DOI: 10.1134/S1995423915020019.
Data on authors
Valery S. Sizikov — Dr. Sci., Professor; ITMO University, Department of Graphic Technologies;
E-mail: sizikov2000@mail.ru
Andrey A. Sergienko — Undergraduate; ITMO University, Department of Graphic Technologies;
E-mail: master-ac@yandex.ru
Daria A. Kondulukova — Undergraduate; ITMO University, Department of Graphic Technologies;
E-mail: dariakondulukova@mail.ru
Поступила в редакцию 21.08.18 г.
For citation: Sizikov V. S., Sergienko A. A., Kondulukova D. A. Spectral method for stable estimating the distortion parameters in inverse problem of image restoration. Journal of Instrument Engineering. 2019. Vol. 62, N 4. P. 379—386.
СПЕКТРАЛЬНЫЙ МЕТОД УСТОЙЧИВОЙ ОЦЕНКИ ПАРАМЕТРОВ ИСКАЖЕНИЙ В ОБРАТНОЙ ЗАДАЧЕ ВОССТАНОВЛЕНИЯ ИЗОБРАЖЕНИЙ
В. С. Сизиков, А. А. Сергиенко, Д. А. Кондулукова
Университет ИТМО, 197101, Санкт-Петербург, Россия E-mail: sizkov2000@mail.ru
Развивается спектральный метод оценки параметров функции рассеяния точки (ФРТ) в задаче восстановления искаженных (смазанных, дефокусированных) изображений. Метод основан на анализе спектра, или преобразования Фурье (ПФ) искаженного изображения. Данный метод дает возможность оценить параметры ФРТ: угол 9 и величину Л смазывания изображения, а также размер р пятна дефокусирования изображения. Это повышает точность восстановления изображения. Спектр смазанного изображения сжимается в направлении смазывания, это позволяет оценить 9 и Л. Спектр дефокусированного изображения также сжимается и тем сильнее, чем больше пятно дефокусирования р. Получены новые оценки параметров смаза 9 и Л с использованием частоты Найквиста и параметра дефокусирования р с использованием функции Бесселя. Приведены результаты применения данного метода для обработки изображений. Развиваемый метод может быть использован для повышения точности восстановления смазанных и дефокусированных изображений путем их математической обработки с помощью решения интегральных уравнений.
Ключевые слова: искажения изображений (смазывание, дефокусирование), функция рассеяния точки, параметры искажения, спектр Фурье искаженного изображения, частота Найквиста, функция Бесселя
СПИСОК ЛИТЕРАТУРЫ
1. Bates R. H. T., McDonnell M. J. Image Restoration and Reconstruction. Oxford: Oxford U. Press, 1986. 336 p.
2. Gruzman I. S., Kirichuk V. S, Kosykh V. P., Peretyagin G. I., Spektor A. A. Digital Image Processing in Information Systems. Novosibirsk: NSTU Publ., 2002. 352 p. (in Russ.).
3. Gonzalez R. C., Woods R. E. Digital Image Processing. 2nd ed. Upper Saddle River: Prentice Hall, 2002. 793 p.
4. Jahne B. Digital Image Processing. Berlin-Heidelberg: Springer-Verlag, 2005. 584 p.
5. Sizikov V. S. Inverse Applied Problems and MatLab. St. Petersburg: Lan' Publ., 2011. 256 p. (in Russ.).
6. Sizikov V. S. Direct and Inverse Problems of Image Restoration, Spectroscopy and Tomography with MatLab. St. Petersburg: Lan' Publ., 2017. 412 p. (in Russ.).
7. Tikhonov A. N., Goncharsky A. V, Stepanov V. V. Inverse Problems of Photoimages Processing // Ill-Posed Problems in Natural Science. Moscow: MSU Publ., 1987. P. 185—195. (in Russ.)
8. Methods of Computer Image Processing / Ed. by V. A. Soifer. Moscow: Fizmatlit Publ., 2001. 784 p. (in Russ.).
9. Gonsales R. C., Woods R. E., Eddins S. L. Digital Image Processing using MATLAB. New Jersey: Prentice Hall, 2004. 609 p.
10. Hansen P. C., Nagy J. G., O'Leary D. P. Deblurring Images: Matrices, Spectra, and Filtering. Philadelphia: SIAM, 2006. 130 p.
11. Sizikov V. S., Sergienko A. A., Rushchenko N. G. New fast algorithms for restoration of images smeared under an angle // J. of Instrument Engineering. 2019. Vol. 62, N 1. P. 69—77. (in Russ.)
12. Voskoboinikov Yu. E. A combined nonlinear contrast image reconstruction algorithm under inexact point-spread function // Optoelectronics, Instrumentation, and Data Processing. 2000. Vol. 43, N 6. P. 489—499. DOI: 10.3103/S8756699007060015.
13. Sizikov V. S., Ekzemplyarov R. A. Operating sequence when noise is being filtered on distorted images // J. of Optical Technology. 2013. Vol. 80, N 1. P. 28—34. DOI: 10.1364/JOT.80.000028.
14. D'yakonov V., Abramenkova I. MATLAB. Processing of Signals and Images. St. Petersburg: Piter Publ., 2002. 608 p. (in Russ.).
15. Voskoboinikov Yu. E., Mukhina I. N. Local regularizing algorithm for high-contrast image and signal restoration // Optoelectronics, Instrumentation, and Data Processing. 2000. N 3. P. 41—48.
16. Donatelli M., Estatico C., Martinelli A., Serra-Capizzano S. Improved image deblurring with antireflective boundary conditions and re-blurring // Inverse Problems. 2006. Vol. 22. P. 2035—2053. DOI: 10.1088/0266-5611/22/6/008.
17. Donatelli M., Huckle T., Mazza M., Sesana D. Image deblurring by sparsity constraint on the Fourier coefficients // Numerical Algorithms. 2016. Vol. 72, N 2. P. 341—361. DOI: 10.1007/s11075-015-0047-x.
18. Sidorov D. N. Methods for Analysis of Integral Dynamic Models: Theory and Applications. Irkutsk: ISU Publ., 2013. 293 p. (in Russ.).
19. Sidorov D. Integral Dynamical Models: Singularities, Signals and Control. Singapore-London: World
Sci. Publ., 2014. 243 p.
20. Sizikov V. S. Spectral method for estimating the point-spread function in the task of eliminating image distortions // J. of Optical Technology. 2017. Vol. 84, N 2. P. 95—101. DOI: 10.1364/JOT.84.000095.
21. Sizikov V. S. Estimating the point-spread function from the spectrum of a distorted tomographic image // J. of Optical Technology. 2015. Vol. 82, N 10. P. 655—658. DOI: 10.1364/JOT.82.000655.
22. Sizikov V. S., Stepanov A. V, Mezhenin A. V., Burlov D. I., Eksemplyarov R. A. Determining imagedistortion parameters by spectral means when processing pictures of the earth's surface obtained from satellites and aircraft // J. of Optical Technology. 2018. Vol. 85, N 4. P. 203—210. DOI: 10.1364/JOT.85.000203.
23. Vasilenko G. I., Taratorin A. M. Image Restoration. Moscow: Radio i Svyaz' Publ., 1986. 304 p. (in Russ.).
24. Korn G. A, Korn T. M. Mathematical Handbook for Scientists and Engineers. NY: McGraw-Hill, 1961.
25. Bronshtein I. N., Semendyaev K. A. Mathematical Handbook for Engineers and Students of Technical Colledges. 13th ed. Moscow: Nauka, 1986. 544 p. (in Russ.).
26. Bracewell R. N. The Hartley Transform. NY: Oxford University Press, 1986. 168 p.
27. Antonova T. V. Methods of identifying a parameter in the kernel of the equation of first kind of the convolution type on the class of functions with discontinuities // Numerical Analysis and Applications. 2015. Vol. 8, N 2. P. 89—100. DOI: 10.1134/S1995423915020019.
Сведения об авторах
Валерий Сергеевич Сизиков — д-р техн. наук, профессор; Университет ИТМО, ОЦ
графических технологий; E-mail: sizikov2000@mail.ru Андрей Александрович Сергиенко — магистрант; Университет ИТМО, ОЦ графических
технологий; E-mail: master-ac@yandex.ru Дарья Александровна Кондулукова — магистрант; Университет ИТМО, кафедра графических технологий; E-mail: dariakondulukova@mail.ru
Ссылка для цитирования: Сизиков В. С., Сергиенко А. А., Кондулукова Д. А. Спектральный метод
устойчивой оценки параметров искажений в обратной задаче восстановления изображений //
Изв. вузов. Приборостроение. 2019. Т. 62, № 4. С. 379—386 (in English).
DOI: 10.17586/0021-3454-2019-62-4-379-386
