CC BY
10Journal of Siberian Federal University. Engineering & Technologies, 2016, 9(3), 435-441
УДК 004.932.4
The Algorithm for Compensating Distortion of Images of Type "Blur"
Vitaly V. Vasilyev*
Military educational scientific center of the air force "Air force Academy named after Professor N. E. Zhukovsky and Y. A. Gagarin" (Voronezh) 54a Starykh Bol'shevikov Str., Voronezh, 394064, Russia
Received 12.12.2015, received in revised form 18.01.2016, accepted 29.01.2016
The paper considers the model of linear blurring with a uniform rectilinear motion of an object when its speed is constant, and its direction is parallel to the lines of the formed image. An algorithm for linear blurring compensation based on the account of the physical processes of its formation when obtaining the images is proposed. A system of equations for the restoration of the original video signal samples undistorted image of the object is obtained. Processed the frame of a real video sequence with a horizontal blur and presented its results. Considered the comparison of the results of the restoration of distorted images by the proposed algorithm with known methods.
Keywords: model of linear blurring of the image, formation, image reconstruction and correction of image distortions.
Citation: Vasilyev V.V. The algorithm for compensating distortion of images of type "blur", J. Sib. Fed. Univ. Eng. technol., 2016, 9(3), 435-441. DOI: 10.17516/1999-494X-2016-9-3-435-441.
Алгоритм компенсации искажений изображений типа «смаз»
В.В. Васильев
ВУНЦ ВВС «ВВА им. проф. Н.Е. Жуковского и Ю.А. Гагарина» Россия, 371600, Воронеж, ул. Старых Большевиков, 54а
В статье рассмотрена модель линейного смаза при равномерном прямолинейном движении объекта, когда его скорость постоянна и ее направление параллельно формируемым строкам изображения. Предложен алгоритм компенсации линейного смаза, основанный на учете физических процессов его формирования при получении изображения. Получена система уравнений для восстановления исходных отсчетов видеосигнала изображения неискаженного
© Siberian Federal University. All rights reserved Corresponding author E-mail address: [email protected]
*
объекта. Выполнена обработка кадра реальной видеопоследовательности с горизонтальным смазом и приведены ее результаты. Рассмотрено сравнение результатов восстановления искаженных изображений предлагаемым алгоритмом с известными способами.
Ключевые слова: модель линейного смаза изображения, формирование, восстановление изображения и исправление искажений изображения.
Most of the information a person perceives through visual images. Now for recording, processing and transferring images the most widespread technical system of measurement and processing of information (imaging devices and computing devices) that are usually present output in the form of digital images [1]. However, what one sees on the screen, i.e. the digital image, not always perfectly reflect the picture of the world. The picture may be distorted: blurred, defocused, noised, and others. All this is due to the imperfection of the recording devices and the image forming conditions. As a result, it becomes necessary restoring distorted images by mathematically way [2]. This work is devoted to the image distortion compensation type "blur".
The blur occurs in cases, when during the time of exposure the frame T recording device or picture of a scene are moving in a certain direction, or by their mutual motion. In general, the blur can be described as the result of multiple overlays, with displacement.
The presence of motion blur is a defect that leads to fuzzy images with imminent information loss. Therefore, the task of restoration of blurred images is very relevant.
Existing methods for reconstruction of blurred images have a number of advantages, such as ease of use (inverse filtering, Wiener filter, filtering with least squares method, recursive Kalman filter, forced deconvolution iterative methods [2-5]) or high enough quality of recovery (regularisation methods [2, 6, 7]). However, they do not take into account the physical processes of blur forming during the image acquisition. The proposed algorithm of compensation of the linear blur is based on the account of these processes.
It should be noted that the task of restoring the blurred images is factorable, i.e. it is possible to separately determine the motion of object horizontally and vertically, and then find the parameters of the total movement. Therefore, for simplicity we will consider a linear blur that occurs in uniform motion object, i.e. with a constant velocity whose direction is parallel to formed lines of the image.
The photosensitive surface (matrix CCD) is a set of light-sensitive cells of size ДххДу, the distance between which can be neglected.
As a result of exposure in each cell of the line CCD accumulates charge [8]
where E(x, y, t) is the intensity of the illumination on the surface of the CCD; k - coefficient of proportionality.
We assume that within the line the intensity of the illumination does not depend on the y coordinate. If the image is stationary, the output of the CCD appears as the line of image formed as a set of signal samples {u,}, ie [-M, M], where (2M+1) - the number of samples of the video signal lines. In this case
(1)
0 (г-1)Д x 0
Ui = Et A x A yT,
(2)
1 iAx
where Ei = — J E(x)dx.
A X (i-l)Ax
As a model of the video signal of the image line adopted known applicative model [9]:
u = Z f^ - j) - j)> (3)
je{F\ jej®i
where uj - is the sample of the video signal of the frame line; f and qj - samples of video signal of the object and background respectively; {F} and {®} - the set of numbers of pixels belonging to the object and the background respectively; {f }n{cP}=0 ; S(i - j) - a single impulse.
Let by the exposure time T, the object is moved along a photosensitive surface for n< m cells (schematically represented in Fig. 1).
Let us consider the formation of the count um+n+1.
During the exposure time Tthe rightmost discrete-analog count Eim , corresponding to the moving object, will not come to a photosensitive cell with number (m+n+1), because will stop at the left edge of it.
Therefore, given (1) - (3), all this time, this section of the photosensitive surface will be projected with the light beam, corresponding to a stationary background, i.e.
(m+n+1)Ax T
um+n+1 = A y j j El+n+1dxdt = Em+n+1AxAyT = 9m+n+1 . (4)
(m + n)A x 0
T n -1
When forming the count um+n for the time T--= — T on the photosensitive cell will
n n
be exposed a stationary background, and then, during the time will be also exposed corresponding
n
part of the moving object, i.e.
um + n
(m + n)A
= A, J
(m + n-1)A x
i n-1
—T
n T
j El+ ndt + j \El+ n^lx - mA x - /1 +
0 ^T (5)
V n
+ Emm 1l-ctIx - mA x - t^
dt
/
dx,
where c(...) - is the unit function of Heaviside.
Performing simple transformations, we can obtain: um+n = ——19 m+n +—fm and
2n 2n
mA x T
fm = A y J J Efdxdt = Ei A x A yT .
(m-1)Ax 0
t=0
t=T
n+1 m+n
Fig. 1. The position of the moving object on the line of the image
1
m
Calculating the remaining counts in the same way can be obtained the system of equations (6), which allows restoring the original counts of the unblurred object.
f + 2 f2 + 2 /3 + ... + 2 fn + fn + j = 2 nu n +,
f2 + 2 f3 + ... + 2 fn + j + fn + 2 = 2 nu n + 2
f m - n + 2 fm - n + 1 + ... + 2 fm -1 + fm ~ 2 nU m
f m - n + 1 + 2 fm - n + 2 + ... + 2 fm-1 + 2 fm = 2 nU m + 1 " Vm + 1
fm - n + 2 + 2 fm - n + 3 + ... + 2 fm = 2 nU m + 2 " 3Vm + 2
(6)
fm - 2 + 2 fm-1 + 2 fm = 2 nU m + n - 2 " (2 n " 5)^m + n - 2 fm-1 + 2 fm = 2 nU m + n -1 " (2 n " 3)Vm + n-1 fm = 2 nU m + n - (2 n - 1)^m + n
In (6) - fx,...fm - are the counts of the video signal of line, corresponding to the unblurred image of the object; un+l,..., um+n - are the measured counts of the video signal of line; 9m+b..., <pm+„ - stationary background counts (data value counts and the magnitude of the displacement of the object n is taken from the previous frame of the video sequence) [10].
In solving the problem of blurring compensation should pay special attention to the formation of fronts and the slices of the distorted image.
In Fig. 2 are shown the video signals, corresponding to the object of the constant brightness b that is at rest, and object moving uniformly and rectilinearly on a stationary background a.
The accepted model of blurring is based on the processes of forming the counts of the blurred image. The slice and the front of pulse of distorted image in this model are differ from those used in the classic descriptions [6] (see. Fig. 3):
U*(x) = -—— (m - x) + b, x e[m,m + n +1], (7)
n +1
TJ. . b - a, b + (2 n -1) a Til
U (x) =-(m + n - x) +---— ,x s[m +1, m + n\. (8)
n 2n
The work of the proposed algorithm is to form the sequence of image frames, selecting neighboring frames, determining the direction of the blur, blurred boundaries of the object and its speed.
On the basis of the received data is the recovery of undistorted image counts by solving the system of equations (6).
Fig. 4 shows the frame of the real video sequence with a horizontal blur. The results of the reconstruction are presented in Fig. 5.
a
1 ... m
t = 0
t = T
1+n m+n+1
Fig. 2. The video signals corresponding to the object of constant brightness on uniform background
b
Fig. 3. The slice of the video signal using the standard (U*(x)) and proposed (U) models of the formation of image blurring
Fig. 4. The distorted frame of the video sequence Fig. 5. The restored frame of the video sequence
On the reconstructed image (Fig. 5) one can see clear boundaries of a moving object (white rectangle). The boundaries of the figure, depicted on a moving object, also restored clearly, but have slight bends, due to the noisiness of the background.
Compare the results of reconstruction of blurred images by proposed algorithm and existing methods.
As a priori knowledge will only use the information about the direction of movement of the object and speed value (defined by the two adjacent frames of a video sequence). Figure 6 shows a fragment of the blurred image (Fig. 6 a), and the results of its recovery by methods of inverse (Fig. 6 b) and Wiener (Fig. 6 c) filtering, with the algorithm of Lucy-Richardson (Fig. 6 d) and the "blind" deconvolution (Fig. 6 e).
Analysis of the restoring of the fragment of the blurred image by the known methods showed that, in the absence of additional information (the energy spectrum of noise, the signal-to-noise, the minimum threshold of a deviation obtained from the original image, etc.), which, as a rule, is not available, the results have significant defects. In addition, the methods of inverse and Wiener filtering requires knowledge of the distorting function that also in most cases is unknown (in the simulation of distortion compensation by known methods as distorting function we have used a single function of Heaviside with displacement). The comparison of results shown in Fig. 5 and 6 clearly demonstrates the high efficiency of the proposed algorithm and a high quality of recovery through the use of information about the process of formation of the blur.
Thus, the proposed algorithm takes into account the processes of formation of counts of a linear blur distorted images, allows solving the problem of blurring compensation in the absence of knowledge about the distorting function. The proposed approach is universal and can be used to obtain the vibrational patterns, the nonlinear caused by camera shake and other types of blurring.
c)
d)
e)
Fig. 6. Compensation of distortions of type "blur": a) original blurred image; b) inverse filtering; c) Wiener filtering; d) the algorithm of Lucy-Richardson; e) blind deconvolution.
References
[1] Detection, identification and determination of parameters of images of objects. Methods and algorithms/ Edited by A.V. Korennoy. M.: Radiotechnika, 2012, 112 p.
[2] Gonzalez R., Woods R. Digital image processing. M: Technosphera, 2005, 1072 p.
[3] Y. Yitzhaky, I. Mor, A. Lantzman, and N. S. Kopeika, Direct method for restoration of motion-blurred images. Journal of Opt. Soc. Am. A. 1998, 15, 6. Pp. 1512-1519.
[4] Wang, Y., Yin, W. Compressed Sensing via Iterative Support Detection. CAAM Technical Report TR09-30, 2009, 13-18.
[5] Pereslavtseva E. E., Filippov M. V. Method of accelerated restoration of images blurred by movement. Electronic scientific and technical periodical "Science and education", 2012. 7730569/340562.
[6] Jiann-MingWu, Hsiao-ChangChen, Chun-ChangWu, Pei-HsunHsu."Blind image deconvolution by neural recursive function approximation", 2010. World Academy of Science, Engineering and Technology vol: 4, 2010-10-22.
[7] M. Martinello, P. Favaro "Single image blind deconvolution with higher-order texture statistics", 2011. D. Cremers et al. (Eds.). Video Processing and Computional Video, LNCS 7082, 124151, 2011.
[8] Andreev A. L. Automated monitoring system. Part 1. Hardware and electronic components. Textbook for course and diploma projects. SPb.: SPbGUITMO, 2005. 88 p.
[9] Processing of multidimensional signals. In 2 books. Book.1. Linear discrete multidimensional signal processing. Methods of analysis and synthesis. / Ed by A.V. Bogoslovskiy. M.: Radiotechnika, 2013, 168 p.
[10] Bogoslovskiy A.V., Bogoslovskiy E.A., Zhigulina I.V., Vasilyev V.V., Ponomarev A.V. The linear blur. M.: Radiotechnika, 2015, № 12, p. 147-153.
