CC BY
21
Research of Digital Camera Dynamic Range on the Imaging Processing Basis
Sutkowski M.1, Saukova Ya.2
'Institute of Microelectronics and Optoelectronics, Warsaw University of Technology, 15/19 Nowowiejska Str., Warsaw 00-665, Poland 2Belarusian National Technical University, Nezavisimosty Ave., 65, Minsk 220013, Belarus
Received 02.02.2017
Accepted for publication 23.08.2017
Abstract
Digital images provide to determine photometric and colorimetric properties of objects subject to validation all elements of a measuring channel (digital camera, software, display) and solve the problem of their limited dynamic ranges. The aim of the study was to explore the dynamic range of a digital camera for use in photometric and colorimetric measurements.
The Laboratory of Photonics at the Institute of Microelectronics and Optoelectronics (Warsaw Technical University, Poland) conducted a comparative experiment to determine the threshold of sensitivity, linearity and range of application the digital camera. Color target sets with certified brightness and chromaticity were created at the terminals and recorded with a digital camera with different exposure times. The authors propose a method to extend the dynamic range of a digital camera for red, green and blue color channel of intensities by pairing the calibration dependencies, and determine the true brightness and color of a point on the object by calculation.
Calibration dependencies (triads) of digital camera for red, green and blue color channels intensities were constructed. These dependences allow determining lower and upper bounds of the dynamic range. Each triad has a form of the hysteresis loop. The experiment showed that the accuracy of this method is ± 3-5 %.
Keywords: digital camera, dynamic range, imaging. DOI: 10.21122/2220-9506-2017-8-3-271-278
Адрес для переписки:
Савкова Е.Н.
Белорусский национальный технический университет, пр. Независимости, 65, г. Минск 220013, Беларусь e-mail: [email protected]
Для цитирования:
Sutkowski M., Saukova Ya.
Research of digital camera dynamic range on the imaging processing basis.
Приборы и методы измерений.
2017. - Т. 8, № 3. С. 271-278.
DOI: 10.21122/2220-9506-2017-8-3-271-278
Address for correspondence:
Saukova Ya.
Belarusian National Technical University, Nezavisimosty Ave., 65, Minsk 220013, Belarus e-mail: [email protected]
For citation:
Sutkowski M., Saukova Ya.
[Research of Digital Camera Dynamic Range on the Imaging Processing Basis].
Devices and Methods of Measurements.
2017, vol. 8, no. 3, pp. 271-278.
DOI: 10.21122/2220-9506-2017-8-3-271-278
Исследование динамического диапазона цифровой камеры на основе технологий обработки цифровых изображений
Сутковский М.1, Савкова Е.Н.2
'Институт микроэлектроники и оптоэлектроники, Варшавский политехнический университет, ул. Нововейская 15/19, Варшава 00-665, Польша 2Белорусский национальный технический университет, пр. Независимости, 65, г. Минск 220013, Беларусь
Поступила 02.02.2017 Принята к печати 23.08.2017
Цифровые изображения позволяют определять фотометрические и колориметрические свойства объектов при условии валидации всех элементов измерительного канала (цифровой камеры, программного обеспечения, дисплея) и решения проблемы ограничения их динамических диапазонов. Целью работы являлось исследование динамического диапазона цифровой камеры для ее дальнейшего использования в фотометрических и колориметрических измерениях.
Лаборатория фотоники в Институте микроэлектроники и оптоэлектроники (Варшавский технический университет, Польша) проводила сличительный эксперимент для определения порога чувствительности, линейности и диапазона применения цифровой камеры. Цветовые мишени были созданы на дисплее в виде аттестованных цветовых однородных полей и регистрировались с помощью цифровой камеры с пошагово увеличивающимся временем экспозиции, а затем осуществлялась обработка изображений и калибровка камеры в красном, зеленом и синем цветовых каналах для получения калибровочных зависимостей. Авторами предложен метод расширения динамического диапазона цифровой камеры для красного, зеленого и синего цветовых каналов интенсивностей посредством сопряжения градуировочных зависимостей и определения истинной яркости точки на объекте расчетным путем.
Полученные калибровочные зависимости (триады) имели форму петли гистерезиса и позволили расчетным путем расширить динамический диапазон. Эксперименты показали, что точность данного метода составляет ± 3-5 %.
Ключевые слова: цифровая камера, диапазон, обработка изображений. БОТ: 10.21122/2220-9506-2017-8-3-271-278
Адрес для переписки:
Савкова Е.Н.
Белорусский национальный технический университет, пр. Независимости, 65, г. Минск 220013, Беларусь e-mail: [email protected]
Для цитирования:
Sutkowski M., Saukova Ya.
Research of digital camera dynamic range on the imaging processing basis.
Приборы и методы измерений.
2017. - Т. 8, № 3. С. 271-278.
DOI: 10.21122/2220-9506-2017-8-3-271-278
Address for correspondence:
Saukova Ya.
Belarusian National Technical University, Nezavisimosty Ave., 65, Minsk 220013, Belarus e-mail: [email protected]
For citation:
Sutkowski M., Saukova Ya.
[Research of Digital Camera Dynamic Range on the Imaging Processing Basis].
Devices and Methods of Measurements.
2017, vol. 8, no. 3, pp. 271-278.
DOI: 10.21122/2220-9506-2017-8-3-271-278
Introduction
Professional and semi-professional digital cameras including thermal imagers can be used as the universal receivers of photonic information generally in visible range. At the same time the current level of digital technology provides opportunities to apply digital cameras during the control and diagnostics of objects at all stages of their life cycle. Since the characteristics of the cameras are not the object of obligatory acknowledgement of conformity manufacturers may indicate exaggerated values. Therefore the need arises to obtain reliable and accurate results of control and testing. As it is known digital cameras have limited dynamic range to reproduce the intensity (about 60 dB) while it is necessary to record the brightness over a wider range. Standard methods for solving this problem are based on the dynamic range compression of the plots.
Currently compression procedures are used to reproduce image details that are situated in the region closed to the saturation matrix photodetector [1]. So it is necessary to move the point A into the region of the inflection of light characteristics of the signal, which produces a brighter light for compressing the amplitude (interval E1-E2, Figure 1a). The point A limits a dynamic range of a camera. A linear dependence of Uout(E) is modified by reducing the slope of the transfer characteristics in the upper part. Inflection points and slope angle affect the light flux of the brightest objects.
Therefore, the possibility exists to compress the excess dynamic range (bright areas don't merge) reducing the slope of the transfer characteristics. Also the nonlinear compression can be applied. It is necessary to make a plot with the non-linear inflection point in the transition mode overload more smoothly (Figure 1b). In this case, the slope of characteristics varies in proportion to the number of parts of the light phase that would be detected. Also more careful study of details in blown highlights can be achieved by shifting the point of inflection at a constant pole tilt, keeping the signal from vivid seats below the limitation level (Figure 1c). These methods allow only a subjective improve the reproduction of bright details of the image. However, when it is necessary to determine illuminance (brightness) in different points of the object simultaneously a principally new approach is needed.
Aspects of cameras range compression are discussed in [2] and [3]. Contrast extension leads to a «splitting» of quantization levels that appear in
the histogram as a comb. The combined method of dynamic range controlling by software tools in [4] is described. The method is based on the dynamic range stretching in dark areas and compression in the blown highlights area of image. However these technologies are used only for visual improvement of images quality and don't allow determining the intensity significantly. They consider the patterns and effects of visual perceptions such as light and color adaptation, the contrast and the contours of fine details etc. [5, 6].
uou,
100 %
85 %
Ig(Uout) 100 %
A
Additional compression
>..--' _ - -**
^^^ i ^^^ 1 A fixed point of
| ^"»inflection l
1 l 1 1 1 1 1 >
-1 0 +1 +2 Set dynamic range b
Steps of the diaphragm
lg(Uoul)
100 %
Additional compression
Steps of the diaphragm
Set dynamic range
c
Figure 1 - Electronic methods of dynamic range compression: a - linear compression with the displacement of the point of saturation down; b - nonlinear compression after the inflection point for different contrast; c - options linear compression with the same contrast
This method is based on repeated digital regarding of extended objects and digital images
a
processing. The brightness of the investigated area of the digital image is compared with the brightness of the certified sources available in the long term picture. The problem of determining the illumination at the object is solved by expanding the dynamic range of optical measurement software. The method developed in the Scientific Research Laboratory of optics and electronics instrument making (Belarusian National Technical University, Minsk, Belarus) [7] and tested in the Laboratory of Photonics of the Institute of Optoelectronics and Microelectronics (Warsaw University of Technology, Warsaw, Poland) allows doing photometric measurements in real time with the specified characteristics error. The main idea of this method is to do series of digital images of the same object and reference samples with different exposures and process the images in RGB and RAW formats. Format RGB provides the ability to define intensity in red, green, and blue color channels of images [8]. There are different technologies of objects multiple registrations with increasing the time of exposure for creating of HRDI images [9, 10]. But they apply in computer games and design. And the technology which is described provides an op-
Measured photometric values of the total 17 samples: Y -
portunity to obtain quantitative data of photometric and colorimetric parameters of the object.
The experimental research of a digital camera dynamic range
To ensure metrological traceability the sets of achromatic and chromatic samples were done on the colorimetric and calibrated display (look Table). The values was applied as five different values represented in standard 8-bit digital number. The samples were then characterized by use of spectrophotometer as real photometric values. The readings were done for 3 different areas and then were averaged. In the Figure 2 there are shown chromacity values of the samples (x, y) plotted on the CIE 1931 chart.
Imaging system callibration procedure is necessary to know real lighting parameters of the examined area of the light panel. The real photometric parameters was determined by Minolta CS-100A non-contact chroma meter. Each measurement was achieved by averaging of three readings done at three neighbouring points of the examined area of the panel. Every measurement was done not more than 10 s after respective image acquisition (see further paragraph).
Table
luminance, x, y - chrominance
Displayed sample values Gray (Gr) R G В
Y X Y Y x | y Y x | y
1 R 0 0,17 0,2527 0,3527 0 X 0 X 0 X
G 0 0 0 0
В 0 0 0 0
2 R 64 11,00 0,3207 0,3437 64 2,93 0,6040 0,3527 0 8,02 0,3220 0,5833 0 1,49 0,1433 0,0600
G 64 0 64 0
В 64 0 0 64
3 R 128 31,43 0,3217 0,3467 128 9,05 0,6190 0,3490 0 25,77 0,3333 0,5810 0 2,48 0,1423 0,0620
G 128 0 128 0
В 128 0 0 128
4 R 196 74,23 0,3240 0,3437 196 19,20 0,6217 0,3497 0 53,77 0,3347 0,5813 0 5,66 0,1443 0,0670
G 196 0 196 0
В 196 0 0 196
5 R 255 121,00 0,3233 0,3437 255 31,40 0,6210 0,3520 0 88,03 0,3343 0,5813 0 9,73 0,1453 0,0713
G 255 0 255 0
В 255 0 0 255
The hardware set-up of the measuring system proposed in this paper is targeted to propose portable and flexible color measurement system. It assumes to use the best possible commercially available image acquisition system with best possible image quality. For measuring application it should offer possibility to control all steps of the image registration procedure, including flexible and fully controlled image data processing. The next assumption is to use area sensor instead of widely used spot measurement units. It should can work quite everywhere, even there where en-
vironmental conditions elliminates use of specialized spectrophotometers. Fulfilling these factors we can offer a system which allow to examine light panels or large illuminating surfaces comprehensively. Regarding to mentioned reasons we decided to use commercial photographic camera with high-quality imaging sensor as the image ac-quision unit. The characterization of the imaging hardware for measurement requirements is highly needed. Also we have to use image processing and analysis without (or as low as possible) influence on the data taken by the sensor.
Figure 2 - Chromacity values of the samples used in the experiment plotted on the CIE 1931 graph. The luminosity is ommitted
As the axquisition unit a Nikon D610 camera was chosen. It has a CMOS (Complementary Metal Oxide Semicondutor) large area imaging sensor offering active area of 35,9 x 24 mm and total pixel count 24,7 million. It is equipped with standard color filter array of Bayer RGB type. The imaging data can be represented as a digital number with 14-bit depth. The final image can be recorded at resolution of 6016 x 4016 pixels [http:/ HYPERLINK «http:// imaging.nikon.com/lineup/dslr/d610/spec.htm»/ imaging.nikon.com/lineup/dslr/d610/spec.htm]. It works with standard imaging lens AF-S Nikkor 2485mm f/3,5-4,5G ED VR. It offers maximum aperture of f/3,5 at f = 24 mm and f/4,5 at 85 mm. It's optical construction consists of 16 optical elemnts in 11 groups including 1 ED (low dispertion) glass and 3 aspherical lens elements. This lens was chosen due to the very good response at the center of the image. Its MTF (modulation transfer function) curves show near perfect characteristics for low spatial frequencies (10 l/mm for both tangential and sagital lines) at the circle with 10 mm diameter. It has very flat characteristic which minimum value is approx. 0,98 [http://imaging.nikon.com/lineup/lens/zoom/nor-malzoom/af-s_24-85mmf_35-45g_ed_yr/index.htm].
The goal of the present experiment is to find an answer of the camera for different exposure values to find a linear part of the registration curve. To perform these tests a reference light panel is needed. In practice the ligh parameters of the image applied to the reference panel utilized for these measurements has
to be well known. As a reference light panel a colori-metric wide gamut LCD monitor was used, in detail Eizo ColorEdge CG245W offering 100 % of sRGB space coverage. It was driven by AMD HD6990 graphic card with correction LUT applied. It is achieved by color callibration procedure with use of datacolor Spyder 3 Elite callibrator. For the tests we used only a small part of the panel at the center to avoid (or minimalise) any angular dependancies and backlight illumination non-uniformity. Measured area had a dimensions of approx. 100 x 100 mm.
Measuring set-up is shown on Figure 3. Object distance was fixed at the value of 700 mm, optical axis was perpemdicular to the panel plane. Imaging lens was set to the focal distance f = 50 mm and middle aperture f/8. The acquired images were saved as a NED (RAW-type) file with 14-bit depth. All incamera processing was ommitted and conversion to tri-chromatic form (demosaicing) was performed with «zero» parameters in commercial Adobe Light-room 4.4 software. The signal amplification in camera was set to eqivalent of ISO 100. Exposure was measured for middle-gray (128, 128, 128) values and correcponds to exposure time t = 1/4 s. The experiment consists of series exposures within the range of relative EV from -5 to +4 with step od 1/3 what correspons exposure time range from 1/125 to 4 s.
Figure 3 - Experimental set-up for characterization of the digital camera
The goal of the experiment was to determine linear part of the camera characteristic. The target was changed in luminosity and in chromacity in five steps from value 0 to 255 (noted as Gr for theoretically neutral grays, R, G, B - for the basic red, green, blue chroma respectively and numbered from 1 to 5 - as in the Table 1). For each constant target the series of vary exposure is done. Acquired image was converted with use of Adobe Camera RAW from raw data (a 14-bit uncompressed NEF file saved by the camera) to tri-chromatic form and
saved as 16-bit TIFF file. Measurements of the image parameters was done by Image J software after cropping of the ROI (Region Of Interest) with the size 585 * 585 pixels at the center of the image.
As result an averaged luminance values of the pixels in three channels are taken. The experimental results are shown oh Figure 4 for gray channel (Gr), R, G, B channels respectively.
Figure 4 - Experimental curves of the digital camera range obtained by recording the self-luminous objects with different exposure time: red colour - R; green - G; blue - B
The figure 4 shows that the triad of characteristic dependences have the shape of the hysteresis loop. Every dependence is not necessarily linear within the active range area. Dependences have different angles of inclination.
The basic concept of measurements
In information systems the process of data con-vertion assumes the operations of filtering, sampling, quantization, encoding, decoding, postfiltration etc. For realization of measurements it is necessary to adhere to the principle of comparison with a measure. This is a difficult task because there are many difficulties with ensuring of the metrological traceability of results, given the diversity of hardware and software, as well as the use of computer ranking scales for the measurement of the intensities of digital images. In addition, the differences and limitations of transmitting devices gamuts prevent the obtaining of reliable measuring information. There are many difficulties arise in solving measurement problems associated with determining the luminance and color characteristics of objects based on processing their
digital images. These difficulties are associated with objective limitations of color coverage, and almost all economic ranges of the transmission device, and establishing the corresponding nominal level of quantization in the color channels as a result of the normalization of the digital image at the brightest point. These problems can be solved by the correct application of available hardware and software according to the proposed method of measurements. The proposed method is based on digital registration of standard objects with different values of exposure time and digital images processing. The metrological traceability of measurement results is provided by reference to the initial samples (called by authors as a line sources with equal brightness) which are nonpoint sources of light acting as the measures in measurements [2]. Within each of the line light sources should vary in brightness with a step depending on the specific measuring task. Thus we can construct a set of calibration dependencies for each source showing the dependence of numerical representation of intensity values (j-axis) of the color channels from the exposure H (x-axis) - R(t), G(t), B(t), an example of which is given in Figure 5 [3]. Further accord-
ing to mated for each consecutive pairs of sources for example one line for the one time. The values of intensities of R, G, B for exposure H pending solid lines in the figure on the j-axis. The exposure H is understand as a product of incident beam intensity and the exposure time (time when photon interact with the imager structure):
k = k = G_ k = _B_ R 255' G 255' B 255'
(5)
Hy = I ■ 'exp,
(1)
where I - denotes intensity of the photon beam incident onto imager; t - denotes exposure time.
A Intensity, re ¡.units / 04 /- Los
/7
/ ...
/.........L*,:.^............. .................................
- - ->
Exposure time, s
Figure 5 - The theoretical calibration curves of digital images color channels
As can be seen in Figure 5, in the red channel, the intensity of Rn lies in the region of saturation. This is due to limitations in the dynamic ranges of imagers of digital cameras and therefore, makes it impossible to obtain valid measurement information. The offset of intensity in the green channel G01 in the area of imager's noise may occur when switching to the image with very low exposure level. Therefore to base on the principle of linearity of the calibration characteristics of the measurement method we can determine on the y-axis conditional point R'n outside physical dynamic range and carry out the normalization of the intensities of R, G, B, adapting them to new values, introducing appropriate broadening coefficients directly in a standardized model transformation color spaces RGB^XYZ [4]:
X = ^ R + ^ G + ^ B,
Y = aI r + ^ G + Ъз g,
Z = R + ^ G + ^ B,
^n k D
(2)
(3)
(4)
where X, Y, Z - coordinates of the color space XYZ; anm - the standardized weighting of intensities R, G, B, kR, ka kB - expanding the coefficients calculated by the formulas [4]:
The highest value of the digital intensity image is given in the denominators. It is dependant on the actual digital quantization level, the highest number is repesented by the 2n where n is the number of bits used. There is 255 gradations of intensity in the case of 8-bit digital representation. New transformed lines of intensity in the color channel and G are shown in Figure 6 by the stipple, but at the saturation level all data is lost. The calculations of the chromaticity coordinates produced by the formulas (2)-(4). Thus it was established experimentally that color coordinates triads of the same reference point on the object in images made with different exposures with a given confidence level correspond to the same chroma. If at least one channel reach its saturation level we can predict linear continuity of the constant chroma value and to make reading over this level. It was also found that the greatest variance of the intensity values observed in the blue channel. The averaged variance of determination of chromaticity coordinates is from 2.8 to 3.1 units approximately. Reliable detection of object real color coordinates depends on the nominal values of quantization steps, playing required scales conventional digital image. The nominal level of quantization represent the ellipsoids of different volumes in three-dimensional color spaces, and the minimum (definitely) uncertainty ellipses is determined by the Mac-Adam [5]. However the constant value of the nominal level of quantization can be maintained in the RGB color space by the mathematical methods of the increment intensity. There is a possibility to define the color characteristics of self-luminous objects and to enter correction factors for secondary emitters by applying models of chromatic adaptation Von-Crease, CIECAM02, Fairchild and others [5, 6].
Data conversion must be performed for each color channel. It should be consider surface properties (reflectance) and parameters of shooting in the calculations. The Figure 6 explains the essence of the developed model. Different signals N^, N'02, X 01, N"02, etc. correspond to the reference brightness values Li, L2 on the abscissa. The linear dependences can be built on control points of the photodetector dynamic range before the transition to the stage of saturation. By pairing the dependency mathematically with sufficient accuracy for practice it is possible to calculate intensity values of points on the object in three color channels, thus broadening the range of two-dimensional colorimetric measurements.
B
B
Figure 6 - The extension of detected brightness dynamic range of the images made with different exposures (different exposure time: t1 < t2 < t3 when other parameters are constant)
The goal of the experiment was to study characteristics of the digital cameras dynamic range for their validation in the information channel. Lower and upper limits of the range, the linearity of color channels were studied during the experiment. The experiment plan included the following stages:
1) to do the sets of standard sources (achromatic and chromatic samples) on the display;
2) to measure the brightness and color settings of samples with the help of spectrophotometer;
3) to take digital images of samples with different exposure times;
4) to do image processing for determination the intensities in RGB channels and the chromaticity coordinates;
5) to obtain the dependences (calibration characteristics of camera);
6) to investigate the characteristics of dynamic range.
Conclusion
1. The resulting diagrams (curves) are representing the characteristic of the registration unit (a camera with the lens). Each registration curve has a linear section where the aerial measurements are possible.
2. General shape of the registration curve show strong nonlinearity where it cannot be callibrated for measurements.
3. The chroma responses of the camera shows significat differencies in spectral characteristics of the RGB Bayer filters used in the image detector of the camera in comparision to reference values (measured by chroma meter).
4. On the B channel there are some errors on the
curve visible. This is due to the very low level of lu-mimance of the blue image displayed on the monitor used in this experiment.
5. With use of aerial sensor we can make averaging measurements of the area light panels or to measure few different sources simultaneously.
6. Due to the possibility of exposure adjuste-ments we can offer a very wide range of light intensity measurements.
7. In the paper the dedicated demosaicing process for the camera we used is ommitted. When applied it can allow to produce more precise results and the chroma response of the camera can be calibrated with use of reference source.
This technology can be used as a basis for calibration and validation methods of cameras, displays and other controlled objects too.
References
1. Ward P. Working with digital cameras: lessons learned operator skills. Moscow, Mir Publ., 2001, 301 p. (in Russian).
2. Lynch R. The Usage of Adobe Photoshop 6. Special edition, Moscow, Publishing house «Williams», 2001, 880 p. (in Russian).
3. Reihard E., Ashikhmin M., Gooch B. and Shirley P. «Colour Transfer between Images». YIEEE Computer Graphics and Application, Appiedperception, September/ October 2001, pp. 34^1.
4. Goden J. Colorimetric in the rendering performance. Moscow, Technosphere Publ., 2008, 328 p. (in Russian).
5. Rudakov P.I., Safonov V.I. Processing of signals and images. MATLAB 5x. Moscow, Technosphere Publ., 1998, 503 p. (in Russian).
6. Margulis D. Photoshop for professionals. Minsk, Popurry Publ., 1999, 304 p. (in Russian).
7. Zuikov I., Saukova Y. The Method of measuring the intensity of the light source. Patent G 01 J 1/00 By 10453. Application № a20051233/ Registered in the State register.
8. Susstrunk S., Buckley R., Swen St. Standard RGB Color Spaces. Laboratory of audio-visual Communication (EPFL), Xerox Architecture Center, Apple Computer Lausanne, Switzerland; Webster, NY; Cupertino, CA. Available at: https://infoscience.epfl.ch/record/34089/ files/SusstrunkBS99.pdf
9. Starovoitov V. V., Golub Yu. I. Digital image, from receipt to processing. Minsk, United Institute of Informatics problems, National Academy of Sciences of Belarus, 2014, 202 p. (in Russian).
10. Dvorkovich V.P., Dvorkovich A.V. Digital video information systems (Theory and Practice), Moscow, Technosphere Publ., 2012, 1008 p. (in Russian).
