CC BY
10UDC 004.932
WHITEBOARD AREA DETECTION BASED ON SPECIAL PATTERN
P.K. Utemuratov1, A.K. Akshabayev2
Abstract
Nowadays online education and webinars are very popular. In such kind of education, usually teachers use special programs for writing something to explain and broadcast it online to students. As practice shows, this kind of sharing information, writing, drawing something using mouse is very hard for teachers, it is easier to explain everything on whiteboard in front of audience. Big problem is detecting whiteboard and translating it online. Many researchers have proposed different solutions for pattern recognitions, and we try to use some of them to create algorithm for detecting whiteboard area. For this purpose we use simple web camera, and special characters on board. This paper proposes algorithms for detecting whiteboard from image frames, using special 'star' pattern.
Keywords: lecture, smart board, pattern detection, whiteboard, image processing.
1. Introduction
Education is one of the most important tools to succeed in our life. Development of economy brought to rapid development of information technology. In twenty first century it is difficult to imagine our life without new technologies and gadgets. All of this brought to be in trend with latest tools and technologies. However the foundation of any company is their staff. So, each company which wants to see own selves at the pick of the pedestal need theirs employees to be equipped with latest information and advanced skills. To fill this gaps there are provided a lot of online courses where any people can gain knowledge in a particular subject. Nowadays there are a lot of online courses with multimedia, pictures, illustration materials which makes easy for understanding the topic. The advantages of E-learning are:
-Learner centered
-Time and location flexible
-Cost-effective for learners
-Potentially available to global audience
-Unlimited access to knowledge
-Archival capability for knowledge reuse and sharing [1]
An interactive whiteboards and other technologies and tools for increasing efficiency of e-learning is taking huge role in the current world [2]. The system of automatically detecting digital images during the lecture would be a great contribution to the area of e-learning. In this paper, we describe a system that takes lecture images from a whiteboard. The application incorporates a variety of automatic and interactive techniques to identify and segment desired content in the camera view, allowing the user to publish a more focused video. There can be several images on the whiteboard. Required region will be detected by labels in the form of little hand-drawn 'star' patterns.
Because digital cameras are becoming accessible to average users, more and more people use digital cameras to take images of whiteboards instead of copying manually, thus significantly increasing the productivity. The system we describe in this paper aims at reproducing the whiteboard content as a faithful, yet enhanced and easily controllable, electronic document through the use of a camera. However, images are usually taken from an angle to avoid highlights created by flash, resulting in unde-sired perspective distortion. They also contain other distracting regions such as walls. The system we have developed uses a series of image processing algorithms. It auto-
1Pernekhan Utemuratov - Masters student of faculty of Information Technologies, Kazakh-British Technical University, Almaty, Kazakhstan.
2Askar K. Akshabayev - Associate Professor of faculty of Information Technologies, Kazakh-British Technical University, Almaty, Kazakhstan.
matically locates the boundary of a whiteboard. We use special 'star' pattern for helping us detecting and for making correct transformation of whiteboard. In the first section we will talk about special pattern based identification of whiteboard. Afterwards, we will discuss about using cascade training algorithms for finding the pattern candidates. Then we will tell about local feature analysis for identification of star pattern and choosing the best quadruples from the candidates set.
2. Choosing special pattern for identification
The first task in broadcasting as a live video streaming of a whiteboard content is to identify the whiteboard itself. The only thing that is interested in the whole image frame is only the rectangle of the whiteboard area. All the other stuff that is apart from the whiteboard can be considered as a noise in each frame. The first assumptions is that we have different types of whiteboards. Not all the types of whiteboards can be detected by well-known edge detection algorithms as: Canny's algorithm, Sobel's algorithm, Laplacian algorithm, Prewitt's algorithm, Roberts's algorithm [3]. The problem is that whiteboard can be without edges, or you cannot use non whiteboard like surface if you pick the way of finding whiteboard only with detecting edges, etc. Our solution is to use special 'start' pattern for identification of four corners of whiteboard. Among different shapes the star pattern is tend to be easy drawn by almost any people. Also in terms of pattern analysis it has a special structure that is less common to be met in an ordinary classroom. In many times star image fits into square. It can give us clear vision of how we should transform rectangular whiteboard matrix to get a frontal view and not a view from some angle.
3. Using cascade training algorithms for pattern detection
There are several well-known feature detector algorithms and techniques. The ones that are supported by many ready hands-on tools for image processing is: Haar, local binary patterns (LBP), and histograms of oriented gradients (HOG) [4]. Haar and LBP features are often used to detect faces because they work well for representing fine-scale textures. The HOG features are often used to detect objects such as people and cars. They are useful for capturing the overall shape of an object. So our decision fall into two feature detectors LBP and Haar. Both cascade classifiers need a positive and negative training data. So we have collected sample of about 600 positive and 1100 negative images. As an image we considered the images of hand-written star shape. You can see some of the sample star images in the Figure 1.
Figure 1. Star image samples
In Table 2 you can see the main comparison that was enough for deciding totally which feature detector to use in our special case. So we come to decision to use Haar cascading technique which is mainly used for face detection applications currently.
Table 2
Comparison table of LBP and Haar
Quality Local Binary Patterns (LBP) Haar
Training time 3-4 hours 15-16 hours
Found candidates for a pattern, on average 50-100 10-15
While setting up the training parameters we set the false negative rate very high, so that there is a low probability of missing pattern that are searching. However,
the bottlenecks are that we will get extra candidates that are not converging with the pattern. That means we should make an extra checks and analysis. You can see from the Figure 2 that the false negative rate is high enough for not missing needed candidates. In Figure 2 you can see the results of pattern detection for the picture that is used for training set. In the picture there is a lot of star pattern, also some of the nonstar patterns. So after the training it correctly identified all the star patterns, however it falsely recognized other regions as a positive sample too. In Figure 3 you can see the results for an image from untrained set. And that it mostly covers all the star image patterns except one, anyway we are assuming that it has a high false negative rate, that we need. Because that one region that the training algorithm couldn't recognize is far from regular hand-drawn star image. Nevertheless, you can see that there is a noises or the candidates for a pattern that are not really looks like a star pattern. One of the main reasons is the quality of image and usage of some image filtering techniques. We will discuss about eliminating those noises in the next section.
Figure 2. Results for image from training set
Figure 3. Results for image
3. Local feature analysis for identification of 'star' pattern
In the previous section we discussed about cascading algorithm which finds the candidates for our pattern from the whole image. It's false positive rate is set to high and we can see it in practice by the results that it is high enough. So the next process comes to clearing out noises and leaving only star patterns. After some standard filtering and thresholding procedures [5] we can get binary image for a region. In Figure 4 you can see results of getting binary images for making the further work easier.
****
Figure 4. Transformation results to binary image
By using corner and edge detection algorithms [6] we can find the outer and inner points of the star image. In the Figure 5 we can see the results of applying edge and corner detection algorithms. And the points '1', '2', '3', '4', '5' are considered as an inner points and the points 'a', 'b', 'c', 'd', 'e' are considered as an outer points of the star image. After making proper matches between inner and outer points we will get different triangles. The cost function in our case is a comparison between perfect star image and the given image. And we will calculate squared error between areas of the triangles of perfect star and given hand-drawn input star. Center of the perfect star will be the mass-center of the given star image that we are processing. And the radius of excircle of the perfect star image is defined by the farthest point from the center. The areas A, B, C that are shown in Figure 6 are calculated in the following way for the given radius of excircle R, with an S being area of pentagon:
And the cost function for the given star image is defined by the following formula:
are triangle areas of type A, B'l-B2.....are triangle areas of type B, and the ( ' is
the area of an inner pentagon.
Figure 5. Star sample after corner and edge detection
Figure 6. Perfect star pattern with types of triangles and pentagon
After calculating cost functions for each candidate star pattern we can set some threshold for removing unnecessary noises from the candidate set. Also we can consider all quadruples of stars and choose four star patterns with the lowest cost and compare the figure similarity to the rectangle. By this way we can even cut the candidate set and settle on the four most promising candidates and finish identification process.
Ученый XXI века • 2016 • № 5-4 (18)
4. Conclusion
We presented a new approach of detection of whiteboard area. We used special 'star' pattern for turning any surface into whiteboard one. We also discussed about the structure of the star pattern in the geometric terms. The paper contains information about cascade training algorithms that are applied to the star pattern with a bunch of positive and negative samples. Moreover, we discussed and compared training algorithms against their learning time performance and false positive rates. The results that are shown in figures are being tested using the ready algorithms of OpenCV framework in C++ language. Overall, the paper provides new and a unique way of whiteboard recognition which in the future can turn into broadcasting lectures online for e-learning purposes.
References
1. Edith Manny-Ikan, Osnat Dagan, "Using the Interactive White Board in Teaching and Learning - An Evaluation of the SMART CLASSROOM Pilot Project", Interdisciplinary Journal of E-Learning and Learning Objects, Volume 7, 2011
2. Dansong Z. (2004). "Can e-learning replace classroom learning?" Communication of the ACM, Vol, 47, No. 5, p75-79
3. E. Argyle. "Techniques for edge detection," Proc. IEEE, vol. 59, pp. 285-286, 1971
4. R. Lienhart and J. Maydt. An extended set of haar-like features for rapid object detection. In Image Processing. 2002. Proceedings. 2002 International Conference on, 2002
5. G. Hendeby, J. Hol, R. Karlsson, and F. Gustafsson, "A graphics processing unit implementation of the particle filter," Measurement, vol. 1, no. 1, p. x0, 2007.
6. Li, X., Hu, W. & Zhang, Z. (2007). Corner Detection of Contour Images using Spectral Clustering. IEEE International Conference on Image Processing, 2007. ICIP 2007.
© P.K. Utemuratov, A.K. Akshabayev, 2016
УДК 004.932
РАСПОЗНАВАНИЕ ОБЛАСТИ ДОСКИ С ПОМОЩЬЮ СПЕЦИАЛЬНОГО УЗОРА
П.К. Утемуратов, А.К. Акшабаев
Аннотация. В настоящее время онлайн образование и вебинары очень популярны. Как правило, преподаватели используют специальные программы для записи некоторых данных и транслирования их через интернет для студентов. Как показывает практика, такого рода обмена информацией - писать, рисовать что-то с помощью мыши очень трудно для учителей. Легче всего объяснить все на доске перед аудиторией. Одна из больших проблем является обнаружение доски и транслирование его контента онлайн. Различные работы предлагают различные решения для распознование образов, и мы стараемся использовать некоторые из них, чтобы создать алгоритм для определения области доски. Для этого мы используем простую веб-камеру, а также специальные рисунки на доске. В данной работе предлагается алгоритм для обнаружения лекционной доски из кадров изображения, используя специальный рисунок в виде 'звезды'.
Ключевые слова: лекция, умная доска, распознавание образов, обработка изображений
© П.К. Утемуратов, А.К. Акшабаев, 2016
