CC BY
3УДК 621.3 Bayaliyev S., Sagymbay G., Tokhmetova K.
Bayaliyev S.
2nd year master student in «Automation and control», Department of «Automation of Production Processes» Abylkas Saginov Karaganda Technical University (Karaganda, Kazakhstan)
Sagymbay G.
2nd year master student in «Automation and control», Department of «Automation of Production Processes» Abylkas Saginov Karaganda Technical University (Karaganda, Kazakhstan)
Tokhmetova K.
Master of Engineering Sciences in «Automation and Control», Senior lecturer, Department of «Automation of Production Processes» Abylkas Saginov Karaganda Technical University (Karaganda, Kazakhstan)
RESEARCH AND DEVELOPMENT OF ALGORITHMS FOR THE FESTO PROCESS LINE MONITORING SYSTEM
Аннотация: this article presents an evolved control system for automatic classification of parts based on their shape, size, height and color based on artificial intelligence technology. Google Net neural network is used to recognize and categorize parts. The system includes an interface to analyze and manage the data, and the sorting process is performed using a PLC from Siemens and a Snap-7 module for Python. The study conducts an in-depth analysis of the current use of neural networks in industry, paying attention to the quality of the raw data and problems with system integration. To train the model, a specialized dataset was trained using the CVAT tool to annotate images of different parts, ensuring high accuracy of the results.
1708
The article emphasizes the importance of this system in improving the efficiency of sorting and quality control processes in manufacturing. The integration of artificial intelligence with industrial automation systems opens new perspectives for the development of smart manufacturing and offers important directions for future research and development in the field of industrial applications of neural networks.
Ключевые слова: Control system, neural networks, vegetable sorting, YOLOv8, industrial automation, Siemens PLC, Django, image annotation.
Introduction.
Automation of manufacturing processes is an important driver in today's global economy, increasing productivity, reducing costs and improving product quality. One of the key technologies in this area is mechanical vision, which enables the automation of monitoring and control tasks on production lines. Mechanical vision is a set of hardware and software that allows cameras and computers to "see" and interpret the environment. On production lines, this opens up new possibilities for automatically sorting objects according to various parameters such as color and size. In addition, mechanical vision systems are able to detect foreign objects, which increases the reliability and safety of the production process. This thesis deals with the development of a monitoring system using mechanical vision for a mechatronic line. The focus is on developing methods and algorithms that can effectively recognize the color and size of objects, as well as detect and classify foreign objects. The aim of the work is to create a system that will provide automated sorting of objects and prompt notification of the dispatcher in case of detection of anomalies on the line. The study of existing technologies and methods used to recognize objects using mechanical vision is an important step in the development of the system. This includes reviewing and analyzing current hardware and software solutions used in industry to automate inspection processes. Selecting and configuring the camera, as well as integrating it with the mechatronic line, requires careful consideration to ensure the reliability and accuracy of the system [1]. Creating software for image processing, recognizing and classifying objects based on their color and size, is a key aspect of this work.
1709
Developing efficient recognition algorithms will allow the system to operate in real time, ensuring high performance and minimal errors. Detecting foreign objects and implementing an alerting system for the dispatcher are critical elements of the monitoring system. The algorithms developed for this purpose must be accurate and fast enough to detect anomalies in a timely manner and prevent possible failures in the production process. The conducted research includes experimental testing of the developed system on the production line, which allows us to evaluate its efficiency and accuracy of operation. Practical testing in real production conditions is an integral part of the work, providing an opportunity to identify and eliminate possible shortcomings of the system.
Materials and methods.
In the rapidly evolving world of industrial automation, the integration of advanced identification and sorting systems plays a key role in improving efficiency and accuracy across many sectors, including manufacturing, logistics and quality control. Such systems are crucial for streamlining operations, minimizing human error, and increasing productivity by automating tasks that have traditionally been both laborintensive and error-prone. The development of mechatronic systems, which seamlessly integrate mechanical, electronic, and computational components, marks a significant advancement in industrial automation. One notable innovation in this domain is the mechatronic eye, a sophisticated device that uses sensors and intelligent algorithms to observe, interpret, and respond to its surroundings much like the human eye. Its capacity to detect objects based on attributes like color and shape and make real-time decisions is essential for today's production lines. Mechatronic eye can enhance the capabilities of automated systems in terms of speed and accuracy, and open up new opportunities for innovation in manufacturing processes. This technology can lead to significant improvements in product quality and production efficiency, providing more accurate and reliable sorting and identification of products in general. This dissertation delves into the creation of advanced algorithms designed to enhance the performance of the mechatronic eye, achieving remarkable efficiency in its operations. The research
1710
emphasizes the concurrent processing of color and shape data, which is essential for cutting-edge sorting technologies. As industries increasingly seek to boost automation and efficiency, the need for sophisticated mechatronic systems has surged. These systems, particularly those featuring advanced sensing technologies like the mechatronic eye, play a vital role in intricate automation processes that demand high precision and flexibility.The mechatronic eye, a prime example of such innovations, stands at the forefront of this technological advancement, embodying the integration of optics, mechanics, and electronics. This device not only automates but also optimizes processes that once depended solely on human vision and decision-making. Its ability to quickly and accurately identify objects based on detailed characteristics such as color and shape makes it an indispensable tool in a variety of industrial settings.
1-Conveyor, 2-Sliders for parts collection, 3-Optical sensors, 4-Induction sensor, 5-Pneumatic cylinder, 6-Pneumatic cylinders with mechanical transmission Figure 1. Components of the sorting unit in the «Festo Technology Line» test bench.
The use of this technology is not only a matter of safety, it is also a matter of integration and intelligence. In addition, yolov8 neuron gels can be used in the same way as the neuron gels in the neuron model, but not in the same mode. To effectively utilize this system, a compatible computer architecture, GoogleNet, and Siemens PLC
1711
controller are essential components. It is recommended to avoid using the Siemens PLC or GoogleNet as a direct hardware component or integrating Python software without proper configuration, as this could potentially lead to malfunction or even physical damage to the Siemens PLC. The "Festo Technologie Gillespie" simulator can be used for training purposes and can also be used for training purposes. The Siemens 300 series PLC 1-series is a component block.
The main features and functionality of Siemens PLCs in this context include:
1. The PLC is a programmable device that allows specific control algorithms to be executed according to the sorting requirements. This provides flexibility and adaptability when working with different products.
2. Siemens PLC input and output modules provide communication with sensors and actuators. These modules act as interfaces, allowing the PLC to receive data from sensors, make decisions based on programmed logic, and control actuators for sorting.
3. Siemens offers real-time processing capabilities, providing immediate response to changes in sorting conditions. This is critical to maintaining the accuracy and efficiency of the sorting process.
The Google Net project computer complements the Siemens PLC, providing higher-level control and integration with the machine vision system. Google Net (or GoogLeNet) is a deep neural network architecture developed by the Google team for the task of image classification. It was presented in 2014 as part of the ImageNet Large Scale Visual Recognition Challenge (ILSVRC). GoogLeNet was one of the first examples of deep convolutional neural networks based on the concept of a network with Inception modules.
Key features of GoogLeNet:
1. Inception modules: GoogLeNet employs an architecture featuring inception modules that integrate convolutions of varying filter sizes (1x1, 3x3, and 5x5) along with max pooling. This design enables the model to capture diverse features at multiple scales, enhancing its overall performance. This helps the network to extract more complex and diverse features from different levels of abstraction.
1712
2. Network depth: GoogLeNet consists of 22 layers, which was significantly deeper compared to previous architectures such as AlexNet. The greater depth of the network allows better processing of complex images and achieving higher accuracy rates.
Figure 2. Monitoring system on GoogleNet.
3. Compression: GoogLeNet uses 1x1 convolution to reduce the number of parameters and computational resources required for the network. This reduces memory consumption and training time, making the model more efficient.
4- Accuracy and Performance: GoogLeNet has shown high accuracy in image classification tasks, winning first place in the ILSVRC 2014 competition with a top-5 error rate of only 6.67%.
5. Lightweighting: Despite its depth, GoogLeNet has a relatively small number of parameters (on the order of 5 million), which is much smaller compared to earlier models such as VGGNet (138 million parameters).
By combining Siemens PLC for real-time control and a PC with Google Net and machine vision, the control system achieves powerful synergies, meeting a variety of product sorting requirements and providing a user-friendly interface for monitoring and controlling the system.
Maximizing the accuracy of the benchmark is important because otherwise its error will exacerbate the error in the subsequent search on the video sequence. For highly loaded recognition, it is reasonable to perform analysis on already captured videos because of too heavy computation for real-time computation. For tasks that
1713
require heavy recognition processing, analyzing pre-recorded videos is often more practical than relying on real-time computations, which can be computationally intensive. This approach is particularly beneficial in the context of benchmark construction, as previously mentioned. However, many scenarios necessitate real-time object recognition, where timely output of relevant information is crucial. In these instances, optimizing the recognition process becomes essential. To achieve this, full segments of the chosen algorithm are applied only to selected reference frames. For intermediate frames, the assumption is made that significant image changes are unlikely, eliminating the need for a complete recalibration of parameters. Key points are expected to remain relatively stable in their positions.
One of the primary applications of these algorithms is in augmented reality technology. The information derived is valuable from multiple perspectives. Firstly, once an object is recognized and its location identified, it becomes possible to overlay both textual and graphical information without needing to place markers on the object. Secondly, the depth map analysis contributes to a more realistic virtual scene. Typically, a virtual scene is simply superimposed over a video feed, appearing to blend with the real world only when the rendered content does not obscure nearer elements.
Thirdly, the depth map facilitates accurate stereometry in virtual reality headsets, a topic that will be explored in greater detail later. Looking ahead, several avenues for advancing this approach can be highlighted. For instance, refining the depth map to account for varying lighting conditions across different areas of an object is a complex challenge, complicated further by shadows and the object's uneven texture. Another significant area for development involves detecting camera movement when markers are no longer visible, in this case, the algorithm must rely on data collected from surrounding objects [3,4].
The most valuable data in this context includes stable points within three-dimensional space that have been verified for accuracy. Ideally, thrsuch points would suffice to maintain positional accuracy in the absence of errors and moving objects, though a larger dataset would be preferable for optimal precision. A more challenging objective is to eliminate the need for markers altogether, which necessitates a more
1714
intricate methodology, as it would lack reference points in the environment. Here, camera movement would be determined by analyzing the movement of distinctive stable key points, and the shape analysis would require comparisons across thror more frames instead of pairs. This introduces significant complexity in descriptor analysis and the classification of key points, especially when forming sets A and B [5].
Object detection has emerged as a crucial technique, experiencing significant growth in popularity in recent years due to its effectiveness in identifying and locating objects within images and videos. This capability has a wide range of practical applications, including the monitoring and tracking of vehicles and pedestrians in traffic systems, as well as the analysis of objects in medical imaging. Detecting and localizing objects accurately and in a timely manner helps improve safety and efficiency in industrial environments. The most optimal method for detecting objects is the YOLO algorithm [6,7].
YOLO (you only watch once) is an object recognition algorithm that has become popular due to its speed and accuracy. It works by partitioning the input invention into a network and representing the bounded frames and class probability for each language network. This process is done by using convolutional neural networks for feature extraction, for object identification.
'•Mil • MtK'ftM, члиШ.««т<>»)[|| i«t»Li • {
(¡conFMme»Г
tor _J «mfid«««, _
1« AtKllfi
t.] juutili.Vl tytt lK).) »ita^.B.il.l Hntr> xK-I.U.I iyivy 1.11.1 !hdp-'0.
чши-t.K.i tyfl^otfiivM.S.I >l!4lytiu-aL».H (ylll« l.Ji,U
[пгт- •, ) A « Г.1«.. <rf ®Lp 1 1 -»Jl.l Г
iiupdjrt*^ ги: - , !hr.-i P"J • pip ! - • J.: ;•:•=
■ ■ *■-. И- c: \uMn№>*<*Vvr!kd№¥4]«C'»\£raj«t> **Sn
i^J^nJ^jnHJ'i^ytfm«,«!*; W't wi rii* Sir
■r I v-v Pi Mtn.py
- ■ ' • "^j.-\j(HfMb.te'VuilT»lyiic«^<*HhrtlMt»tVii»mto«dVve O.i'.Y^lwKt .pt to yelevAl
Figure 3. Loading the Yolo model.
To enable a mechanical vision system to analyze and classify objects on a conveyor line based on several parameters - color, shape, and size - the thesis proposes
1715
an integrated approach that includes the use of machine learning and image processing algorithms. Here is how it works as a result:
1. Image Capture: The mechanical vision system camera continuously captures images of objects on the production line. The camera's high frame rate and resolution provide clear images of objects in motion, which is critical for subsequent accurate analysis.
2. Image preprocessing: The acquired images are subjected to preprocessing, which includes lighting and contrast correction and noise filtering. This is necessary to improve the quality of the visual data before it is analyzed.
3. Segmentation: Segmentation algorithms are applied to select objects in an image and separate them from the background. This can be achieved using thresholding, where the image is converted to two-color (black and white) and objects are selected based on color intensity.
4. Color Detection: Color classification algorithms, such as RGB or HSV color space analysis, are used to determine the dominant color of an object. These methods allow the system to accurately classify objects by color, even under variable light conditions.
5. Shape Determination: Contour analysis algorithms and morphological operations are used to determine the shape of objects. Contour analysis provides a vector representation of the shape of an object, which is important for recognizing and classifying complex shapes.
6. Size measurement: Object dimensions (width and height) are determined based on scaled pixel coordinates of its boundaries. This can be done using geometric moment calculation methods or direct measurements of distances between control points on the object contour.
7. Classification and Analysis: After information about each object is collected and processed, the system uses pre-trained machine learning models to classify objects based on their color, shape, and size. The classification results can be used for sorting, qualitative analysis, or other operational tasks.
1716
This approach illustrates how sophisticated machine vision and AI technologies can seamlessly integrate into industrial workflows, enhancing automation as well as the precision and efficiency of production activities [8,9].
Л -- Square -
Figure 4. Recognizing objects by parameters.
Results and discussion.
Based on the conducted research and experimental data, it can be concluded that the proposed system is highly effective in automating the processes of monitoring and quality control on production lines.
Figure 5. Detecting an object on the lines.
The peculiarity of the developed system is its ability not only to recognize objects by color and size without stopping the conveyor, but also to determine their geometric parameters in dynamics. Such functionality was achieved through the use of
1717
modern image processing and machine learning algorithms, which were integrated into the system software. By instantly analyzing objects as they move along the conveyor belt, the camera integrated into the mechanical vision system classifies them according to predetermined attributes like height, width, and color.This capability marks a considerable advancement in production technology.
When implemented in a practical setting, the system demonstrated high reliability and accuracy, as evidenced by successful testing outcomes on an operational production line. The system effectively copes with the tasks of identification and sorting, which helps to improve product quality and optimize processes.
Thus, the developed mechanical vision system is a powerful tool for automating and increasing the efficiency of production lines, which is important for modern manufacturing [10].
Conclusion.
In the course of this thesis work, a monitoring system using mechanical vision was developed and investigated for a FESTO mechatronic production line. The developed system has shown its efficiency in automatic recognition of objects by color and size, their sorting, as well as in the detection of foreign objects with subsequent notification of the dispatcher about the resulting anomalies on the line. The studies and experiments carried out have shown the system's high level of accuracy and reliability, validated through successful testing under actual production line conditions. By leveraging advanced technologies in image processing, machine learning, and neural networks, significant progress was made in automating monitoring and quality control functions within production environments. Particular attention has been paid to the integration of the mechanical vision system with the FESTO mechatronic line, and the development of algorithms that enable the system to operate in real time. The algorithms created enable effective image pre-processing, identifying objects based on predetermined characteristics, which substantially boosts productivity and reduces error rates. Implementing machine vision systems can greatly enhance both the automation and optimization of production workflows, ultimately leading to lower
1718
production costs and improved product quality. This transformation improves the technological process and contributes to the efficiency of the entire production process. In conclusion, the mechanical vision control system developed for the Festo mechatronic network has significant potential for application in areas that require accurate and reliable observation and sorting of objects. Further research can be focused on extending the functionality of the system, including improving the algorithms for object detection and classification, as well as adapting the system to different production conditions and requirements.
СПИСОК ЛИТЕРАТУРЫ:
1719
1. Szeliski, R. Computer Vision: Algorithms and Applications [Text] - Springer, 2011. - 832p;
2. Bishop, Ch.M. Pattern Recognition and Machine Learning [Textbook] - Springer New York, 2006. - 778p. ISBN 978-0-387-31073-2;
3. Onwubolu, G.C. Mechatronics: Principles and Applications [Textbook] -Elsevier, 2005. - 672p;
4. Corke, P. Robotics, Vision and Control: Fundamental Algorithms in MATLAB [Textbook] - Springer, 2011. - 570p;
5. Batchelor, B.G. Machine Vision for Industrial Applications. In: Batchelor, B.G. (eds) Machine Vision Handbook: Springer London, - 2012. - 60p;
6. Kumar, K., Kumar, P., Kshirsagar, V., Bhalerao, R.H., Shah, K., Pulin, K. A Real-Time Object Counting and Collecting Device for Industrial Automation Process Using Machine Vision [Text], I.E.E.E. Sensors Journal, Volume 23, Issue 12, 2023, Pages 13052 - 13059;
7. Shanmugamani, R. Deep Learning for Computer Vision: Expert techniques to train advanced neural networks using TensorFlow and Keras [Textbook] - Packt Publishing Ltd, 2018. - 310p;
8. Demant, C., Streicher-Abel, B., Garnica, C.. Industrial Image Processing: Visual Quality Control in Manufacturing [Text] - Springer Science & Business Media, 2013. - 369p;
9. Russell, S., Norvig, P. Artificial Intelligence: A Modern Approach [Text], Artificial Intelligence, Volume 175, Issues 5-6, 2011, Pages 935-937;
10. Davies, E.R. Machine Vision: Theory, Algorithms, Practicalities [Textbook] -Elsevier, 2004. - 934p
1720
