CC BY
40
УДК 614.846.6
MOUSTAFA M. KURDI
HYBRID COMMUNICATION NETWORK OF MOBILE ROBOT
AND QUAD-COPTER
Belarusian National Technical University
This paper introduces the design and development of QMRS (Quadcopter Mobile Robotic System). QMRS is a real-time obstacle avoidance capability in Belarus-132N mobile robot with the cooperation of quadcopter Phantom-4. The function of QMRS consists of GPS used by Mobile Robot and image vision and image processing system from both robot and quad-copter and by using effective searching algorithm embedded inside the robot. Having the capacity to navigate accurately is one of the major abilities of a mobile robot to effectively execute a variety ofjobs including manipulation, docking, and transportation. To achieve the desired navigation accuracy, mobile robots are typically equipped with on-board sensors to observe persistent features in the environment, to estimate their pose from these observations, and to adjust their motion accordingly. Quadcopter takes offfrom Mobile Robot, surveys the terrain and transmits the processed Image terrestrial robot. The main objective of research paper is to focus on the full coordination between robot and quadcopter by designing an efficient wireless communication using WIFI. In addition, it identify the method involving the use of vision and image processing system from both robot and quadcopter; analyzing path in real-time and avoiding obstacles based-on the computational algorithm embedded inside the robot. QMRS increases the efficiency and reliability of the whole system especially in robot navigation, image processing and obstacle avoidance due to the help and connection among the different parts of the system.
Keywords: Image processing, Multi-functional Mobile robot, Quad-copter Mobile Robot System, Wireless communication.
Introduction
A robot is an electro-mechanical device designed to accomplish variety of tasks in different aspects of our life. Service robots mainly designed to serve humans in multilateral sectors of education, medical, military, security, emergency cases, entertainment, research, production and agriculture [1].
Unmanned Ground Vehicles (UGV), often called ground robots, which are used both in military and civilian purposes, for the protection and rescue of people in the performance of tasks in life-threatening conditions. Generally, these ground robots used to produce work with explosive ordnance and mine clearance, reconnaissance, search and rescue operations, inspection of buildings and infrastructure, patrolling as well as freight traffic. There are many obstacles (rock, big stones, buildings, mountains, cars and civilians) at outdoor environment which considered as a problem for each mobile robot. To overcome this dilemma, we designed and developed a Quad-copter Mobile Robotic System [2] for agriculture purpose application to help emergency situations (in the liqui-
dation of the threat of explosion, poisoning, fire-fighting and in agriculture for spraying pesticides). The method involves using domestic serial chassis mini-tractor Belarus-123N includes different functions like capturing real world data using digital image processing used to detect its obstacle which is found in its path. To increase the efficiency of mobile robot, we use Phantom-4 Vision Quadcopter to help the mobile in navigation and in direction.
For detecting and avoiding obstacles in their paths, robots should be able to monitor and adapt their surrounding by embedding system to analyze its view and environment [3]. Distinctive cameras from both Mobile Robot and Quad-copter and sensors are used to detect obstacle [4].
For navigation and position, mobile robot has to solve two problems: to determine the current location, and build path of the upcoming journey, free from obstacles.
The first problem is solved by system of visual image positioning from robot and quadcopter. Image with moving robot is analyzed to highlight specific points of their unique ensemble allows
for the repeated analysis of the image, the new range of travel, to reliably determine its location. The problem of detection of obstacles encountered on the way of movement of the robot is very typical. However, robots cope with this task stereo video camera. Depth map construction space in front of the stereo cameras, and accordingly, in the way of movement of the robot, allows the robot to choose the path free of obstacles directions.
This article is concentrating on an international project (Belarus, Kazakhstan and Azerbaijan) to create a multifunctional Quad-copter Mobile Robotic system based on the chassis serial domestic mini-tractors and commercial quad-copter to introduce the concept of controlling such a robot using wireless communication [5].
Belarus 132n mobile robot
During the stage of designing and implementing the mobile robot, we address reliability and simplicity of design of the robotic system. We use platform chassis of tractors "Belarus-132N" [5].
Belarus-132N is an international project (Belarus, Kazakhstan and Azerbaijan) is a four-wheeled small-size unmanned ground terrain vehicle, dimensions 120x120x180 cm (length, width and height respectively), and weighing about 500 kg. The power and traction, originally designed for plowing the soil, it is sufficient to ensure that the movement of the trolley up to 500 kg, or clearing blockages with regular attachments. Physical specifications allow transporting it in simple and small vehicle with a medium wheelbase or a conventional single-axle trailer. General view of the current prototype robotic system with the original attachments designed for firefighting is shown in Fig. 1.
The Belarus-132N Mobile Robot consists of the following systems and components: a video system for driving, positioning and navigation; mechatronic motion control system; on-board computer; telecommunication systems with a remote control unit; staff attachments.
Fig. 1. General View of the Belarus 132N Robotic System
Quadcopter
A quadcopter or quad-rotor is a multi-rotor helicopter that is departed and landed by four rotors. Quadcopter is a small Unmanned Aerial Vehicle (UAV) has significantly increased in recent decades. Quadcopter is one of the most successful vertical take-off and landing vehicle with autonomous flight control and stable hovering capabilities. These features have significantly increased the uses of Quadcopter in all researches in the universities including Electrical, Mechanical, Computer science, and Control system, Real-Time Embedded Systems, Robotics and Navigation. The UAVs are usually used for military, research or civilian operations. The main applications in civilian operations are search and rescue operations, agriculture, fire control, search and rescue operations and inspection of critical areas [6].
A Quad-copter (Quad-rotor helicopter) is a multi-rotor aerial vehicle that is taken-off and lands by four rotors. It has two rotors per axes and each of the axes is aligned with the other where two rotors run clockwise (CW) and two rotors counter-clockwise (CCW) as shown in Fig. 2.
A wireless remote control is used to interact with the flight controller system for changing the flight path. Quad-copter uses the Stable Mode. In this mode, data from accelerometer and gyro sensor is combined to calculate the quadcopter angle. Once the remote control stick is moved to non-idle location, the quadcopter angle is changed accordingly and held at the angle as shown in Fig. 3.
Phantom-4 Vision Quad-copter is a lightweight, multi-functional integrated aircraft and camera with a camera remote-control by DJI VISION APP
Fig. 2. Rotating of rotors per axis of Quad-copter
Throttle control oo oo oo oo Move down Move up Pitch control oo oo oo oo Move forward Move backward
Roll control oo oo oo oo Bend left Bend Right Yaw control OO 00 oo oo Rotate left Rotate right
Normal Speed () High Speed
Fig. 3. Different control mode related to the movement of Quad-copter
Fig. 4. General View of Phantom-4 Pro Vision Quad-copter
as shown in Fig. 4. Its range extender increases Wi-Fi distance to 5 km. Due to its reliability, Phantom-4 Vision Quad-copter use anti-vibration camera platform with single axis stabilization. It operates at low-voltage protection with virtual radar aircraft locator on mobile device. The image processing uses HD Video Recording (H.264 and H.265) and save the sequence of image in JPEG and DNG picture formats and video in MP4/MOV (AVC/H.264; HEVC/H.265) formats. The specifications of Phantom-4 Vision Quad-copter are shown in the Table.
Specification of Phantom-4 Vision Quad-copter
Weight (Battery & Propellers Included) 1388 g
Diagonal Size (Propellers Excluded) 350 mm
Max Ascent Speed S-mode P-mode 6 m/s 5 m/s
Max Descent Speed S-mode P-mode 4 m/s 3 m/s
Max Speed S-mode: 45 mph (72 kph) A-mode: 36 mph (58 kph) P-mode: 31 mph (50 kph)
Max Flight Time Approx. 30 minutes
Satellite Positioning Systems GPS/GLONASS
Forward Vision System
Vision System Backward Vision System Downward Vision System
Altitude Range 0-33 feet (0 - 10 m)
FOV 84° 8.8 mm/24 mm
Lens (35 mm format equivalent) f/2.8- f/11 auto focus at 1 m -o>
Video:
ISO Range 100-3200 (Auto) 100-6400 (Manual) Photo: 100-3200 (Auto) 100-12800 (Manual)
Photo JPEG, DNG (RAW), JPEG + DNG
Video MP4/MOV (AVC/H.264; HEVC/H.265)
Operating Temperature Range 32° to 104 °F (0° to 40 °C)
Capacity (Battery) 5870 mAh
Voltage (Battery) 15.2 V
Battery Type LiPo 4S
Energy (Battery) 89.2 Wh
Net Weight 468 g
Max Charging Power 100 W
Rated Power (Charger) 100 W
Stabilization 3-axis (pitch, roll, yaw)
Controllable Range Pitch: -90° to +30°
Max Controllable Angular Speed Pitch: 90°/s
Angular Control Accuracy ±0.01°
Proposed work
Development and construction of QMRS consists of two parts: Mobile Robot and Quadcopter. The robot is equipped with sensors and cameras used for surveillance operation in outdoor environment area. For surveillance we use image processing concepts. Image processing systems receive data from a variety of cameras and from Quadcopter to determine the distance to an object (obstacle) with the captured output to an external computer [7] to increase efficiency as shown in Fig. 5.
Fig. 5. General view of Quad-copter Mobile Robotics System
Thus, the present project of QMRS has four main levels of computing modules: an external computer (user); on-board computer; multi-channel controller actuators; quadcopter. Each module was necessary to define the specification of functions, commands to initialize them, the service information data and inter-module interfaces [5].
Quadcopter Mobile Robotic System
Automatic detection and tracking of objects observed in a sequence of images is an actual problem, often occurring in the development of navigation systems manned and unmanned aircraft. The Quadcopter Mobile Robotic System consists of Quadcopter and Mobile robot by using image processing to determine the optimal route for mobile robot.
An essential requirement for the developed algorithm is the ability to implement it in the onboard data processing systems, enough effective approach to the automatic detection of objects in the conditions of uncertainty of the location and orientation.
Algorithm of coordination QMRS
The QMRS detects, tracks any moving or stable objects in outdoor environment by using image processing system [8].
Each noticeable object in outdoor environment has following properties:
1. Identifier - a unique number assigned to the new object detection. Number assigned to this object as long as there is no decision on the loss of the object;
2. Coordinates the left edge of the object.
3. Coordinates the top edge of the object.
4. Width.
5. Height
6. Life Time - the number of frames of a video sequence in which the object was present since the first detection
7. Object type.
The composition of the group of robots involved of one main (or base) mobile robot (MBR) with quad-copters/s on board.
The algorithm of solving the problem and specification of functions of robots:
1) MBR has modes of remote, semi-autonomous and autonomous control. On board carries the server on which the loaded geographic information system (GIS) with a digital map of the site location. (Similar map launched on computer operator remote PC). MBR has the GPS - receiver, which determines current coordinates on map; XY rotary video system that allows MBR and computer operator to control the movement and "to look round" during the stop, and other sensors of the current situation.
2) The operator can control the MBR remotely (receiving video data from the board) or in semi-automatic mode, by specifying a point on a digital map, which must arrive, or reference point in the video frame in the direction of which it is necessary to move (or concerning which it is necessary to perform the movement). Data from the GPS-receiver is constantly updated, are linked to a digital map and compared with video analytics.
3) In the simplest case, the quadcopter plays a supporting role by duplicating the rotary video system onboard the MBR. If necessary (or on command from the remote control) the quadcop-ter takes off and hovers over the robot at a height of 15-20 meters. Video data from quadcopters transmitted to the MBR server wirelessly, which increases visibility and allows operating effective-
Fig. 6. Communication of different parts of Quad-copter Mobile Robotic System (QMRS)
ly the movement of remote control mode, and also serve for updating the situation on a digital map of the district site area.
4) Mobile robots can be controlled remotely by the operator, remotely or autonomously MBR. Optionally they can be equipped with video cameras, GPS, other sensors, and technological equipment such as manipulator. The simplest examples of the application - the automatic following of the MBR and the management of transmission of wireless communication or transferring the video picture from hard-to-reach places on the MBR to the operator. In more difficult situation - MBR distributed on the territory according to the given program, specified coordinates with elements of autonomous control.
Mobile Robot and Quad-copter Communication System
Communications and communications protocols will play an important role in mobile robot systems able to address real world applications [10] as shown in Fig. 6.
A mobile robot system might employ communications in any or all of the following modes:
• Between a robot and control user to send command tasking from the user to the mobile robot and to receive back from the robot system status and environmental data.
• Between a quadcopter and control user to send command from the user. Quadcopter synchronizes with user control to see what the camera located at quadcopter records. The user use joystick or remote control in order to direct the quadcopter in space.
• Between robot and quad-copter in a hybrid robot system to support both planning and coordi-
nated execution of cooperative behaviors between robots i.e. mobile robot and quadcopter.
• Between robot and the robotic system developer, to eliminate errors and increment implementation productivity and efficiency by providing software downloading and system debugging implements to exercise and validate hardware as well as software.
• Between quad-copter and the robotic system developer, to eliminate errors and increment implementation productivity and efficiency by providing software downloading and system debugging implements to exercise and validate hardware as well as software.
Conclusion and future work
The objective of this project is to design a communication network for hybrid robots (QMRS) to transmit real-time video back to a host computer for monitoring and controlling purpose. The choice of using WI-FI to develop the communication system resulted in a system that is capable of transmitting data within a local wireless network.
Quad-copter Mobile Robotic System is a real-time obstacle avoidance capability in Belarus-132N mobile robot with the cooperation of quadcopter Phantom-4. The article identifies the method that use vision and image processing system from both robot and quadcopter; analyzing path in real-time and avoiding obstacles based-on the computational algorithm embedded inside the robot.
The next step of the future work will involve positioning and navigation of mobile robot in a merging cooperation of quad-copter in an unknown environment by using digital map, GPS, image processing, and sensors.
References
1. G. Bekey, R. Ambrose, V. Kumar, D. Lavery, A. Sanderson, B. Wilcox, J. Yuh, and Y. Zheng. Robotics: State of the Art and Future Challenges, Imperial College Press, 2008, pp. 89-92.
2. H. Jaffa, A. Soh, W. Hasan, M. Marhaban, S. Rashid. "A Wireless Flipper Robot Using Interface Free Controller," IEEE Trans. on Industrial Electronics and Applications, ISBN 978-1-4673-6322-8, June 2013.
3. C. Karaoguz, T. Rodemann, B. Wrede, C. Goerick. "Learning Information Acquisition for Multitasking Scenarios in Dynamic Environments," IEEE Transactions on Autonomous Mental Development, vol. 5, No. 1, March 2013.
4. I. Mezei, V. Malbasa, I. Stojmenovic. "Task Assignment in Wireless Sensor and Robot Networks", IEEE Trans. telecommunication forum, ISBN 978-1-4673-2984-2, January 2012.
5. A. Dadykin, M. Kurdi, M. Tatur. "Multifunction System of Mobile Robotics," presented at the 3rd International Conference on Electrical, Electronics, Computer Engineering and their Applications, Beirut, April 21-23, 2016.
6. UA-Vision. (2013). "UAV Applications", Portugal, Torres Vedras. [Online]. Available: http://www.uavision.com/#!ap-plications/c1tsl.
7. A. Cavoukian. (August 2012). Privacy and Drones: UnmannedAerial Vehicles. Information and Privacy Commissioner, Toronto, Canada, Privacy by Design. [Online]. Available: http://www.ipc.on.ca/images/Resources/pbd-drones.pdf
8. T. Neumann, A. Ferrein, S. Kallweit and I. Scholl. "Towards a Mobile Mapping Robot for Underground Mines," in Proc. of the 2014 PRASA, RobMech and AfLaI Int. Joint Symposium, Cape Town, South Africa, 2014.
9. I. Sa, P. Corke. "Vertical Infrastructure Inspection Using a Quadcopter and Shared Autonomy Control," Field and Service Robotics, Edited by K. Yoshida, S. Tadokoro. Berlin, Heidelberg: Springer Berlin-Heidelberg, 2014, pp. 219-232.
10. W. Gage. 'Network Protocols for Mobile Robot Systems," in Proc. Space and Naval Warfare Systems Center, Pittsburgh PA, 1997, pp. 107-118.
Поступила После доработки Принята к печати
03.02.2017 27.02.2017 06.03.2017
Moustafa M. Kurdi
КОММУНИКАЦИОННАЯ СЕТЬ ДЛЯ ГИБРИДА МОБИЛЬНОГО РОБОТА
И КВАДРОКОПТЕРА
Белорусский национальный технический университет
В статье рассматривается проектирование и разработка МРКК (мобильного робототехнического комплекса с квадрокоптером). МРКК в составе мобильного робота на базе трактора Беларусь-132М в режиме реального времени обеспечивает возможности преодоления препятствий при взаимодействии с квадрокоптером Фантом-4. Функционально комплекс состоит из GPS, мобильного робота и системы обработки изображений, получаемых как от робота, так и от квадрокоптера, также ипользуется эффективный алгоритм поиска, встроенный внутри робота. Способность точной ориентации является одной из основных возможностей мобильного робота для эффективного выполнения множества рабочих функций, включая различные манипуляции, стыковку и транспортировку. Для достижения требуемой точности навигации, мобильные роботы, как правило, оснащены датчиками на борту для наблюдения непрерывных изменений в окружающей среде, оценки позиции, исходя из этих наблюдений, и корректировки их движения, соответственно. Квадрокоптер взлетает с мобильного робота, обследует местность и передает обработанное изображение наземному роботу. Основной целью исследовательской работы является концентрация внимания на полной координации между роботом и квадрокоптером путем разработки эффективной беспроводной связи с использованием Wi-Fi. Кроме того, определения метода, включающего в себя использование визуального наблюдения и системы обработки изображений от обоих: робота и квадрокоптера, анализируя путь в режиме реального времени и избегая препятствий на основании вычислительного алгоритма, встроенного внутри робота. МРКК повышает эффективность и надежность всей системы, особенно в навигации роботов, обработке изображений и избегании препятствий благодаря системе помощи и связи между различными частями комплекса.
Ключевые слова: обработка изображений, многофункциональный мобильный робот, мобильный робототехни-ческий комплекс с квадрокоптером, система беспроводной связи.
Moustafa M. Kurdi received the BE degree from the Department of Computer Engineering, Beirut Arab University, Lebanon, in 2002, and the MBA degree from the Department of Business Management, Lebanese International University at Lebanon, in 2010. He is presently a Ph.D. candidate at the Electrical and Computer Engineering Department of Belarusian National Technical University, studying control, system analysis and data processing.Currently, he is coordinator of faculty of applied sciences, American University of Technology (Lebanon). He has more than 10 years of academic teaching experience in the fields of computer, software and networking engineering. Mr. Kurdi is member of IEEE. His current research interests include robotics, computer vision, intelligent mobile robots, navigation system, image watermarking, and design automation. Email: [email protected].
