CC BY
4Russian Journal of Nonlinear Dynamics, 2019, vol. 15, no. 4, pp. 487-495. Full-texts are available at http://nd.ics.org.ru DOI: 10.20537/nd190407
MSC 2010: 68T40, 70Q05, 68T45
Experimental Investigations of the Control Algorithm of a Mobile Manipulation Robot
Yu. L. Karavaev, V. A. Shestakov, K. S. Yefremov
This paper presents experimental investigations of the control algorithm of a highly ma-neuverable mobile manipulation robot. The kinematics of a mobile manipulation robot, the algorithm of trajectory planning of the mobile robot to the point of object gripping are considered. By realization of the algorithm, the following tasks are solved: solution of the inverse positional task for the mobile manipulation robot; motion planning of the mobile manipulator taking into account the minimization of energy and time consumption per movement. The result of the algorithm is a movement to the point of gripping of the manipulation object; grasping and loading of the object. Experimental investigations of the developed algorithms are given.
Keywords: mobile manipulation robot, motion planning, trajectory discretization, Kinect
1. Introduction
The mobile manipulation robot (MMR) is a mobile platform on which manipulation and gripping mechanisms are placed. In comparison with stationary manipulators, mobile manipulators can perform operations in an extended working area. The direction of development of
Received June 09, 2019 Accepted October 02, 2019
The work of V. A. Shestakov and K.S. Yefremov (Sections 1, 2 and 3) was supported by the Russian Science Foundation under grant 19-71-30012 and performed in Steklov Mathematical Institute of Russian Academy of Sciences. The work of Yu. L. Karavaev (Sections 4 and 5) was supported by the Russian Foundation for Basic Research and the Udmurt Republic, grant No. 18-48-183004 and was carried out within the framework of the state assignment of the Ministry of Education and Science of Russia (1.2405.2017/4.6).
Yury L. Karavaev karavaev_yury@istu.ru
M. T. Kalashnikov Izhevsk State Technical University ul. Studencheskaya 7, Izhevsk, 426069 Russia
Vyacheslav A. Shestakov slafik9526@gmail.com Kirill S. Yefremov ks.efremov18@gmail.com
V. A. Steklov Mathematical Institute of RAS ul. Gubkina 8, Moscow, 119991 Russia
and research on MMR has been of great interest for many years in various fields. The mobile platform provides transport operations, which consist in moving objects within some space. The choice of the type of mobile platform is determined by operating conditions of the robot [1-3]. The design of a mobile platform with omniwheels is the most promising from the point of view of practical application due to the possibility of omnidirectional movement.
The most important characteristics of manipulator that determine the efficiency of maneuvering are the precision of positioning, the number of degrees of mobility, the shape of the working area, the angle and the coefficient of service. These characteristics are determined by the type of the task. The inverse positional problem is the main element of a manipulator control system.
In order to interact with surrounding objects, additional devices can be placed on board the mobile robot: lighting devices and information-measuring devices in the form of various sensors. Encoders, laser or ultrasonic range finders can be used as information-measuring devices. It is possible to use other devices: stereo cameras [4], RGB-D cameras [5] to build a map of the depth of environment and object recognition, as well as time-of-flight cameras installed on the corners of the mobile platform to build a scene 360 degrees around a robot [2].
In previous works, the control tasks of a highly maneuverable mobile platform were discussed in [6-8], and the theoretical research on the mobile manipulation robot was conducted, in which the design was analyzed and the influence of the center of mass position on the trajectory of motion was determined [9, 10].
2. Kinematics
The structural scheme of the MMR on a highly maneuverable platform with omniwheels is shown in Fig. 1. The mobile platform with omniwheels can be represented as three separate kinematic pairs with 1 degree of freedom. To be able to carry out technological operations of gripping and loading an object on the mobile platform, a manipulator is installed on it. The degrees of freedom of the manipulator are chosen so that, when it is installed on the mobile base, the mechanical system will have 6 degrees of freedom.
The kinematic model of MMR in general form is a product of a sequence of matrices of transition from the base coordinate system to the i system:
0t
?T -1 T -I T
i-1
t.
The position of the end effector of the manipulator in the base coordinate system is determined by the column vector equal to the fourth column of the matrix [11], the first three elements of which are the coordinates of the gripper's working point in the base coordinate system.
0t
cos 03 — sin 03 0 -d1 sin 03 + r2
sin 03 cos 03 0 d1 cos 03 + r1
0 0 1 0
0 0 0 1
4 rp
5 T
1 0
0
0
0 cos 05 — sin 05 L2 0 sin 05 cos 05 0 0 0 0 1
7 t
t
cos 06
sin 0
1 0
0
0
0 cos 04 — sin 04 0 0 sin 04 cos 04 L1 0 0 0 1
0 sin 06 0
1 0 L3 + L4 0 cos 06 d3
0 0 1
0 10 20 30 40 Y, cm
(a)
Y, cm w rn An 20
-50 -40 cm
(b)
Fig. 2. The service area a) of the manipulator and b) of the mobile manipulation robot.
The service areas of the manipulator and of the mobile manipulation robot [9, 10] are shown in Fig. 2.
Combining 3DOF (2P1R) mobile platforms with omniwheels and 3DOF (3R), the manipulator allows a significant increase in the functionality and an expansion of its service area.
3. Motion planning
The omniwheel platform control can be implemented using the control torque or angular velocity of wheels [6, 7]. In the case of angular velocity control, the dynamic processes of the system are determined by the parameters of drives. Moment control allows one to take into
account the changing parameters of the system, for example, the position of the center of mass. However, such a control is difficult to implement under real conditions.
Next, we consider the planning of the robot movement based on the control of the mobile platform using the angular velocities of the wheels, based on the model that ensures no slipping at the contact point of each wheel [6].
The movement of the manipulator in order to grip the object from the surface of the movement is determined unambiguously in view of the absence of redundancy degrees of freedom. The number of degrees of freedom of the mobile platform allows the robot to move to a given point in several ways:
1. Turning the platform at angle p and moving over distance r.
2. Moving the platform at angle p over distance r, turning the platform at angle p.
3. Moving the platform at distance r with a change of orientation in the process of motion at angle p.
fi = tf i + Gi
(1)
Gi
1
(ai, ti)h.
■(v + uJri, ai),
0 -1 0 1
ei, x
Fig. 3. Structural scheme of the mobile platform.
where tfi is the angular speed of rotation of the i wheel, v = (vx ,vy) is the linear velocity of the mobile platform, w is the angular velocity of the platform, hi is the radius of the i wheel, ri, ti, ai are the vectors characterizing the position, plane and direction of the roller axis of each wheel, they are constant in the moving axes.
Thus, the direct control of the movement of the mobile platform is carried out using the angular velocity of the wheels.
The modeling of the system to determine the most optimal way of movement is carried out. The movement of the mobile platform in different ways for the same period of time is considered.
Further, the kinetic energy of the wheels of the mobile platform is considered as the energy spent on control. The kinetic energy of the wheels consists of the kinetic energy of translational and rotational motions:
n
n
^Ti = ^
i=l i=l
im.i(v + OjJri, V + OjJri) + ^Ii^i + ^IiU
where mi is the mass of the i wheel, Ii is the moment of inertia of the wheel relative to the axis of rotation, and Ii is the moment of inertia of the wheel relative to the diameter.
Based on the comparison of the energy, which is spent on the control of the mobile platform, with the help of numerical modeling, it is concluded that the variant with simultaneous angular and linear displacement is optimal. Also, this option is the most preferable because in this method only one cycle of acceleration-braking is carried out.
j
The formulation of the motion task is as follows: the mobile manipulation robot from a point with coordinates p0 = (x0,y0) (motion in the plane is considered) with the initial orientation needs to move to the end point with coordinates p = (xk,yu) and the end orientation . The maximum angular speed of rotation of the wheels ,J/max and acceleration time tp are set. Then it is possible to determine the components of the velocity vector for a given maximum speed of rotation of the wheels: vmax = (Xmax,ymax,^max). Let s = (sx,sy,s^) be the path that will be overcome during acceleration when the speed vmax is reached:
vmaxt
_ bP
2
To ensure simultaneous linear and angular movement, it is necessary to determine the maximum of required time:
(2tv, Pi < 2si,
7; 21 + Pi ~p. >2s (2)
bp T ^max ) ft ^
T = max(Ti,T2 T).
Based on the time value obtained, the acceleration and maximum velocity values must be redetermined. Redefinition is required to ensure the same time spent on linear and angular displacement.
Position feedback has been introduced to improve the precision of positioning. In that case, the speed v = (vx,vy, w) of the mobile platform depends on the distance S = (xactuaifactual, factual) to a given point. xactual, yactual and 0actual are the x and y coordinates and the angle of rotation of the mobile platform relative to the starting point p0.
The components of the velocity vector of the mobile platform are determined as follows:
„max o
S
Pi < 2si:
0 < Si < si,
Vi ~ \ 2vfxfa-Sj) - ^
Pi
, si < Si ^ Pi
Pi > 2si:
■ v-
xSi
0 < Si < si,
Vi = { vmax, si < Si < (Pi - si), (4)
Graphs of variants of the speed diagram obtained according to this algorithm are shown in Fig. 4.
Similarly, a calculation for the angular velocity w is carried out. The minimum of energy required for movement is provided by simultaneous linear and angular movements. The minimum of time is provided by using the maximum of acceleration and speed of the mobile platform.
v, m/s
v, m/s A
0 Si pi s, mm
0 s.
Pi — Si Pi s, mm
(a) (b)
Fig. 4. Graphs of the speed diagram.
4. Control system
The scheme showing the structure of the control system of the mobile manipulation robot is presented in Fig. 5.
The vision system (Kinect 2.0) allows one to obtain data about the environment in the form of rgb images and depth maps. The navigation system provides the coordinates of the robot position. In the laboratory system Vicon Motion Capture is used as a navigation system. On the basis of these data, the object of manipulation is recognized and the control actions are calculated in the form of angular velocities of the wheels and rotation angles. Microcontrollers provide control at the drive level.
Control system
Fig. 5. Structure scheme of the control system.
Let us consider the description of object detection. To minimize the computational cost, RGB-based image recognition is selected. To search for symmetric objects of monotonous color, the color recognition method is selected. The purpose of this method is to find the moment of the image [12-14]. The coordinates of an object in space are determined based on the coordinates of the center of the object in pixels and the distance to it that is obtained by Kinect [14]:
7U Cu 7V cv
x = d—-—, y = a—-—,
fu fv
where d is the distance to the object returned by the depth sensor; u, v are the coordinates of the object in pixels;
Cu, Cv are the coordinates of the optical center in pixels; fu, fv is the focal length f, measured in pixel width and height.
The coordinates of the object are necessary to determine control actions for movement of the omniwheel mobile platform to the capture point.
5. Experiment
Consider an example of planning a motion trajectory for an MMR in order to capture certain objects that are within the visibility range of a robot's vision system. The object that needs to be moved has coordinates x, y in the coordinate system associated with the robot. The coordinates are obtained using data from the vision system. The point to which it is necessary to move in order to grip the object has coordinates xk and yk. Then the control actions are formed as follows:
1. Based on the set maximum wheel speed, the time required to move the robot to a given point is determined according to Eq. (2).
2. According to the time value obtained, the dependence of the speed vector's components of the robot movement is determined by Eq. (3) or (4).
3. Individual angular speeds of the wheels of the mobile platform are determined by Eq. (1).
The photos of the prototype of a highly maneuverable mobile manipulation robot are shown
in Fig. 6.
An experiment was conducted for specific values of the coordinates of the object's capture point: xk = 300, yk = 400. Graphs of the trajectories obtained are shown in Fig. 7. The trajectory is divided into three sections (see Fig. 7): acceleration corresponds to interval 1, movement with constant speed corresponds to interval 2 and deceleration corresponds to interval 3. The analytical description of the movement speed on these segments corresponds to Eq. (4).
The results obtained prove the efficiency of the proposed algorithm for controlling a mobile robot. The trajectory of the mobile robot is somewhat different from linear, however, the position feedback ensures the achievement of the target point with a given orientation.
On the basis of the proposed motion planning, control programs for the object recognition are developed in the C++ programming language in the ROS (Robot Operating System) using OpenCV libraries.
The control program of motion planning for the mobile manipulation robot are developed in the C# programming language in the Microsoft Visual Studio. A video of the experiments can be viewed at: https://youtu.be/docNyRm-vUw.
Stepper motors
Dimensions of the robot 500x500x600 (mm)
Fig. 6. Prototype of a mobile manipulation robot.
(b)
Fig. 7. Graphs of dependences: a) trajectory of the mobile robot and b) orientation of the mobile platform.
6. Conclusion
This work presents the algorithm and control system of a highly maneuverable mobile manipulation robot, providing recognition of objects; movement of the mobile robot to the point of object gripping and its loading on the mobile platform. Based on investigations of the mathematical model of the mobile robot, an optimal algorithm for planning the trajectory of motion by the criterion of the minimum energy and time is proposed. The investigations of the efficiency of the developed algorithms were carried out using the prototype of the highly maneuverable mobile manipulation robot.
References
[1] Aviles, O., Mauricio, F., Mauledoux, M., and Rubiano, O., Electronic Architecture for a Mobile Manipulator, International Journal of Online Engineering (iJOE), 2018, vol. 14, no. 2, pp. 133-142.
[2] Rohrig, Ch., Heß, D., and Künemund, F., Motion Controller Design for a Mecanum Wheeled Mobile Manipulator, in Proc. of the IEEE Conf. on Control Technology and Applications (CCTA, Mauna, Lam, Hawaii, USA, 2017), pp. 444-449.
[3] Hvilshoj, M. and Bogh, S., Little Helper: An Autonomous Industrial Mobile Manipulator Concept, Int. J. Adv. Robot. Syst., 2011, vol.8, no. 2, pp. 80-90.
[4] Domel, A., Kriegel, S., Kaßecker, M., Brucker, M., Bodenmuller, T., and Suppa, M., Toward Fully Autonomous Mobile Manipulation for Industrial Environments, Int. J. Adv. Robot. Syst., 2017, vol. 14, no. 4, pp. 1-19.
[5] Nieuwenhuisen, M., Droeschel, D., Holz, D., Stückler, J., Berner, A., Li, J., Klein, R., andBehnke, S., Mobile Bin Picking with an Anthropomorphic Service Robot, in Proc. of the IEEE Internat. Conf. on Robotics and Automation (Karlsruhe, 2013), pp. 2327-2334.
[6] Borisov, A. V., Kilin, A.A., and Mamaev, I. S., An Omni-Wheel Vehicle on a Plane and a Sphere, Nelin. Dinam., 2011, vol.7, no. 4, pp. 785-801 (Russian).
[7] Kilin, A. A. and Bobykin, A. D., Control of a Vehicle with Omniwheels on a Plane, Nelin. Dinam., 2014, vol.10, no. 4, pp. 473-481 (Russian).
[8] Kilin, A., Bozek, P., Karavaev, Yu., Klekovkin, A., and Shestakov, V., Experimental Investigations of a Highly Maneuverable Mobile Omniwheel Robot, Int. J. Adv. Robot. Syst., 2017, vol. 14, no. 6, pp. 1-9.
[9] Karavaev, Yu. L. and Trefilov, S. A., Deviation Based Discrete Control Algorithm for Omni-Wheeled Mobile Robot, Nelin. Dinam., 2013, vol.9, no. 2, pp. 91-100 (Russian).
[10] Karavaev, Yu. and Shestakov, V., Construction of a Service Area of a Highly Maneuverable Mobile Manipulation Robot, Intellekt. Sist. v Proizv., 2018, vol.16, no. 3, pp. 90-96 (Russian).
[11] Zenkevich, S.L. and Yushchenko, A. S., Fundamentals of Manipulation Robots, Moscow: MGTU, 2004 (Russian).
[12] Prokop, J., Richard, P., and Reeves, A., A Survey of Moment-Based Techniques for Unoccluded Object Representation and Recognition, CVGIP-Graph. Model. Im., 1992, vol. 54, no. 5, pp. 438-460.
[13] Mercimek, M., Gulez, K., and Mumcu, T.V., Real Object Recognition Using Moment Invariants, Sadhana, 2005, vol. 30, no. 6, pp. 765-775.
[14] Lachat, E., Macher, H., Mittet, M.-A., Landes, T., and Grussenmeyer, P., First Experiences with Kinect v2 Sensor for Close Range 3D Modelling, in ISPRS: Internat. Arch. of the Photogram-metry, Remote Sensing and Spatial Information Sciences (2015): Vol. XL-5/W4, pp.93-100.
