CC BY
3Russian Journal of Nonlinear Dynamics, 2022, vol. 18, no. 5, pp. 843-858. Full-texts are available at http://nd.ics.org.ru DOI: 10.20537/nd221213
NONLINEAR ENGINEERING AND ROBOTICS
MSC 2010: 93C85
Vision-Based Robotic Comanipulation for Deforming Cables
K. Almaghout, A. Klimchik
Although deformable linear objects (DLOs), such as cables, are widely used in the majority of life fields and activities, the robotic manipulation of these objects is considerably more complex compared to the rigid-body manipulation and still an open challenge. In this paper, we introduce a new framework using two robotic arms cooperatively manipulating a DLO from an initial shape to a desired one. Based on visual servoing and computer vision techniques, a perception approach is proposed to detect and sample the DLO as a set of virtual feature points. Then a manipulation planning approach is introduced to map between the motion of the manipulators end effectors and the DLO points by a Jacobian matrix. To avoid excessive stretching of the DLO, the planning approach generates a path for each DLO point forming profiles between the initial and desired shapes. It is guaranteed that all these intershape profiles are reachable and maintain the cable length constraint. The framework and the aforementioned approaches are validated in real-life experiments.
Keywords: robotic comanipulation, deformable linear objects, shape control, visual servoing
1. Introduction
Deformable linear objects (DLOs), such as cables, ropes and sutures, are involved in innumerable everyday life scenarios, such as cable management in industry or at home, thread packing
Received September 15, 2022 Accepted December 12, 2022
This work was supported by the Russian Science Foundation (project no. 22-41-02006).
Karam Almaghout
Robotics and Computer Vision Institute, Innopolis University ul. Universitetskaya 1, Innopolis, 420500 Russia
Alexandr Klimchik [email protected]
School of Computer Science, University of Lincoln Brayford Pool, Lincoln LN6 7TS, United Kingdom
in production lines or suturing in medical surgeries. Due to the fact that DLOs have a high degree of freedom, which makes the modeling and controlling of these objects highly difficult and expensive, the automation of DLOs manipulation is still an open challenge in robotics community. Over the past several years there has been an increase of interest in research conducted into the robotic manipulation of DLOs [1-3].
Shape control of a DLO is a common task which has many practical implementations, such as cable routing in automotive industry [4] and cable management [5]. The shape control task aims to deform the DLO into a designated shape. This task consists of two main stacks, which are perception and manipulation planning of the DLO [6].
Perception of DLOs includes recognition, state estimation, and tracking using sensory systems like vision, force, tactile, and others [7, 8]. Vision sensors are widely used for perception in the robotic manipulation since they are affordable and proper for tasks that include objects detection and position localization for both rigid and deformable objects [9, 10]. In the vision-based robotic manipulation of DLOs, a group of researchers considers using add-on markers as feature points and manipulating these points [11-13]. The main drawback of this method is that it is not practically feasible to add these markers each time a DLO manipulation is performed. The other group deals with the DLO contour points as the DLO features [14, 15]. This method is more practical than the former, but it requires more computations since it deals with a relatively larger number of points (the contour points).
The manipulation planning stack, also known as the control stack, is to compute how the DLO will behave and deform by applying certain manipulation sequences of the robotic system. Some studies discussed the manipulation planning of DLOs using single-robot systems. Researchers in [12] presented a model-based shape control for DLOs grasped by a manipulator. The DLO has markers as feature points, and the proposed method manipulates these feature points towards their reference points in a sequential manner. In [16] a cell concept to robotize the wire routing task using a single manipulator equipped with an innovated light weight end effector is developed. Multirobot systems are also used for the shape control tasks, since they improve the performance of the systems in terms of accuracy, computational cost, and flexibility. Thus, they have superiority over single-robot systems in such tasks [17]. In [18], dynamic control schemes are developed based on the discrete elastic rod model of the DLO to achieve the shape control of a flexible cable using human-like robotic systems. Other works proposed manipulation planning approaches based on an online estimation of the local deformation model of DLOs, assuming that a small change in the DLO is linearly related to a small displacement of the robot [14, 19, 20]. Researchers in [13] developed a shape control adaptive control scheme where the Jacobian matrix is estimated offline based on collected data and updated online during manipulation using neural networks.
In this paper, we introduce a new framework to achieve shape control of a DLO based on bimanual manipulation. For simplicity, we will refer to DLOs as "cable" throughout the rest of the paper. Based on visual feedback and classical image processing methods, a featureless cable is captured and modeled as a set of uniformly distributed points. An approximate model of Jacobian matrix mapping between these cable points and the configuration of the robots' end effectors is developed as the manipulation planning model. The aim of this Jacobian is to compute the required motion of the robots' end effectors to deform the cable into a desired shape. Furthermore, we propose an additional algorithm to generate the waypoints of each cable point starting from the initial position towards the desired one. These waypoints are represented as an intermediate profiles maintaining the cable length constraint during the manipulation. Figure 1 depicts the proposed framework, where the cable is fixed to the end effectors and is tracked by
Desired Cable Shape
an RGB camera. Experiments are carried out on a real system to validate the performance of the developed framework. The main contributions of this work are:
1. Develop a new framework for shaping featureless DLOs.
2. Propose a new virtual feature points (VFPs) algorithm for DLOs representation and tracking.
3. Introduce a new manipulation planning approach that describes the motion of the DLO feature points as a function of the motion of the robots, and maintains the DLO length constraint during the manipulation by a set of intermediate profiles.
4. Validate the proposed framework in real-world experiments.
The remainder of this paper is organized as follows: Section 2 presents preliminaries including cable representation and the assumptions considered. The proposed approaches in perception and manipulation planning methods are described in Section 3. The details of experiments and the results are discussed in Section 4. Finally, we conclude and address some future works in Section 5.
2. Preliminaries
Figure 2 shows a cable grasped by two manipulators at its two ends and the desired shape. The cable and the desired shape are represented by N uniformly distributed points. ls is the
distance between each two points. Let P = ordinates of the cable points and T = nates of the shape points, where pi =
T T T T
pi pi pi ••• Pn
T
G
x1 be the set of co-
tT tT tT L1 L2 l3
T
tT
T
xpi ypi
and t =
G R2N x1 be the desired coordi-
T
xti Vti ,
for i = 1, 2, ...,N.
7 Pi
Fig. 2. Schematic of the robots' end effectors, the cable (grey), and the desired shape (green).
(a) The cable at full-stretched configuration
(b) The cable at any random configuration
Fig. 3. The cable grasped by two robots in fully-stretched and random configurations
Let R =
T T rT rr
1 '2
T
G
5x1
be the configuration set of the robots' end effectors, where rm =
xm ym fm
T
, for m = 1, 2.
Let Dim be the Euclidean distance between pi and rm when the cable is fully stretched, Fig. 3a, and let dim be the Euclidean distance between pi and rm at any configuration of the cable, Fig. 3b. Dim and dim are given as follows:
dim \\Pi rm
D = n 1
im im" si
(2-1) (2-2)
where nim is the order of p^ with respect to rm. Hence, we have Dim — dim ^ 0 for all cable configurations.
The objective of the task is to deform the cable to fit the desired shape, i.e., guiding the cable points towards the desired ones. The cable points are continuously tracked by an RGB camera placed perpendicularly over the workspace plane. Since we are able to control the end effectors' configuration, we need to formulate the motion of the cable points as a function of motion of the end effectors. To achieve this goal, we consider the following assumptions:
• The manipulation is quasi-static, which means that the robots manipulate the cable at a relatively slow velocity.
• The cable ends are fixed to the end effectors, thus, the grasping task is beyond the scope of the paper.
The cable is unstretchable.
Both the initial and desired shapes are reachable for the robots within the workspace and the camera frame.
Both the initial and desired shapes have no overlapping, and the following inequality holds for initial and final shapes and during the manipulation:
n
IP1-P2I <2- (2-3)
3. Methodology
3.1. Virtual Feature Points (VFPs)
In this work, we are dealing with a featureless cable, where no markers (features) are added to the cable. Thus, to track the cable during the task, we have developed a new cable perception algorithm to model the cable as a set of virtual feature points (VFPs) to be tracked during the manipulation. Compared to the perception methods in [11-13], where different kinds of physical markers are used, the VFPs method is practically more efficient. Furthermore, it requires less computations than the methods that consider the cable contour [14, 15], since it deals with a significantly smaller number of data compared with contour points. Based on classical image processing techniques, the proposed method takes a colored video frame as an input. This frame includes the cable attached to end effector tools, where one of the tools is marked, Fig. 4. The algorithm starts detecting the marked tool as the starting point based on a color-based segmentation. Next, the cable is detected and segmented by a sequence of edge-detection methods, and morphological operations. Once we get the cable segmented, we apply the Guo-Hall thinning algorithm [21] to get the center line of the cable with a one-pixel width. Then the algorithm starts sampling the cable from the end grasped by the marked tool towards the other end of the cable. The sampling is done by applying a circle-shape mask, where the center of this mask is placed at the starting point and the intersection point between the cable and the mask is considered as a VFP. The mask slides to the obtained VFP, checks for the intersection point as a new VFP and so forth towards the end of the cable. Finally, the algorithm returns the cable modeled by N uniformly distributed VFPs, where the distance between each two points ls equals the radius of the mask. The steps of the VFPs algorithm are illustrated in Algorithm 1 and Fig. 4.
cable
(marked tool)
Fig. 4. Input of the VFPs algorithm: the camera frame shows a cable grasped by two end effector tools, and the marked tool
3.2. Manipulation planning
Once the cable is sampled into N feature points, the desired shape can be represented similarly by N desired points. Thus, the manipulation planning stack can be re-stated as guiding the cable points towards the desired ones. The key problem of this planning is to compute the Jacobian matrix that maps the motion of these cable points to the motion of the robots' end effectors. In this work, we propose an approximate model to compute the Jacobian matrix. First, let us consider that each cable point pi is rigidly connected to the robot's end effector rm, at the fully-stretched configuration. Thus, the coordinates of pi with respect to rm can be given as
pi
xrm + Dim yrm + Dim sin
(3.1)
where d1 = , d2 = n — y2.
Algorithm 1: The Virtual Feature Points (VFPs) Algorithm
1 input:
2 The camera frame.
4 output:
5 A frame with sampled cable.
6 P set of the cable VFPs coordinates.
8 begin
9 10 11
12
13
14
15
1. Get starting point ^ Segment the marked tool.
2. Detect the cable ^ Applying segmentation and edge-detection techniques.
3. Extract the cable centerline ^ Applying morphological operations and thinning algorithm.
4. Place the center of the circular mask on the starting point.
5. Get the VFP ^ The intersection point between the mask and the centerline feature point.
6. Slide the mask to the new VFP.
7. Repeat 5 and 6 till the end of the cable centerline.
3
7
-----Cable centerline O Mask
• Starting point • Feature points
Fig. 5. The steps of generation of the virtual feature points by sliding the mask along the cable and extracting the intersection points
Taking the first derivative of Eq. (3.1), we obtain
pim
xrm ßmPmDim ^m Vrm + ßm'~f>mDim ^m
(3.2)
where ß1 = 1 and ß2 = —1.
Equation (3.2) can be rewritten in the following form:
pi = Jim rm, (3.3)
where Jim is the Jacobian matrix that maps the motion of rm to pi and is given as follows:
1 0 -PmDim sin e„
o 1 PmDim c°s em
(3.4)
During the cable manipulation, it can be observed that the motion of pi undergoes two factors:
a) The diminishing rigidity property of the cable, which means that the rigid connection between pi and rm is inversely proportional to the order of pi with respect to rm. In another expression, the grasped points move rigidly with the end effectors, the points near the grasped ones move almost rigidly and the farther points move less rigidly. Berenson, in [22], showed that this decrease in rigidity is exponentially proportional to the distance from the grasped points.
b) The cable point pi, shown in Fig. 3, tends to behave as if it is rigidly connected to r,m as much as Dmi — dmi converges to zero.
Based on the above, we define a new function representing the diminishing rigidity as an exponential function:
vim = e-Km(Dm-dm), (3.5)
where Kim is the rate of decrease in rigidity and is set experimentally.
Since the cable is guided by its two ends, the motion of point pi is subject to the motion of both end effectors, r1 and r2. Thus, an additional function aim that describes the motion of pi as a ratio of the motion of the end effector rm is
n ■
= 1_ '±mL
^im ^ n
(3.6)
Then we multiply the Jacobian Jim in Eq. (3.4) by aim and ¡iim to obtain the final formula that maps the motion between the pi and rm:
Jim aim№i:
1 0 -PmDim sin 0 1 Pm Dim cos
Finally, the Jacobian of pi undergoes the guidance of two robots and is given as
Ji
Jil Ji2
2x6
(3.7)
(3.8)
Since the cable is manipulated in a low velocity (quasi-static manipulation), we rewrite Eq. (3.3) for pi undergoing the motion of two end effectors as
and the formula for all cable points is
AP
APi — Ji AR,
J2Nv 6 AR6
2Nx l — J2Nx6AR6xl
(3.9) (3.10)
where AP is the displacement between the current position of the cable points and the desired
AP = T - P. (3.11)
Thus, the motion of the end effectors required to guide the cable towards the desired shape can be computed as follows:
AR = J+AP, (3.12)
where J + is the Moore - Penrose pseudo-inverse:
J+ = JJT J )-1 JT (3.13)
and AR is bounded to avoid any excessive changes in motion during the manipulation.
Compared to the other methods proposed in the literature [13, 14, 19, 20], this method does not require precollected data and involves less computations and amount of data to be processed, which will reduce the execution time and storage memory. Table 1 shows a qualitative comparison between the proposed model and some recent related models in the literature.
Table 1. Qualitative comparison between our model and other recent models
Analytical Numerical Data-driven Online estimation
The proposed approach /
S.Jin et al. [19] / /
R. Lagnea et al. [20] / /
Y. Yang et al. [13] /
J.Zhu et al. [14] / /
To evaluate the proposed Jacobian model, an average error, eavg, and maximum error, emax,
are considered as the metrics of performance:
1 N
e-avg = Jj^WU - (3-14)
i=1
emax = max(|T - P|), (3.15) where |Z| e Rrax1 is the element-wise absolute value of ( e Rrax1.
3.3. Intermediate profiles
To avoid excessive stretching and maintain the length constraint of the cable during the manipulation, we propose an algorithm that generates waypoints of the cable points starting from the initial shape towards the desired one. These waypoints are represented as a set of intermediate profiles where all of them are reachable and the length of these profiles equals the cable length, Fig. 6. Thus, the cable will move along these profiles towards the desired one and the cable length constraint will be maintained.
This algorithm computes the distance dis between each cable point and its corresponding point in the desired profile. Then the number of the intermediate profiles is obtained by dividing the maximum distance dismax by a user-defined step A (which represents the maximum allowed step between each two waypoints). Then, for each generated intermediate profile, the algorithm
Desired Shape
Intermediate Profiles
Initial Shape
Fig. 6. Intermediate profiles representing the waypoints (blue) of the cable points (red)
Algorithm 2: The intermediate profiles algorithm
1 input:
2 T ^ the desired shape points
3 P ^ the cable points
4 A ^ a user-defined step
6 output:
7 profiles ^ set of intermediate profiles
9 begin
10 11 12
13
14
15
16
17
18
19
20 21 22
23
24
dis = T — P dismax = max(dis) n0 = [A-1 dismax] step = no • dis
i = 0
profilei = P
while max (T — profileJ > A do
i = i + 1
profilei = profile¿_ i + step
E ^ the distance between each two points in profilei if max E > ls then
step = dis • no step = 0.8 • step i = i — 1
else
step = dis ■ n0
5
8
checks whether it equals the cable length, otherwise the step size is reduced. The pseudo-code of this method is illustrated in Algorithm 2.
Once the intermediate profiles are generated, the system starts computing the required AR to guide the cable from the current shape to the desired shape along the intermediate ones.
4. Experiments results 4.1. System setup
The experimental setup of this work is shown in Fig. 7. Two KUKA LBR iiwa 14 manipulators are used. Each manipulator has 7 degrees of freedom and is equipped with a special tool to attach the cable's end. The utilized camera is Intel Realsense D435 mounted perpendicularly to the workspace plane. Images are captured at a rate of 16 frames per second with a size of 640 x 480. To track the position of the cable VFPs in the workspace plane in mm, the camera is calibrated using a chessboard and OpenCV library [23, 24] and the camera intrinsic matrix K is obtained. Then the cable points coordinates in the physical image coordinates are computed as follows:
(4.1)
x' fx 0 cx u
y' = 0 f C yy v
1 0 0 1 1
k-1
where x' and y' are the physical cable point coordinates in the image plane, in mm. u and v are the cable point coordinates in the image plane, in pixels. fx and fy are the focal lengths. cx and Cy are the principal point coordinates [25].
Fig. 7. The hardware setup Next, we obtain the position of the points in the camera coordinates as follows [26]:
xc x'
yc = zc y'
zc 1
(4.2)
where zc is the normal measurable distance between the cable and the camera pinhole. Since the workspace is over a 2D plane and the plane incline is calibrated, zc is constant for all points within the workspace. Figure 8 shows the coordinate systems of the camera {Xc, Yc, Zc}, workspace {Xw, Yw, Zw}, and the robots {Xrm, Yrm, Zrm}.
Robot 1
Camera
Yc/1 Robot2
\zc
* til
Fig. 8. Coordinate systems
In the last step, we transform the position of the points to the workspace coordinates as follows:
xw 0 -1 0 0 xc
Vw -1 0 0 0 Vc
zw 0 0 -1 zc zc
1 0 0 0 1 1
(4.3)
T
where T is the transformation matrix (extrinsic matrix) from the camera coordinates to the workspace coordinates. In this hardware setup, the camera plane is parallel to the workspace plane, thus, the distance between the workspace plane and the camera plane zc is only required. In general cases, the extrinsic matrix requires more steps to be obtained if the camera plane is not parallel to the cable operational plane.
The robots are connected to a computer via a LAN network. The algorithms pipeline is built in the Robotic Operating System (ROS) framework [27], where the image processing node is written in Python based on the OpenCV library, and the robots' control computation and command nodes are written in C++ based on the ROS metapackage for the KUKA LBR iiwa developed by Hennersperger et al [28]. The robots' linear and angular velocities are limited to 0.030 m/s and 0.050 rad/s, respectively.
4.2. Results
The task starts by detecting and modeling the cable into N points. The camera captures the workspace, and the VFPs algorithm processes the captured frame to extract the starting point and cable centerline. Then it starts applying the mask to generate N virtual feature points of the cable. Figure 9 shows the block diagram of the VFPs algorithm process. Figure 10 shows the output of the VFPs algorithm for different cases and segment lengths ls, where ls is defined as the radius of the circular mask.
The next step is generating the intermediate profiles, which are the waypoints of the cable points. Figure 11 shows the generated intermediate profiles for different cases. For these experiments we considered A = 0.5ls. It can be seen that the generated profiles have the same length
Output frame
Mask
Fig. 9. The VFPs algorithm process. The images of the cable centerline, starting point, and the mask are in inverted color
(a) ls = 15 mm (b) ls = 25 mm (c) ls = 40 mm
Fig. 10. The VFPs algorithm output for different configurations and segment lengths ls
Fig. 11. The generated intermediate profiles (light blue) between the cable initial configuration (red) and the desired shape (green)
of the cable and they have a small step among each other, which ensures that the robots do not make any undesirable large displacement that may lead to overstretching of the cable. Thus, the cable length constraint will be maintained during the task.
Once the intermediate profiles are generated, the robots start cooperatively manipulating the cable towards the designated shape throughout these profiles. Figures 12 and 13 show two experiments. In both experiments, the cable approached the desired shape. The average and maximum errors of these experiments are listed in Table 2. Figure 14 shows how the errors decrease during the manipulation. It is worth mentioning that the number of the cable points is determined by the segment length ls. Experiments are carried out for different segment lengths ls. The performance was slightly the same when the ls = 0.1 to 0.2 of the total cable length L. A smaller number of VFPs (for ls > 0.2L) decreases the accuracy of the manipulation, while a higher number is redundant and increases the computational cost.
Fig. 12. Experiments 1: (a) initial shape; (b) intermediate stage of the manipulation; (c) final shape
Fig. 13. Experiments 2: (a) initial shape; (b) intermediate stage of the manipulation; (c) final shape
Table 2. Average and maximum errors (e and emax)
Experiment 1 (Fig. 12) Experiment 2 (Fig. 13)
eavg, mm 4 15
emaxi mm 8 24
Experiments show that the VFPs algorithm has a reliable performance in terms of accuracy and speed. It was able to detect and model the cable for different segment lengths even for cases where the cable was at the border of the image frame. However, it fails when the cable is partially out of the frame or occluded. The planning model was able to guide the cable towards the target shape. Even though, in some experiments, the model was not able to finalize the task
(a) (b)
Fig. 14. The average and maximum errors of (a) Experiment 1 and (b) Experiment 2
with high accuracy (e.g., Fig. 13), it shows a stable performance, where the robots did not move unpredictably or diverge from the desired shape. This limitation in fine fitting of the cable to the desired shape can be held by considering the cable dynamics, e.g., bending, in the manipulation planning model.
5. Conclusion
In this paper, we have introduced a new framework for the shape control problem of a cable on a 2D plane using two manipulators. A virtual feature points (VFPs) algorithm is developed to model the cable as a set of N points to be further tracked and manipulated. The algorithm detects the cable and generates the VFPs on the cable for different configurations and for different segment lengths. The order of VFPs is guaranteed since the algorithm starts sampling from the same point (starting point on the marked tool). The algorithm detects the cable robustly even though the cable was at the border of the frame. However, it fails when there is occlusion or the cable is partially out of the frame, which occurs rarely in the task considered (shape control). Another algorithm is proposed to generate waypoints of the cable VFPs towards the desired shape. This algorithm generates a set of intermediate profiles between the initial and final cable shapes. These profiles guarantee that the cable is not manipulated unpredictably and the cable length constraint is maintained. Moreover, we have proposed a new approximate model to compute the Jacobian matrix, which maps between the cable points and the robots' end effectors configuration, considering the order of the points with respect to each end effector and the diminishing rigidity property of DLOs. Real-life experiments are carried out to evaluate the proposed framework. The experiments showed an adequate performance of the system. For some cases, the robots did not fit the cable accurately to the target shape. After carrying out multiple experiments, the final accuracy of shaping the cable was related to the initial shape configuration and position relative to the desired one. The shapes considered in this work have no more than 2 inflectional points, since shapes with more inflectional points are more challenging due to their instability. In the future, these complex shapes will be considered as a new case study which will be carried out by decomposing the complex shape into primitive subshapes and deforming the cable in a sequential manner with partial fixation for each achieved primitive subshape. Additional potential future works are: the improvement of the VFPs algorithm to tackle the case where part of the cable is hidden; the manipulation planning model will be further enhanced to obtain higher accuracy and robust performance; and exploiting force sensors
to improve the manipulation performance. Moreover, analyzing the impact of the cable dynamic properties, e. g., bending and stretching, and their parameters relating to the manipulation task will be topics of interest in the future.
Conflict of interest
The authors declare that they have no conflict of interest.
References
[1] Zhu, J., Cherubini, A., Dune, C., Navarro-Alarcon, D., Alambeigi, F., Berenson, D., Ficuciello, F., Harada, K., Kober, J., Li, X., Pan, J., Yuan, W., and Gienger, M., Challenges and Outlook in Robotic Manipulation of Deformable Objects, IEEE Robot. Autom. Lett, 2022, vol. 29, no. 3, pp. 67-77.
[2] Cirillo, P., Laudante, G., and Pirozzi, S., Vision-Based Robotic Solution for Wire Insertion with an Assigned Label Orientation, IEEE Access, 2021, vol. 9, pp. 102278-102289.
[3] Chang, P. and Padir, T., Model-Based Manipulation of Linear Flexible Objects: Task Automation in Simulation and Real World, Machines, 2020, vol. 8, no. 3, Art. 46, 22 pp.
[4] Zhu, J., Navarro, B., Passama, R., Fraisse, Ph., Crosnier, A., and Cherubini, A., Robotic Manipulation Planning for Shaping Deformable Linear Objects with Environmental Contacts, IEEE Robot. Autom. Lett, 2019, vol. 5, no. 1, pp. 16-23.
[5] White, J.R., Satterlee, P. E., Jr., Walker, K.L., and Harvey, H. W., Remotely Controlled and/or Powered Mobile Robot with Cable Management Arrangement, Patent US No. 4 736 826 (12 Apr 1988).
[6] Yin, H., Varava, A., and Kragic, D., Modeling, Learning, Perception, and Control Methods for Deformable Object Manipulation, Sci. Robot., 2021, vol. 6, no. 54, eabd8803, 16 pp.
[7] Sanchez, J., Mohy El Dine, K., Corrales, J. A., Bouzgarrou, B.-Ch., and Mezouar, Y., Blind Manipulation of Deformable Objects Based on Force Sensing and Finite Element Modeling, Front. Robot. AI, 2020, vol. 7, Art. 73, 10 pp.
[8] Delgado, A., Corrales, J. A., Mezouar, Y., Lequievre, L., Jara, C., and Torres, F., Tactile Control Based on Gaussian Images and Its Application in Bi-Manual Manipulation of Deformable Objects, Robot. Auton. Syst, 2017, vol. 94, pp. 148-161.
[9] Almaghout, K., Boby, R. A., Othman, M., Shaarawy, A., and Klimchik, A., Robotic Pick and Assembly Using Deep Learning and Hybrid Vision/Force Control, in Proc. of the 2021 Internat. Conf. "Nonlinearity, Information and Robotics" (NIR, Innopolis, Russia, Aug 2021), 6 pp.
[10] Sanchez, J., Corrales, J. A., Bouzgarrou, B. Ch., and Mezouar, Y., Robotic Manipulation and Sensing of Deformable Objects in Domestic and Industrial Applications: A Survey, Int. J. Rob. Res., 2018, vol. 37, no. 7, pp. 688-716.
[11] Tang, T., Wang, Ch., and Tomizuka, M., A Framework for Manipulating Deformable Linear Objects by Coherent Point Drift, IEEE Robot. Autom. Lett., 2018, vol. 3, no. 4, pp. 3426-3433.
[12] Li, X., Wang, Z., and Liu, Y.-H., Sequential Robotic Manipulation for Active Shape Control of Deformable Linear Objects, in Proc. of the 2019 IEEE International Conference on Real-Time Computing and Robotics (RCAR, Irkutsk, Russia, Aug 2019), pp. 840-845.
[13] Yang, Y., Stork, J. A., and Stoyanov, T., Learning to Propagate Interaction Effects for Modeling Deformable Linear Objects Dynamics, in Proc. of the 2021 IEEE Internat. Conf. on Robotics and Automation (ICRA, Xi'an, China, Oct 2021), pp. 1950-1957.
[14] Zhu, J., Navarro, B., Fraisse, Ph., Crosnier, A., and Cherubini, A., Dual-Arm Robotic Manipulation of Flexible Cables, in Proc. of the 2018 IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems (IROS, Madrid, Spain, Oct 2018), pp. 479-484.
[15] Zhu, J., Navarro-Alarcon, D., Passama, R., and Cherubini, A., Vision-Based Manipulation of De-formable and Rigid Objects Using Subspace Projections of 2D Contours, Robot. Auton. Syst, 2021, vol. 142, 103798, 35 pp.
[16] Heisler, P., Steinmetz, P., Yoo, I.S., and Franke, J., Automatization of the Cable-Routing-Process within the Automated Production of Wiring Systems, Appl. Mech. Mater., 2017, vol. 871, pp. 186-192.
[17] Herguedas, R., Lopez-Nicolas, G., Aragues, R., and Sagues, C., Survey on Multi-Robot Manipulation of Deformable Objects, in Proc. of the 24th IEEE Internat. Conf. on Emerging Technologies and Factory Automation (ETFA, Zaragoza, Spain, Sep 2019), pp. 977-984.
[18] Lv, N., Liu, J., and Jia, Y., Coordinated Control of Flexible Cables with Human-Like Dual Manipulators, J. Dyn. Syst. Meas. Control Trans. ASME, 2021, vol. 143, no. 8, 081006, 11 pp.
[19] Jin, Sh., Wang, Ch., and Tomizuka, M., Robust Deformation Model Approximation for Robotic Cable Manipulation, in Proc. of the 2019 IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems (IROS, Macau, China, Nov 2019), pp. 6586-6593.
[20] Lagneau, R., Krupa, A., and Marchal, M., Automatic Shape Control of Deformable Wires Based on Model-Free Visual Servoing, IEEE Robot. Autom. Lett., 2020, vol. 5, no. 4, pp. 5252-5259.
[21] Guo, Z. and Hall, R. W., Fast Fully Parallel Thinning Algorithms, CVGIP: Image Understanding, 1992, vol. 55, no. 3, pp. 317-328.
[22] Berenson, D., Manipulation of Deformable Objects without Modeling and Simulating Deformation, in Proc. of the 2013 IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems (Tokyo, Japan, Nov 2013), pp. 4525-4532.
[23] Bradski, G., The OpenCV Library, Dr. Dobb's Journal of Software Tools, 2000, vol. 25, pp. 120, 122-125.
[24] Zhang, Zh., A Flexible New Technique for Camera Calibration, IEEE Trans. Pattern Anal. Mach. Intell., 2000, vol. 22, no. 11, pp. 1330-1334.
[25] Heyden, A. and Pollefeys, M., Multiple View Geometry, in Emerging Topics in Computer Vision, G. Medioni, S.B.Kang (Eds.), Upper Saddle River, N.J.: Prentice Hall, 2004, pp. 45-108.
[26] Wang, Y. M., Li, Y., and Zheng, J. B., A Camera Calibration Technique Based on OpenCV, in Proc. of the 3rd Internat. Conf. on Information Sciences and Interaction Sciences (ICIS, Chengdu, China, Jun 2010), pp. 403-406.
[27] Robotic Operating System (ROS Melodic Morenia), https://www.ros.org (23 May 2018).
[28] Hennersperger, Ch., Fuerst, B., Virga, S., Zettinig, O., Frisch, B., Neff, Th., and Navab, N., Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility Study, IEEE Trans. Med. Imaging, 2017, vol. 36, no. 2, pp. 538-548.
