An overview of collaborative robotic manipulation in multi-robot systems

doi:10.1016/j.arcontrol.2020.02.002

Annual Reviews in Control

Volume 49, 2020, Pages 113-127

https://doi.org/10.1016/j.arcontrol.2020.02.002 Get rights and content

Abstract

Robotic manipulation aims at combining the versatility and flexibility of mobile robot platforms with manipulation capabilities of robot manipulators. This survey paper comprehensively reviews the state-of-the-art development of collaborative robotic manipulation from the perspective of modelling, control and optimization. Then, the recent results in this field can be categorized into coordination of multiple fixed manipulators, mobile robots and mobile manipulators, respectively. A classification and comparison of various issues and promising approaches is given. Finally, a short discussion section is given to summarize existing research and to point out several future research directions.

Introduction

Robotic systems are going through tremendous changes and increasingly bringing significant socioeconomic impacts to human lives (Valavanis & Saridis, 2012). For example, industrial robots (especially robot manipulators) have been widely and successfully used in conventional production processes due to their high endurance, speed, and precision in structured industrial environments (Hu, Tay, & Wen, 2012). With the developments in robotic/network technologies and the requirements on efficiency and quality in production processes, collaborative manipulation of robot manipulators (either fixed or mobile) has attracted increasingly attention due to practical potential in various applications and theoretical challenges (Hvilshoj, Bogh, Nielsen, & Madsen, 2012). Take manufacturing and automotive applications for example, the assembling, transporting, painting, packaging, and welding usually require high flexibility and maneuverability in practice. Notice that most of these tasks involve large maneuverability and manipulability, and cannot be carried out by a single robot. Hence, the cooperation of multiple robot manipulators arises as an important technique for increasing efficiency and enhancing flexibility in industrial applications. In addition to the high task complexity that is impossible for a single robot to accomplish, some other motivations for developing the multi-robot systems are below (Fink, Ribeiro, & Kumar, 2012a): 1) distributed property, i.e., the task is inherently distributed; 2) more time/energy efficiency, i.e., multiple robots can complete missions faster with low operational costs; 3) less system requirements, i.e., building some resource-constrained robots is much easier than developing a single powerful robot; 4) strong robustness and adaptivity, i.e., the robots can increase robustness against robot failures and self-adapt to changes; and 5) high scalability and flexibility, i.e., large-scale robot systems can function well in the presence of many robots.

Multi-robot systems usually refer to a group of fixed/mobile robots with sensing, computation, communication, and actuation capabilities that enable them to accomplish certain tasks cooperatively in a distributed fashion.¹ Applications for these systems, as depicted in Fig. 1, include assembly and painting of fixed manipulators, transportation and formation of mobile robots, and manipulation and 3D printing of mobile manipulators. Coordination of these multiple robots needs systematic methodologies to design and analyze cooperative strategies to control multi-robot systems. To further illustrate the mechanism behind the whole system, Fig. 2 shows an integrated multi-robot coordination framework. The robot physical space consists of a large number of robots, while the communication space describes the communication network among the robots. With the help of sensor and control networks, a decision maker in the distributed monitoring and control center can employ control algorithms and techniques to make decisions, and then influence the physical robots.

Due to the vast amount of literature, it would be challenging to exhaustively review existing results on collaborative robotic manipulation. Rather than the exhaustive review, this survey focuses on the characterization of the multi-robot coordination in terms of the robot functionalities and control challenges as illustrated in Fig. 3. We further categorize the existing works into coordination of the fixed manipulators, mobile robots, and mobile manipulators, according to the robot functionalities and controlled variables as follows.

•
Manipulator coordination: a team of fixed robot manipulators cooperatively complete certain tasks (e.g., consensus, synchronization) either in the configuration space or task space by using local interaction rules. Coordination only considers the control of robotic manipulators.
•
Mobile robot coordination: a team of mobile robots cooperatively complete certain tasks (e.g., rendezvous, formation, swarming, flocking, coverage control, surveillance, mapping) based on local interaction rules. Coordination needs to consider sensing and communication abilities.
•
Mobile manipulator coordination: mobile robot manipulators (manipulators mounted on a mobile robot platform) cooperatively complete certain tasks by using local interaction rules. Coordination is required to consider robotic manipulation as well as locomotion in the control.

Fig. 4 describes a block diagram to facilitate the understanding of this classification, representing several elementary fixed manipulators, mobile robots, and combination of them.

In response to this classification, this paper reviews various distributed control and coordination technologies that enable the robots to achieve team-level and global tasks efficiently by using local interaction rules. Specifically, we firstly formulate a general distributed coordination problem and further discuss distinctions of robots’ collective behaviors in terms of robot tasks and control objectives. We then review recent results on coordination of fixed manipulators, mobile robots, and mobile manipulators, respectively, focusing on robot modeling, control, design, and optimization. A classification and comparison of various issues and promising approaches is carried out.

The organization of this paper is given as follows. In Section 2, preliminaries on graph theory and communication network description are firstly presented. Then, the general distributed coordination problem with respect to various collective behaviors is provided. From 3 Coordination of robot manipulators, 4 Coordination of mobile robots, 5 Coordination of mobile manipulators, the existing results on coordination of fixed manipulators, mobile robots, and mobile manipulators are reviewed, respectively. Finally, we conclude the paper with a short discussion on future research directions.

Section snippets

Graph theory

Let $G$ = ${V, E}$ represent a graph and $V$ ∈ {1, 2, ⋅⋅⋅, N} denote the set of vertices. The set of edges is denoted as $E$ ⊆ $V \times V$ . Every robot is represented by a node in the graph. An edge is an ordered pair $(j, i) \in E$ if node j can be directly supplied with information from node i. We assume $(i, i) \notin E$ . $N_{i}$ = ${j \in V ∣ (j, i) \in E}$ denotes the neighborhood set of vertex i. An undirected graph is connected if there exists an undirected path between any two nodes. A digraph $G$ contains a directed spanning tree if

Kinematic and dynamic models of robotic manipulators

Consider a team of N robotic manipulators with the Euler-Lagrange (EL) dynamics described by Feng, Hu, Ren, Dixon, and Mei (2018) $M_{i} (q_{i}) {\ddot{q}}_{i} + C_{i} (q_{i}, {\dot{q}}_{i}) {\dot{q}}_{i} + G_{i} (q_{i}) + f_{i} ({\dot{q}}_{i}) = τ_{i} + d_{i},$ where $q_{i} \in R^{n}$ is the vector of generalized coordinates, M_i(q_i) ∈ $R^{n \times n}$ is the inertia matrix, $C_{i} (q_{i}, {\dot{q}}_{i}) {\dot{q}}_{i}$ ∈ $R^{n}$ is the vector of Coriolis and centrifugal torques, $G_{i} (q_{i}) \in R^{n}$ is the vector of gravitational torque, $f_{i} ({\dot{q}}_{i}) \in R^{n}$ represents the unknown unmodelled dynamics (or frictions), τ_i $\in R^{n}$ is the control input, and $d_{i} \in R^{n}$ is

Coordination of mobile robots

In Section 3, many different control strategies using only measurable information (e.g., the relative displacements) have been proposed such that synchronization/consensus of multiple fixed robotic manipulators can be achieved. In contrast, for the coordination of a group of mobile robots, only the relative distances of neighbors are employed to achieve the system-level coordination, e.g., rendezvous, formation, swarming/flocking, connectivity preservation, and obstacle avoidance.

One objective

Dynamic models of mobile manipulators

Consider a group of N mobile manipulators. When taking the constraints of mobile robot platforms into account, the EL dynamics can be described by: $M_{i} (q_{i}) {\ddot{q}}_{i} + C_{i} (q_{i}, {\dot{q}}_{i}) {\dot{q}}_{i} + G_{i} (q_{i}) + f_{i} ({\dot{q}}_{i}) + d_{i} = τ_{i} + J_{i}^{T} h_{i},$ $M_{i} (q_{i}) = [\begin{matrix} M_{b i} M_{b m i} \\ M_{m b i} M_{m i} \end{matrix}], C_{i} (q_{i}, {\dot{q}}_{i}) = [\begin{matrix} C_{b i} C_{b m i} \\ C_{m b i} C_{m i} \end{matrix}],$ $G_{i} (q_{i}) = [\begin{matrix} G_{b i} \\ G_{m i} \end{matrix}], f_{i} ({\dot{q}}_{i}) = [\begin{matrix} f_{b i} \\ f_{m i} \end{matrix}], d_{i} = [\begin{matrix} d_{b i} \\ d_{m i} \end{matrix}],$ $τ_{i} = [\begin{matrix} τ_{b i} \\ τ_{m i} \end{matrix}], J_{i} = [\begin{matrix} A_{i} 0 \\ J_{b i} J_{m i} \end{matrix}], h_{i} = [\begin{matrix} h_{b i} \\ h_{e i} \end{matrix}],$ where $q_{i} = {[q_{b i}, q_{m i}]}^{T} \in R^{n}$ denotes the vector of generalized coordinates with $q_{b i} \in R^{n_{b}}$ representing the generalized coordinates for the robot bases, and $q_{m i} \in R^{n_{m}}$

Conclusions and open research issues

As an emerging field of research in robotics, collaborative robotic manipulation has been recently attracting more and more attention. This paper presented a brief technical review on collaborative robotic manipulation. The classification on coordination of fixed robot manipulators, mobile robots, and mobile manipulators was presented. Many existing issues and a variety of approaches were outlined for collaborative robotic manipulation. Some open problems and current research activities were

References (93)

S. Chung et al.
Phase synchronization control of complex networks of lagrangian systems on adaptive digraphs
Automatica
(2013)
X. Dong et al.
Time-varying formation control for general linear multi-agent systems with switching directed topologies
Automatica
(2016)
J. Dong
Finite-time connectivity preservation rendezvous with disturbance rejection
Automatica
(2016)
Y. Dong et al.
A leader-following rendezvous problem of double integrator multi-agent systems
Automatica
(2013)
M. Hvilshoj et al.
Autonomous industrial mobile manipulation (AIMM): Past, present and future
Industrial Robot: An International Journal
(2012)
Z. Li et al.
Robust adaptive control of cooperating mobile manipulators with relative motion
IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
(2009)
A. Marino
Distributed adaptive control of networked cooperative mobile manipulator
IEEE Transactions on Control Systems Technology
(2018)
R.M. Murray
Recent research in cooperative control of multi-vehicle systems
ASME Journal on Dynamic Systems, Measurement, and Control
(2006)
E. Nuno et al.
Achieving consensus of Euler–Lagrange agents with interconnecting delays and without velocity measurements via passivity-based control
IEEE Transactions on Control Systems Technology
(2018)
C. Sun et al.
Robust finite-time connectivity preserving coordination of second-order multi-agent systems
Automatica
(2018)

Taylor, C. J. (2001). Multibots. University of Pennsylvania. Retrieved from...

G. Wen et al.

Flocking of multi-agent dynamical systems with intermittent nonlinear velocity measurements

International Journal of Robust and Nonlinear Control

(2012)

M.M. Zavlanos et al.

Graph-theoretic connectivity control of mobile robot networks

Proceedings of the IEEE

(2011)

M.M. Zavlanos et al.

Potential fields for maintaining connectivity of mobile networks

IEEE Transactions on Robotics

(2007)

A. Abdessameud et al.

Synchronization of lagrangian systems with irregular communication delays

IEEE Transactions on Automatic Control

(2014)

A. Abdessameud et al.

Leader-follower synchronization of euler-lagrange systems with time-varying leader trajectory and constrained discrete-time communication

IEEE Transactions on Automatic Control

(2016)

A. Ajorlou et al.

A class of bounded distributed control strategies for connectivity preservation in multiagent systems

IEEE Transactions on Automatic Control

(2010)

Asian, Scientist. (Oct. 15, 2018). Robots team up to 3D print in concrete. Retrieved from...

Ben, H. (Oct. 10, 2018). Automotive plastic painting by CMA robotics. Retrieved from...

H. Bai et al.

Cooperative load transport: A formation-control perspective

IEEE Transactions on Robotics

(2010)

H. Cai et al.

Leader-following consensus of multiple uncertain Euler–Lagrange systems under switching network topology

International Journal of General Systems

(2014)

H. Cai et al.

The leader-following consensus for multiple uncertain Euler–Lagrange systems with an adaptive distributed observer

IEEE Transactions on Automatic Control

(2016)

Y. Cao et al.

Finite-time connectivity-preserving consensus of networked nonlinear agents with unknown Lipschitz terms

IEEE Transactions on Automatic Control

(2016)

Y. Cao et al.

An overview of recent progress in the study of distributed multi-agent coordination

IEEE Transactions on Industrial Informatics

(2013)

F. Chen et al.

Distributed average tracking of networked Euler–Lagrange systems

IEEE Transactions on Automatic Control

(2015)

N. Chopra et al.

On synchronization of networked passive systems with time delays and application to bilateral teleoperation

IEEE/SICE int. conf. instrum., control inf. technol.

(2005)

S. Chung et al.

Cooperative robot control and concurrent synchronization of lagrangian systems

IEEE Transactions on Robotics

(2009)

G. Dai et al.

Distributed coordination and cooperation control for mobile manipulators

IEEE Transactions on Industrial Electronics

(2017)

DeBord, M. (Oct. 27, 2018). Tesla is going to have to spend an absolutely staggering amount of money in the next few...

D.V. Dimarogonas et al.

On the rendezvous problem for multiple agents

IEEE Transactions on Automatic Control

(2007)

X. Dong et al.

Time-varying formation tracking for linear multi-agent systems with multiple leaders

IEEE Transactions on Automatic Control

(2017)

Y. Dong et al.

Leader-following connectivity preservation rendezvous of multiple double integrator systems based on position measurement

IEEE Transactions on Automatic Control

(2014)

Y. Dong et al.

Leader-following consensus with connectivity preservation of uncertain euler-lagrange multi-agent systems

International Journal of Robust and Nonlinear Control

(2017)

Z. Feng et al.

Connectivity-preserving flocking for networked lagrange systems with time-varying actuator faults

Automatica

(2019)

Z. Feng et al.

Distributed coordination of multiple Euler–Lagrange systems

IEEE Transactions on Control of Network Systems

(2018)

Z. Feng et al.

Robust connectivity preserving rendezvous of multi-robot systems under unknown dynamics and disturbance

IEEE Transactions on Control of Network Systems

(2017)

J. Fink et al.

Robust control for mobility and wireless communication in cyber-physical systems with application to robot teams

Proceedings of the IEEE

(2012)

J. Fink et al.

Robust control for mobility and wireless communication in cyber-physical systems with application to robot teams

Proceedings of the IEEE

(2012)

A. Franchi et al.

Distributed estimation of states and parameters in multi-agent cooperative load manipulation

IEEE Trans. Control Netw. Syst.

(2019)

G.C. Frank et al.

Distributed adaptive tracking control for synchronization of unknown networked lagrangian systems

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

(2011)

Futch, M. (Oct 18, 2017). Rise of the warehouse robots. Retrieved from...

V. Gazi

Swarm aggregations using artificial potentials and sliding-mode control

IEEE Trans. Robotics

(2005)

V. Gazi et al.

Stability analysis of swarms

IEEE Transactions on Automatic Control

(2003)

S. Ghapani et al.

Fully distributed flocking with a moving leader for lagrange networks with parametric uncertainties

Automatica

(2016)

G. Hu et al.

Cloud robotics: Architecture, challenges and applications

IEEE Network

(2012)

A. Jadbabaie et al.

Coordination of groups of mobile agents

IEEE Transactions on Automatic Control

(2003)

Cited by (0)

^☆: The work is supported by the National Research Foundation of Singapore (M4061925.048), Delta-NTU Corporate Lab Program with the project reference of Delta-NTU CORP-SMA-RP8.

View full text

Review ArticleAn overview of collaborative robotic manipulation in multi-robot systems☆

Abstract

Introduction

Section snippets

Graph theory

Kinematic and dynamic models of robotic manipulators

Coordination of mobile robots

Dynamic models of mobile manipulators

Conclusions and open research issues

Automatica

Automatica

Automatica

Automatica

Industrial Robot: An International Journal

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

IEEE Transactions on Control Systems Technology

ASME Journal on Dynamic Systems, Measurement, and Control

IEEE Transactions on Control Systems Technology

Automatica

International Journal of Robust and Nonlinear Control

Proceedings of the IEEE

IEEE Transactions on Robotics

Synchronization of lagrangian systems with irregular communication delays

IEEE Transactions on Automatic Control

Leader-follower synchronization of euler-lagrange systems with time-varying leader trajectory and constrained discrete-time communication

IEEE Transactions on Automatic Control

A class of bounded distributed control strategies for connectivity preservation in multiagent systems

IEEE Transactions on Automatic Control

Cooperative load transport: A formation-control perspective

IEEE Transactions on Robotics

Leader-following consensus of multiple uncertain Euler–Lagrange systems under switching network topology

International Journal of General Systems

The leader-following consensus for multiple uncertain Euler–Lagrange systems with an adaptive distributed observer

IEEE Transactions on Automatic Control

Finite-time connectivity-preserving consensus of networked nonlinear agents with unknown Lipschitz terms

IEEE Transactions on Automatic Control

An overview of recent progress in the study of distributed multi-agent coordination

IEEE Transactions on Industrial Informatics

Distributed average tracking of networked Euler–Lagrange systems

IEEE Transactions on Automatic Control

On synchronization of networked passive systems with time delays and application to bilateral teleoperation

IEEE/SICE int. conf. instrum., control inf. technol.

Cooperative robot control and concurrent synchronization of lagrangian systems

IEEE Transactions on Robotics

Distributed coordination and cooperation control for mobile manipulators

IEEE Transactions on Industrial Electronics

On the rendezvous problem for multiple agents

IEEE Transactions on Automatic Control

Time-varying formation tracking for linear multi-agent systems with multiple leaders

IEEE Transactions on Automatic Control

Leader-following connectivity preservation rendezvous of multiple double integrator systems based on position measurement

IEEE Transactions on Automatic Control

Leader-following consensus with connectivity preservation of uncertain euler-lagrange multi-agent systems

International Journal of Robust and Nonlinear Control

Connectivity-preserving flocking for networked lagrange systems with time-varying actuator faults

Automatica

Distributed coordination of multiple Euler–Lagrange systems

IEEE Transactions on Control of Network Systems

Robust connectivity preserving rendezvous of multi-robot systems under unknown dynamics and disturbance

IEEE Transactions on Control of Network Systems

Robust control for mobility and wireless communication in cyber-physical systems with application to robot teams

Proceedings of the IEEE

Robust control for mobility and wireless communication in cyber-physical systems with application to robot teams

Proceedings of the IEEE

Distributed estimation of states and parameters in multi-agent cooperative load manipulation

IEEE Trans. Control Netw. Syst.

Distributed adaptive tracking control for synchronization of unknown networked lagrangian systems

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Swarm aggregations using artificial potentials and sliding-mode control

IEEE Trans. Robotics

Stability analysis of swarms

IEEE Transactions on Automatic Control

Fully distributed flocking with a moving leader for lagrange networks with parametric uncertainties

Automatica

Cloud robotics: Architecture, challenges and applications

IEEE Network

Coordination of groups of mobile agents

IEEE Transactions on Automatic Control

Review Article
An overview of collaborative robotic manipulation in multi-robot systems☆