# Robust Cooperative Manipulation without Force/Torque Measurements: Control Design and Experiments

This paper presents two novel control methodologies for the cooperative manipulation of an object by N robotic agents. Firstly, we design an adaptive control protocol which employs quaternion feedback for the object orientation to avoid potential representation singularities. Secondly, we propose a control protocol that guarantees predefined transient and steady-state performance for the object trajectory. Both methodologies are decentralized, since the agents calculate their own signals without communicating with each other, as well as robust to external disturbances and model uncertainties. Moreover, we consider that the grasping points are rigid, and avoid the need for force/torque measurements. Load sharing coefficients are also introduced to account for potential differences in the agents' power capabilities. Finally, simulation and experimental results with two robotic arms verify the theoretical findings.

## Authors

• 5 publications
• 1 publication
• 13 publications
• ### Cooperative Manipulation via Internal Force Regulation: A Rigidity Theory Perspective

This paper considers the integration of rigid cooperative manipulation w...
11/04/2019 ∙ by Christos K. Verginis, et al. ∙ 0

• ### Decentralized Impedance Control for Cooperative Manipulation of Multiple Underwater Vehicle Manipulator Systems under Lean Communication

This paper addresses the problem of cooperative object transportation fo...
05/11/2019 ∙ by Shahab Heshmati-alamdari, et al. ∙ 0

• ### Four-Arm Manipulation via Feet Interfaces

We seek to augment human manipulation by enabling humans to control two ...
09/11/2019 ∙ by Jacob Hernandez Sanchez, et al. ∙ 0

• ### A Distributed Predictive Control Approach for Cooperative Manipulation of Multiple Underwater Vehicle Manipulator Systems

This paper addresses the problem of cooperative object transportation fo...
06/23/2019 ∙ by Shahab Heshmati-alamdari, et al. ∙ 0

• ### Communication-based Decentralized Cooperative Object Transportation Using Nonlinear Model Predictive Control

This paper addresses the problem of cooperative transportation of an obj...
03/21/2018 ∙ by Christos K. Verginis, et al. ∙ 0

• ### Decentralized Adaptive Control for Collaborative Manipulation of Rigid Bodies

In this work, we consider a group of robots working together to manipula...
05/06/2020 ∙ by Preston Culbertson, et al. ∙ 0

• ### Design, Fabrication and Control of an Hydraulic Elastomer Actuator

This paper presents design, fabrication and control of a compliant 2D ma...
06/13/2018 ∙ by Mahdi Momeni Kelageri, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## I Introduction

Multi-agent systems have gained significant attention the last years due to the numerous advantages they yield with respect to single-agent setups. In the case of robotic manipulation, heavy payloads and challenging maneuvers necessitate the employment of multiple robotic agents. Although collaborative manipulation of a single object, both in terms of transportation (regulation) and trajectory tracking, has been considered in the research community the last decades, there still exist several challenges that need to be taken into account by on-going research, both in control design as well as experimental evaluation.

Early works develop control architectures where the robotic agents communicate and share information with each other, and completely decentralized schemes, where each agent uses only local information or observers, avoiding potential communication delays (see, indicatively, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]). Impedance and hybrid force/position control is the most common methodology used in the related literature [11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 8, 9, 23, 10, 24], where a desired impedance behavior is imposed potentially with force regulation. Most of the aforementioned works employ force/torque sensors to acquire feedback of the object-robots contact forces/torques, which however may result in performance decline due to sensor noise or mounting difficulties. When the grasping object-agents contacts are rigid, the need for such sensors is redundant, since the overall system can be seen as a closed-chain robot. Regarding grasp rigidity, recent technological advances allow end-effectors to grasp rigidly certain objects, motivating the specific analysis.

In addition, many works in the related literature consider known dynamic parameters regarding the object and the robotic agents. However, the accurate knowledge of such parameters, such as masses or moments of inertia, can be a challenging issue, especially for complex robotic manipulators.

Force/torque sensor-free methodologies have been developed in [16, 8, 6, 4, 25, 22, 26, 19, 21]; [16] develops a leader-follower communication-based scheme by partly accounting for dynamic parametric uncertainty, whereas [8] and [4] employ partial and full model information, respectively; [6] develops an adaptive control scheme that achieves boundedness of the errors based on known disturbance bounds, and [25]

proposes an adaptive estimator for kinematic uncertainties, whose convergence affects the asymptotic stability of the overall scheme. In

[21] and [22] adaptive fuzzy estimators for structural and parametric uncertainty are introduced, with the latter not taking into account the object dynamics; [26] develops an adaptive protocol that guarantees boundedness of the internal forces, and [19] employs an approximate force estimator for a human-robot cooperative task.

Another important feature is the representation of the agent and object orientation. The most commonly used tools for orientation representation are rotation matrices, Euler angles, unit quaternions, and the angle/axis convention. In this work, we employ unit quaternions, which do not suffer from representation singularities and can be tuned to avoid undesired local equilibria, issues that characterize the other methods.

Unit quaternions in the control design of cooperative manipulation tasks have been employed in [11], where the authors address the gravity-compensated pose regulation of the grasped object, as well as in [12], where a model-based force-feedback scheme is developed.

Full model information is employed in the works [13, 10, 7, 1, 23, 17, 9, 15]; [7] employs a velocity estimator, [23] uses a linearized model, and [15, 14] considers kinematic and grasping uncertainties. Adaptive control schemes are developed in [20], where redundancy is used for obstacle avoidance and [27], where the object dynamics are not taken into account; [28] and [29] propose protocols based on graph-based communication by neglecting parts of the overall system dynamics, and [18], [29] consider leader-follower approaches. An observer-based (for state and task estimation) adaptive control scheme is proposed in [24]. Model-based force-control control protocols with unilateral constraints are developed in [30, 31]. Formation control approaches are considered in [32, 31] and a navigation-function scheme is used in [33]; [34] includes hybrid control with intermittent contacts and in our previous works [35, 36] we considered MPC approaches for cooperative object transportation. Finally, internal force and load distribution analysis in cooperative manipulation tasks is performed in a variety of works (e.g., [37, 38, 39, 40, 41]).

Note that most of the aforementioned adaptive control schemes (except e.g., [21]) employ the usual regressor matrix technique to compensate for unknown dynamic parameters [42, 43], which assumes a known structure of the dynamic terms. Such structures can still be difficult to obtain accurately, especially when complex manipulators are considered. Moreover, in terms of load distribution, many of the related works use load sharing coefficients (e.g., [4, 5, 21]), without proving that undesired internal forces do not arise, or the standard Moore-Penrose inverse of the grasp matrix (e.g. [6, 17]), which has been questioned in [37].

### I-a Contribution and Outline

In this paper we propose two novel nonlinear control protocols for the trajectory tracking of an object that is rigidly grasped by robotic agents, without using force/torque measurements at the grasping points. More specifically, our contribution lies in the following attributes:

1. Firstly, we develop a decentralized control scheme that combines

• adaptive control ideas to compensate for external disturbances and uncertainties of the agents’ and the object’s dynamic parameters,

• quaternion modeling of the object’s orientation that avoids undesired representation singularities.

2. Secondly, we propose a decentralized control scheme that does not depend on the dynamic structure or parameters of the overall system and guarantees predefined transient and steady-state performance for the object’s center of mass, using the Prescribed Performance Control (PPC) scheme [44].

3. We carry out extensive simulation studies and experimental results that verify the theoretical findings.

Moreover, both control schemes employ the load distribution proposed in [40] that provably avoids undesired internal forces.

The first control scheme is an extension of our preliminary work [45], where we designed a similar adaptive quaternion-based controller, guaranteeing, however, only local stability, and no experimental validation was provided. Furthermore, we have employed the PPC scheme in our previous work [46] to design timed transition systems for a cooperatively manipulated object. In this work, however, we perform a more extended and detailed analysis by deriving specific bounds for the inputs of the robotic arms (i.e., joint velocities and torques), as well as real-time experiments. It is worth noting that PPC has been also used for single manipulation tasks in [47, 48, 49].

The rest of the paper is organized as follows. Section II provides the notation used throughout the paper and necessary background. The modeling of the system as well as the problem formulation are given in Section III. Section IV presents the details of the two proposed control schemes with the corresponding stability analysis, and Section V illustrates the simulation and experimental results. Finally, Section VI concludes the paper.

## Ii Notation and Preliminaries

### Ii-a Notation

The set of positive integers is denoted by and the real -coordinate space, with , by ; and are the sets of real

-vectors with all elements nonnegative and positive, respectively. The

identity matrix is denoted by , the -dimensional zero vector by and the matrix with zero entries by . Given a matrix , we use , where

is the maximum eigenvalue of a matrix. The vector connecting the origins of coordinate frames

and } expressed in frame coordinates in -D space is denoted as . Given ,

is the skew-symmetric matrix defined according to

. The rotation matrix from to is denoted as , where is the -D rotation group. The angular velocity of frame with respect to is denoted as and it holds that [43] . We further denote as the Euler angles representing the orientation of with respect to , with . We also define the set . In addition, denotes the -dimensional sphere. For notational brevity, when a coordinate frame corresponds to an inertial frame of reference , we will omit its explicit notation (e.g., etc.). Finally, all vector and matrix differentiations are expressed with respect to an inertial frame , unless otherwise stated.

### Ii-B Unit Quaternions

Given two frames and , we define a unit quaternion describing the orientation of with respect to , with , subject to the constraint . The relation between and the corresponding rotation matrix as well as the axis/angle representation can be found in [43]. For a given quaternion , its conjugate, that corresponds to the orientation of with respect to , is [43] . Moreover, given two quaternions , the quaternion product is defined as [43]

 ζ1⊗ζ2\coloneqq[φ1φ2−ϵ⊤1ϵ2φ1ϵ2+φ2ϵ1+S(ϵ1)ϵ2]∈S3, (1)

where .

For a moving frame (with respect to ), the time derivative of the quaternion is given by [43]:

 ˙ζB/A=12E(ζB/A)ωAB/A, (2a) where E:S3→R4×3 is defined as: E(ζ)\coloneqq[−ϵ⊤φI3−S(ϵ)],∀ζ=[φ,ϵ⊤]⊤∈S3. Finally, it can be shown that E(ζ)⊤E(ζ)=I3,∀ζ∈S3 and hence (2a) implies ωAB/A=2E(ζB/A)⊤˙ζB/A. (2b)

### Ii-C Prescribed Performance

Prescribed performance control, recently proposed in [44], describes the behavior where a tracking error evolves strictly within a predefined region that is bounded by certain functions of time, achieving prescribed transient and steady state performance. The mathematical expression of prescribed performance is given by the inequalities , where are smooth and bounded decaying functions of time satisfying and , called performance functions. Specifically, for the exponential performance functions , with , appropriately chosen constants, the terms are selected such that and the terms represent the maximum allowable size of the tracking error at steady state, which may be set arbitrarily small to a value reflecting the resolution of the measurement device, thus achieving practical convergence of to zero. Moreover, the decreasing rate of , which is affected by the constants in this case, introduces a lower bound on the required speed of convergence of . Therefore, the appropriate selection of the performance functions imposes performance characteristics on the tracking error .

### Ii-D Dynamical Systems

Consider the initial value problem:

 ˙σ=H(σ,t),σ(0)∈Ω, (3)

with where is a non-empty open set.

###### Definition 1.

[50] A solution of the initial value problem (3) is maximal if it has no proper right extension that is also a solution of (3).

###### Theorem 1.

[50] Consider problem (3). Assume that is: a) locally Lipschitz on for almost all , b) piecewise continuous on for each fixed and c) locally integrable on for each fixed . Then, there exists a maximal solution of (3) on with such that .

###### Proposition 1.

[50] Assume that the hypotheses of Theorem 1 hold. For a maximal solution on the time interval with and for any compact set there exists a time instant such that .

## Iii Problem Formulation

Consider fully actuated robotic agents (e.g. robotic arms mounted on omnidirectional mobile bases) rigidly grasping an object (see Fig. 1). We denote by , the end-effector and object’s center of mass frames, respectively; corresponds to an inertial frame of reference, as mentioned in Section II-A. The rigidity assumption implies that the agents can exert both forces and torques along all directions to the object. In the following, we present the modeling of the coupled kinematics and dynamics of the object and the agents, which follows closely the one in [6, 4].

### Iii-a Robotic Agents

We denote by , with , the generalized joint-space variables and their time derivatives of agent , with . Here,

consists of the degrees of freedom of the robotic arm as well as the moving base. The overall joint configuration is then

, with . In addition, the inertial position and Euler-angle orientation of the th end-effector, denoted by and , respectively, can be derived by the forward kinematics and are smooth functions of , i.e. , . The generalized velocity of each agent’s end-effector , can be computed through the differential kinematics [43], where is a smooth function representing the geometric Jacobian matrix, [43]. We define also the set which contains all the singularity-free configurations. The differential equation describing the dynamics of each agent in task-space coordinates is [43]:

 Mi(qi)˙vi+Ci(qi,˙qi)vi+gi(qi)+di(qi,˙qi,t)=ui−fi, (4)

where is the positive definite inertia matrix, is the Coriolis matrix, is the gravity term, is a vector representing unmodeled friction, uncertainties and external disturbances, is the vector of generalized forces that agent exerts on the grasping point with the object and is the task space wrench, that acts as the control input; is related to the input torques, denoted by , via , where is a generalized inverse of [43]. Moreover, concerns redundant agents () and does not contribute to end-effector forces. The agent task-space dynamics (4) can be written in vector form as:

 M(q)˙v+C(q,˙q)v+g(q)+d(q,˙q,t)=u−f, (5)

where , , , , , , .

### Iii-B Object

Regarding the object, we denote by , the pose and generalized velocity of its center of mass, with . We consider the following second order dynamics, which can be derived based on the Newton-Euler formulation:

 ˙xO=JO(ηO)vO, (6a) MO(xO)˙vO+CO(xO,˙xO)vO+gO(xO)+dO(xO,˙xO,t)=fO, (6b)

where is the positive definite inertia matrix, is the Coriolis matrix, is the gravity vector, a vector representing modeling uncertainties and external disturbances, and is the vector of generalized forces acting on the object’s center of mass. Moreover, is the well-known object representation Jacobian and is not well-defined when , which is referred to as representation singularity. A way to avoid the aforementioned singularity is to transform the Euler angles to a unit quaternion for the orientation. Hence, can be transformed to the unit quaternion [43], for which, following Section II-B and (2), one obtains and , Moreover, it can be proved that

 ∥JO(ηO)∥=√|sin(θO)|+11−sin2(θO), (7a) ∥JO(ηO)−1∥=√1+sin(θO)≤√2, (7b)

where is the matrix inverse. which constitutes a singularity-free representation.

### Iii-C Coupled Dynamics

In view of Fig. 1, one concludes that the pose of the agents and the object’s center of mass are related as

 pEi(qi) =pO+pEi/O(qi)=pO+REi(qi)pEiEi/O, (8a) ηEi(qi) =ηO+ηEi/O, (8b)

, where and are the constant distance and orientation offset vectors between and . Following (8), along with the fact that, due to the grasping rigidity, it holds that , one obtains

 vi=JOi(qi)vO, (9)

where is the object-to-agent Jacobian matrix [45] for which it can be further proved that

 ∥JOi(x)∥≤∥∥pEiO/Ei∥∥+1,∀x∈Rni,i∈N. (10)

The kineto-statics duality along with the grasp rigidity suggest that the force acting on the object’s center of mass and the generalized forces , exerted by the agents at the grasping points, are related through , where , with , is the full column-rank grasp matrix. By using the latter along with (5), (6), (9) and its derivative, we obtain the overall system coupled dynamics:

 ˜M(x)˙vO+˜C(x)vO+˜g(x)+˜d(x,t)=[G(q)]⊤u, (11)

where , , , , is the overall state , , and we have omitted the arguments for notational brevity. Moreover, the following Lemma, whose proof can be found in [45], is necessary for the following analysis.

###### Lemma 1.

The matrix is symmetric and positive definite and the matrix is skew symmetric.

The positive definiteness of implies , , where and are positive unknown constants.

We are now ready to state the problem treated in this paper:

###### Problem 1.

Given a desired bounded object smooth pose trajectory specified by , , with bounded first and second derivatives, and , as well as , determine a decentralized control law in (11) such that one of the following holds:
1)
2) , , for positive .

Part in the aforementioned problem statement corresponds to the asymptotic stability that will be guaranteed by the control scheme of Section IV-A, and part is associated with the predefined transient and steady state performance that will be guaranteed in Section IV-B. The requirement is a necessary condition needed to ensure that tracking of will not result in singular configurations of , which is needed for the control protocol of Section IV-B. The constant can be taken arbitrarily close to .

To solve the aforementioned problem, we need the following assumptions regarding the agent feedback, the bounds of the uncertainties/disturbances, and the kinematic singularities.

###### Assumption 1 (Feedback).

Each agent has continuous feedback of its own state .

###### Assumption 2 (Object Geometry).

Each agent knows the constant offsets and .

###### Assumption 3 (Kinematic Singularities).

The agents operate away from kinematic singularities, i.e., evolves in a closed subset of , .

Assumption 1 is realistic for real manipulation systems, since on-board sensor can provide accurately the measurements . The object geometric characteristics in Assumption 2 can be obtained by on-board sensors, whose inaccuracies are not modeled in this work. Finally, Assumption 3 states that the that achieve are sufficiently far from singular configurations. Since each agent has feedback from its state , it can compute through the forward and differential kinematics the end-effector pose and the velocity , . Moreover, since it knows and , it can compute and , by inverting (8) and (9), respectively. Consequently, each agent can then compute the quaternion signals and .

Note that, due to Assumption 2 and the grasp rigidity, the object-agents configuration is similar to a single closed-chain robot. The considered multi-agent setup, however, renders the problem more challenging, since the agents must calculate their own control signal in a decentralized manner, without communicating with each other. Moreover, each agent needs to compensate its own part of the (possibly uncertain/unknown) dynamics of the coupled dynamic equation (11), while respecting the rigidity kinematic constraints. Regarding Assumption 2, our future directions include its relaxation to uncertain/unknown object offsets for some agents, which would then not have exact feedback of the object’s pose. In that case, the team would need to cooperate in a leader-follower fashion for the compensation/estimation of the state by these agents.

## Iv Main Results

In this section we present two control schemes for the solution of Problem 1. The proposed controllers are decentralized, in the sense that the agents calculate their control signal on their own, without communicating with each other, as well as robust, since they do not take into account the dynamic properties of the agents or the object (mass/inertia moments) or the uncertainties/external disturbances modeled by the function in (11). The first control scheme is presented in Section IV-A, and is based on quaternion feedback and adaptation laws, while the second control scheme is given in Section IV-B and is inspired by the Prescribed Performance Control (PPC) Methodology introduced in [44].

### Iv-a Adaptive Control with Quaternion Feedback

The proposed controller of this section is based on the techniques of adaptive control, whose aim is the design of control systems that are robust to constant or slowly varying unknown parameters. For more details, we refer the reviewer to the related literature (e.g., [51] and the references therein).

Firstly, we need the following assumption regarding the model uncertainties/external disturbances.

###### Assumption 4 (Uncertainties/Disturbance parameterization).

There exist constant unknown vectors and known functions , such that , , , where and are continuous in and , respectively, and bounded in .

The aforementioned assumption is motivated by the use of Neural Networks for approximating unknown functions in compact sets

[51]. More specifically, any continuous function can be approximated on a known compact set by a Neural Network equipped with Radial Basis Functions (RBFs) and using unknown ideal constant connection weights that are stored in a matrix as ; represents the parametric uncertainty and represents the unknown nonparametric uncertainty, which is bounded as in . In our case, the functions , play the role of the known function and , and , represent the unknown constants and the number of layers of the Neural Network, respectively. Nevertheless, in view of Neural Network approximation, Assumption implies that the nonparametric uncertainty is zero and that and are known functions of time. These properties can be relaxed with non-zero bounded nonparametric uncertainties and unknown but bounded time-dependent disturbances, i.e. and , where , are bounded. In that case, instead of asymptotic convergence of the pose to the desired one, we can show convergence of the respective errors to a compact set around the origin. For more details on Neural Network approximation and adaptive control with illustrative examples, we refer the reader to [51, Ch. 12].

The desired Euler angle orientation vector is transformed first to the unit quaternion [43]. Then, we need to define the errors associated with the object pose and the desired pose trajectory. We first define the position error . Since unit quaternions do not form a vector space, they cannot be subtracted to form an orientation error; instead we should use the properties of the quaternion group algebra. Let be the unit quaternion describing the orientation error. Then, it holds that [43]

which, by using (1), becomes:

 eζ=[eφeϵ]\coloneqq[φOφd+ϵ⊤OϵdφOϵd−φdϵO+S(ϵO)ϵd]. (12)

By employing (2) and certain properties of skew-symmetric matrices [52], the dynamics of can be shown to be:

 ˙ep= ˙pO−˙pd (13a) ˙eφ= 12e⊤ϵeω (13b) ˙eϵ= −12[eφI3+S(eϵ)]eω−S(eϵ)ωd, (13c)

where is the angular velocity error, with , as indicated by (2b).

Due to the ambiguity of unit quaternions, when , then . If , then , which, however, represents the same orientation. Therefore, the control objective is equivalent to

 limt→∞[ep(t)⊤,|eφ(t)|,eϵ(t)⊤]⊤=[0⊤3,1,0⊤3]⊤.

The left hand side of (4), after employing (9) and its derivative, becomes

 Mi(qi)˙vi+Ci(qi,˙qi)vi+gi(qi)+di(qi,˙qi,t)= Mi(qi)(JOi(qi)˙vO+˙JOi(qi)vO)+Ci(qi,˙qi)JOi(qi)vO+ gi(qi)+di(qi,˙qi,t).

which, according to Assumption 4 and the fact that the manipulator dynamics can be linearly parameterized with respect to dynamic parameters [42], becomes

 Mi(qi)JOi(qi)˙vO+(Mi(qi)˙JOi(qi)+Ci(qi,˙qi)JOi(qi))vO +gi(qi)+di(qi,˙qi,t)=Yi(qi,˙qi,vO,˙vO)ϑi+δi(qi,˙qi,t)¯di, (14a) ∀i∈N, where ϑi∈Rℓ,ℓ∈N, are vectors of unknown but constant dynamic parameters of the agents, appearing in the terms Mi,Ci,gi, and Yi:S×Rni+12→R6×ℓ are known regressor matrices, independent of ϑi,i∈N. Without loss of generality, we assume here that ℓ is the same for all agents. Similarly, the dynamical terms of the left hand side of (6b) can be written as MO(xO)˙vO+CO(xO,˙xO)vO+gO(xO)+dO(xO,˙xO,t) =YO(xO,˙xO,vO,˙vO)ϑO+δO(xO,˙xO,t)¯dO, (14b)

where is a vector of unknown but constant dynamic parameters of the object, appearing in the terms , and is a known regressor matrix, independent of . It is worth noting that the choice for and is not unique. In view of (14), the left-hand side of (11) can be written as:

 ˜M(x)˙vO+˜C(x)vO+˜g(x)+˜d(x,t)=YO(xO,˙xO,vO,˙vO)ϑO +δO(xO,˙xO,t)¯dO+[G(q)]⊤(˜Y(q,˙q,vO,˙vO)ϑ+˜δ(q,˙q,t)¯d), (15)

where ,