Safety in human-multi robot collaborative scenarios: a trajectory scaling approach

07/16/2021 ∙ by Martina Lippi, et al. ∙ University of Salerno 0

In this paper, a strategy to handle the human safety in a multi-robot scenario is devised. In the presented framework, it is foreseen that robots are in charge of performing any cooperative manipulation task which is parameterized by a proper task function. The devised architecture answers to the increasing demand of strict cooperation between humans and robots, since it equips a general multi-robot cell with the feature of making robots and human working together. The human safety is properly handled by defining a safety index which depends both on the relative position and velocity of the human operator and robots. Then, the multi-robot task trajectory is properly scaled in order to ensure that the human safety never falls below a given threshold which can be set in worst conditions according to a minimum allowed distance. Simulations results are presented in order to prove the effectiveness of the approach.



There are no comments yet.


page 6

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The close cooperation of humans and robots is a highly desirable feature since it allows to benefit of the outperforming reasoning capabilities of humans and the extreme precision and strength of robots. However, it is straightforward to recognize that the human safety is of the utmost importance in such a scenario which requires, at least, the robots to be controlled in such a way to not harm human operators Haddadin et al. (2009). In this regard, initial regulations about human safety with respect to industrial robots can be found in the American ANSI/RIA R15.06, in the European EN 775 or in the more general international standard ISO 10218 and the technical specification document ISO/TS 15066. In detail, the latter exactly focuses on human-robot collaborative scenarios and envisages four possible safe interactions:

  1. safety-rated monitored stop, i.e. robots are required to stop when humans enter the working area;

  2. hand guiding, i.e. robots are required to follow human manual guidance;

  3. speed and separation monitoring, i.e. robots have to keep a minimum safety distance from operators;

  4. power and force limiting, i.e. robots are required to mitigate human harm in the case of impact.

It is clear that interactions 3 and 4 involve integrating the robot autonomous task with human safety requirements. As highlighted in Robla-Gómez et al. (2017), this also requires the inclusion of different sensors whose features depend on the nature of the interaction: from sensors for detecting the presence of human operators for collision prevention, e.g. motion capture systems, range sensors or artificial vision systems as in Flacco et al. (2012), to sensors for assessing force exchange when an impact occurs, e.g. force or tactile sensors as in Cirillo et al. (2016). Although power and force limiting is crucial in the case of physical human-robot interaction where contact is unavoidable, distance monitoring would be more suitable for pure coexistence in the working area. In the latter scenario, it becomes relevant to quantify the level of human safety, looking at the overall structure of the manipulator as a source of danger to humans, so that the robots behaviour can be adapted accordingly. An index based on distance, velocity and inertial contributions is proposed in Kulić and Croft (2006) and is evaluated for the nearest point between each link and the operator; then, such danger index is exploited to generate a virtual repulsive force according to artificial potential field theory in Khatib (1985). The study presented in Lacevic et al. (2013) also focuses on defining an assessment of human safety which is now based on velocity and distance terms and is extended to the overall structure of the manipulator by a proper integration along each link; a gradient-based technique is then adopted to drive the manipulator. The previous approaches rely on pursuing evasive actions to increase safety, however, in industrial settings, it is generally recommended to follow the desired task path without deviating from it. This guideline is broadened in Zanchettin et al. (2016) where robot velocity is modulated in accordance with the distance from the operator while preserving the nominal task path. A further approach is presented in Liu and Tomizuka (2016) and Kimmel and Hirche (2017) where the safe interaction problem is modeled as an invariance control problem, i.e. it is based on defining a safe set of robot states for which no collision occurs and then on making this set invariant.
The previous works show how research regarding human-robot interaction and, in particular, human safety is a hot topic; however, to the best of authors’ knowledge, the case of interaction between human operators and strictly cooperative robot systems has not yet been investigated. In this context, human safety must certainly remain the highest priority task, but at the same time the coordination of the robot team must be managed.
Motivated by these considerations, this study presents a general solution for handling the human safety in a scenario composed by multiple cooperative robots. Starting from the definition of a safety index depending on the human operator’s state and on the state of a generic point of the robot structure, first the safety associated to the whole robot and, then, to whole team are computed. This safety measure is adopted to properly modify the robots trajectories in order to preserve the cooperative task and so as to not violate the safety requirements. However, because of the constraint represented by the task itself, this might result in a too restrictive strategy that might lead to the violation of the established safety requirements. If this case occurs, the task is interrupted and an impedance-based strategy is adopted; the task is then recovered when the safety conditions are restored.
The devised solution presents several desirable features with respect to other solutions cited above: (i) it works for general expressions of the safety index, (ii) it explicitly takes into account the multi-robot nature of the task, (iii) it does not modify the task path or require the task to be aborted unless if strictly necessary.
The paper is organized as follows. Section 2 introduces the mathematical background and the problem setting. In Section 3, the adopted safety index is defined and analyzed in detail, while in Section 4 this index is exploited to define a safe human-robot interaction strategy. Finally, numerical simulations and conclusions are presented in Sections 5 and 6, respectively.

2 Mathematical Background

In this paper, we consider a multi-robot work-cell in which human and robots are allowed to share the same area. In particular, the cell is composed by  worker robots which execute the main work the cell is aimed to.
We assume that robots are manipulators eventually mounted on a mobile base whose general model is


where (,

) is the joint position (velocity, acceleration) vector,

is the joint torque vector, is the symmetric positive definite inertia matrix, is the centrifugal and Coriolis terms matrix, is the matrix modeling viscous friction, is the vector of gravity terms, is the manipulator Jacobian matrix, and is the vector of interaction forces between the robot’s end-effector and the environment. Let (, ) be the joint position (velocity, acceleration) reference of the  th robot, the following assumption is made.

Assumption 1

Each robot is equipped with an inner motion control loop which guarantees tracking of a reference joint trajectory, i.e. (, ).

This assumption is realistic for all commercial platforms and makes the devised solution suitable also for off-the-shelf robotic platforms for which the low level control layer is generally not made accessible to directly set the input in (1).
The second order kinematic relationship is such as


where is the end-effector configuration of the  th manipulator with respect to the world frame expressed in terms of position and orientation , and is the input of the assumed virtual model. For the sake of notation compactness, the dependence of from its parameter is generally omitted in the following.
For the purpose of the overall description of the cell, let us introduce the collective vectors


where .
In what follows, with and we denote the null and identity matrices in , respectively, and with we denote the column vector in with all zero elements.

2.1 Problem setting

It is assumed that the cooperative task assigned to robots is defined by means of a task function as


being the task Jacobian matrix. A flexible formulation for the task function is given by the absolute-relative variables as in Basile et al. (2012). In detail, the absolute variables define the position and orientation of the centroid of the end-effector configurations:


with , while the relative variables represent the team formation:




Hence, in virtue of (5) and (6), the task function in (4) is


with and .

The objective is to compute the input in (2) in order to have tracking a nominal task trajectory , allowing human operators to enter the cell during execution. In such a scenario, the safety of the humans is the highest priority task and the robot trajectory must be modified accordingly. To the aim, the nominal trajectory is first properly modified in order to generate a human-safe trajectory which is the trajectory actually tracked as it will be detailed in the following.
Moreover, herein it is not of interest to design algorithms for human detection, while the focus is on defining a human-safe strategy for the coordination of cooperating robots. Hence, the following assumption is made.

Assumption 2

If human operators are in the nearby of the work-cell, either robots are able to detect them or this information is made available to robots. This information might concern, for instance, the position of the head or the chest of the human, or a set of representative points.

As stated above, the control input for the  th robot has to be such that, globally, the cooperative task described according to the task function in (8) tracks the reference . Hence, the  th virtual input in (2) can be selected resorting to a standard closed loop inverse kinematic law:


being the task tracking error, an arbitrary vector of joint accelerations such as which might be exploited to locally optimize secondary objectives, , positive gains and


a selection matrix. It is easy to recognize that in virtue of (4) and (2) it holds

where and which leads to the following exponentially stable linear second order dynamics

3 Human safety assessment

In this section, we focus on formulating an index to assess the level of human safety with respect to the team of robots. The devised safety strategy tries to have the robots follow the task trajectory as much as possible in compliance with human safety requirements. The basic idea is to parameterize the nominal trajectory for in (8) through a non-negative non-decreasing scalar function

with and the initial and final time instant, respectively, and to have the robots cooperatively track


where is a properly scaled version of which takes into account the human safety; obviously, this strategy allows the robots to preserve the task path.
Let us introduce a general safety index which allows to quantify the level of human safety with respect to a generic moving point belonging to the robot structure


where and are the position and velocity of point , respectively, is the distance between the point and the human operator’s position assumed to be available (see Assumption 2), is the distance derivative and , are generic scalar functions such as the following properties hold:

  1. [label=Property 0.,itemindent=*]

  2. is a non negative continuous monotonically increasing function with respect to ;

  3. is a continuous monotonically increasing function with respect to and such that:

    1. , with ;

    2. and .

The ratio behind Property is that the human-safety with respect the point increases with the distance . Concerning Property , function is such as the safety index increases for positive values of with a slope that might be modulated by . The motivation behind the asymptotic bound in Property (a) for is that it prevents the safety index to reach a too high value for high values of and arbitrarily small values of the distance ; in this way, the distance parameter is always the high priority feature. Finally, Property (b) ensures that for finite values of the index is sensitive to variation of velocity such as by changing the value of can be modified.

By leveraging the approach in Lacevic et al. (2013), the evaluation of the safety function in (12) can be easily extended to the entire structure of the  th manipulator by properly integrating (12) along its structure and obtain a cumulative safety index . In particular, the measure of human safety with respect to the  th link of the  th manipulator can be obtained by integrating along the volume of link


In order to make the computation of (13) affordable, the generic link of the  th robot is simplified as a segment starting at and ending at , thus (13) becomes


Finally, the safety index associated to the  th manipulator with links is


where an additional virtual link is introduced to account for the end-effector and the cooperative task to achieve.

Concerning the derivative of the safety measure in (15), the following lemma holds.

Lemma 1

The derivative of the cumulative safety function (15) associated to the  th robot is linear in the path parameter acceleration , i.e. it is


where the expressions of , are provided in the proof.

Proof. Let us consider the time derivative of the safety function in (12) associated with a generic point () belonging to the  th link of the  th robot; it is


with , whose second time derivative is


where coefficients , are defined as

At this point, let us consider the well-known relation between the linear acceleration of a point belonging to the structure of a manipulator and its joint variables, i.e.


being the positional Jacobian matrix associated to . Now, by partially deriving the reference task function in (11) with respect to , it holds

and in virtue of (9), equation (19) can be expressed as


where , are defined as

By folding (18) and (20) in (17), the term becomes


where the expressions of and are

Therefore, in virtue of (14),(15) and (21), it finally holds

with , defined as below

where the dependencies of and on their parameters are now made explicit for the sake of completeness.
This completes the proof.

In the multi-robot case, the overall safety function (and its derivative), which accounts for all the worker robots in the team, can be easily deduced by combining the safety functions in (15) associated to each manipulator


We are now ready to formally state the following problem.

Problem 1

Let us consider a multi-robot system composed by mobile manipulators performing the cooperative task defined as in (8) for which a desired trajectory parametrized with respect to a scalar function is assigned. Moreover, let us also assume that a minimum value for function in (22) is assigned; then, our objective is to properly scale so as to generate a new reference trajectory such that , .

Remark 1

Problem 1 requires that a minimum value for function is defined; thus, the problem arises on how to choose this lower bound. A first strategy consists in tuning via experimental trials based on the human feeling about the experienced level of safety resorting to techniques similar to Acharya et al. (2006). Another strategy consists in selecting such as, defined as the distance between the human operator and the team


ensures that for some value  (Lacevic et al. (2013)). As an example, the second strategy is pursued in the Section 5.

The next section provides a possible solution to Problem 1.

4 The human-robot avoidance strategy

An overview of the devised strategy for solving Problem 1 is provided in Figure 1; in particular, the proposed approach foresees tracking the nominal trajectory until the level of human safety is above the minimum accepted value, i.e. ; if the nominal trajectory leads the safety level to its minimum value, then a velocity modulation is applied while preserving the nominal path and, if this is not enough to guarantee

, then the requirement of preserving the path is relaxed. It is worth remarking that, in the case of redundant robots, each manipulator also exploits the extra degrees of freedom to maximize the cumulative safety index. b1[cc][][0.8]

Nominal trajectory tracking () b2[cc][][0.8] Scaled trajectory tracking b3[cc][][0.8] Path deformation l1[lc][][0.7] Trajectory scaling () l2[lc][][0.7] Recovery of the nominal trajectory l3[lc][][0.7] Constraint violation () l4[cc][][0.7] Zero deformation terms () l4[cc][][0.7] Recovery of the nominal trajectory

Figure 1: High-level scheme of the human avoidance strategy; transition conditions are detailed in Section 4.

4.1 Human-robot avoidance via trajectory scaling

By leveraging the approach in Dahl and Nielsen (1990) designed for torque-limited path following of industrial robots, a scaling parameter is introduced that is function of according to the following relation


where () might be either negative or positive and is adopted to properly scale the nominal path parameter while it is such as (i.e., ) in nominal conditions (no safety issue arises). Moreover, it is required that at any instant


The constraints (25) and (26) ensure that no reverse motion occurs along the path and that the end-point of the nominal trajectory is not overcome, respectively.
By folding (16) in (22), the expression of can be stated as follows


with , defined as

and which is linear in . At this point, we are ready to determine the scaling terms , and such that the minimum safety condition is met. Therefore, starting from the constraint , the lower () and the upper () bounds on the parameter are computed as




The ratio behind (28) and (29) is that, as long as , no constraint on (and then on and ) is set; while, in the case , the computed bounds are such as ensures that and, then, that does not fall below .
The derived bounds in (28) and (29) are used within the following dynamic system to compute :


where , and are positive constants and is any saturation function that bounds in the range . The first equation in (30) is such as to continuously bring to zero (i.e., to ), while, in the second equation, this value is saturated according to the computed bounds. Thus, when the saturation function does not alter the input value, the scaling term (, ) asymptotically converges to zero.

Remark 2

Constraints in (25) and (26) imply that the scaling strategy does not generally guarantee the condition to be fulfilled. For example, in the case and , it is evident that further scaling would violate the constraint in (25).

Because of Remark 2, a different avoidance strategy is presented in the following section which is adopted when the condition cannot be secured by the scaling strategy presented above and that, in brief, allows the path constraint to be violated (see Figure 1).

4.2 Human-robot avoidance via nominal path deformation

In the case the dynamics in (30) leads to one of the constraints (25) and (26) being violated, the cooperative task is aborted and other avoidance strategies need to be adopted. To this aim, two cases should be considered: loosely connected robots and tightly connected robots (as in a multi-robot transportation task of rigid objects). In the first case, once the task has been aborted, the human safety can be guaranteed independently by each robot adopting, for example, the approach devised in Lacevic et al. (2013). Therefore, this case is not investigated in this paper. In the more interesting case of tightly connected robots, the avoidance strategy needs to be compliant with the kinematic constraint that consists in having constant in any reference frame attached to the grasped object. For this reason, the avoidance strategy must consist in properly modifying and, as in the previous case, exploiting the local redundancy. In detail, let be the time instant in which the path constraint is relaxed, then the reference trajectory is modified as


where the displacement is computed according to the following dynamics