Multi-DoF Time Domain Passivity Approach based Drift Compensation for Telemanipulation

09/04/2019 ∙ by Andre Coelho, et al. ∙ DLR 0

When, in addition to stability, position synchronization is also desired in bilateral teleoperation, Time Domain Passivity Approach (TDPA) alone might not be able to fulfill the desired objective. This is due to an undesired effect caused by admittance type passivity controllers, namely position drift. Previous works focused on developing TDPA-based drift compensation methods to solve this issue. It was shown that, in addition to reducing drift, one of the proposed methods was able to keep the force signals within their normal range, guaranteeing the safety of the task. However, no multi-DoF treatment of those approaches has been addressed. In that scope, this paper focuses on providing an extension of previous TDPA-based approaches to multi-DoF Cartesian-space teleoperation. An analysis of the convergence properties of the presented method is also provided. In addition, its applicability to multi-DoF devices is shown through hardware experiments and numerical simulation with round-trip time delays up to 700 ms.



There are no comments yet.


page 1

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Despite being able to autonomously fulfill a significant range of objectives, state-of-the-art robots still need human assistance to perform more complex or unforeseen tasks [18]. The level of human participation in robotic tasks can range from supervised autonomy [11] to direct teleoperation [2]. In the latter, an important characteristic of the telemanipulation setup is to be able to passively interact with the environment and the human operator. Among the passivity-based telemanipulation approaches [1, 13] developed to solve that issue, Time Domain Passivity Approach (TDPA, [9, 16]) presents the advantage of adapting the energy dissipation necessary to passivate the teleoperation channel based on measurements of the flow and effort variables acting on the system. This characteristic allows the implementation of a model-independent passivity observer and passivity controller (PO-PC) pair, which is robust to varying time delays and package loss in the communication channel. The adaptive characteristic of TDPA results in better performance compared to other passivity-enforcing controllers for teleoperation, e.g. wave-variable methods (see [6]).

Nevertheless, in spite of being able to render the communication channel passive, TDPA presents the drawback of creating position drift between master and slave devices whenever the PO-PC pair is applied in admittance configuration, which is necessary in many telemanipulation architectures ([2, 16, 5]). In order to tackle this issue, Artigas et al. [3] proposed a compensator that modifies the existing PO-PC in order to inject energy into the system to compensate for the existing drift. Later, Chawda et al. [7] adapted Artigas’ compensator in order keep the original TDPA formulation by using a virtual velocity injection source before the PO-PC. Despite being able to successfully compensate for the drift, these methods generate force spikes when the compensation action is allowed into the system after drift has been accumulated. In order to achieve position synchronization while keeping the forces within their normal range, a TDPA-based drift compensator was developed by the authors [8]

. In that paper, the previously existing compensation methods were adapted to produce smoother signals while removing the drift. The efficacy of TDPA-based compensation methods was experimentally shown through application to one-degree-of-freedom (1-DoF) devices or in a concatenated manner, treating each DoF as an independent system. Nevertheless, no multi-DoF application of those compensators has been tackled to this date.

Fig. 1: Suspended Aerial Manipulator from [17], whose model was used to validate the proposed approach.

In light of that, this paper aims at providing an extension of the previously presented drift compensators [3, 7, 8] to multi-DoF robotic systems. In addition, a convergence analysis is provided. It is shown that, if the gains are kept within a given range and if allowed by the passivity condition, the presented method is able to successfully reduce the accumulated drift caused by admittance type passivity controllers (PCs) in TDPA. Together with the previously presented single-DoF analyses, this paper contributes to demonstrating the effectiveness of the TDPA-based drift compensators and their applicability to different teleoperation setups. In this paper, in addition to hardware experiments with commercially available haptic devices, teleoperation of the dynamic model of a Suspended Aerial Manipulator [17] (see Fig. 1) is simulated.

Ii Dynamics of Kinematically Redundant Manipulators

The dynamic model of a robotic manipulator with joints can be written as


where is a set of generalized coordinates, is the inertia matrix,

is a vector of Coriolis and centrifugal forces, and

is the gravitational generalized torque vector. The generalized torque vector is the sum of control and external torques.

In case kinematically redundant robots are considered, where the number of Cartesian task coordinates is less than the number of joint generalized coordinates, i.e., , the set of task coordinates can be extended to


where is the set of nullspace velocities. is the Jacobian matrix that maps generalized to Cartesian coordinates. Under the assumption of full row rank of , the matrix can be constructed as [15]


where is a full row rank nullspace base matrix, such that .

Such formulation allows the manipulator dynamics to be written as


The choice of as in (3) generates a block diagonal matrix . By compensating the gravity torque and the cross-coupling terms of such that the nullspace task has no influence on the Cartesian one [15], the dynamics of the Cartesian-space task can be rewritten as


where is a Cartesian-space wrench.

The above described dynamic decoupling allows for the application of TDPA for Cartesian-space teleoperation of kinematically redundant manipulators without having to take the energy generated by the nullspace task into account.

Iii Time Domain Passivity Approach

Iii-a Overview

In contrast to methods where a damping element is designed for the worst case scenario ([1, 13]), TDPA consists in adding adaptive damping components in order to dissipate only the necessary amount of energy, computed using measurements of the forces and velocities being exchanged.

In TDPA, the communication channel is usually represented by one or more Time Delay Power Networks (TDPNs, [16]), which are two port networks that exchange velocities and forces. In addition to constant or variable time delays, TDPNs can also model package losses in the signals being transmitted. Fig. 2 shows the signal flow of the TDPN. and are the energies computed on the master and slave sides, respectively. The and subscripts are used to represent the direction of flow, namely into or out of the channel.

Fig. 2: Signal flow of the TDPN.

The pairs / and / from Fig. 2 are the flow-effort pairs on each side of the TDPN, such that


A sufficient condition for passivity of a TDPN network is that


where and are the observed left-to-right and right-to-left energy flows observed on the right and left-hand sides of the TDPN. and are the forward and backward delays, respectively.

One of the most common teleoperation schemes is the P-F architecture [10], where the master velocity is sent through the channel and serves as desired velocity to the slave. In turn, the force produced by the slave-side controller is sent back to the master. Following the framework presented by Artigas et al. [4], using a hybrid of circuit and network representation, the slave side of the P-F architecture can be represented as shown in Fig. 3. There, the communication channel is represented by a TDPN. and are the velocities of the master and slave devices. is the force exerted by the slave-side controller. is the delayed master velocity, and is the delayed slave control force applied to the master device. and are the admittance-type passivity controller and the drift compensation velocity source, which will be addressed subsequently.

Fig. 3: Slave side of a P-F architecture. The PO-PC () is applied in admittance configuration. is the drift compensator velocity.

Iii-B Passivity Observer

In order to take into account the energy removed by the passivity controllers up to the previous time steps ( and ), the energy flow on each side of the TDPN is computed as


Iii-C Passivity Controller

The passivity controller acts as an adaptive damping in order to guarantee the passivity of the channel. It can be applied in impedance or admittance configuration, according to the architecture requirements. In Fig. 3 the PC () is being applied in admittance configuration in order to modify the velocity coming out of the channel.

In order to fulfill the passivity conditions from (8) and (9) for the Cartesian-space task, two constructions for the passivity controller are possible, namely the concatenated version and the coupled one.

Iii-C1 Concatenated PO-PC

The concatenated version consists of adding a passivity observer to each degree of freedom and computing as a diagonal matrix, whose diagonal elements are given by


where is the sampling time.

Iii-C2 Coupled PO-PC

In addition to the concatenated version, the passivity controller can be applied in a coupled manner. For that purpose, the impedance PC presented by Ott et al. [14] can be adapted to the admittance case as




For both cases, the velocity removed by the passivity controller from the delayed master velocity in order to keep the system passive will be


and the resulting velocity used as a reference by the slave will be


assuming all velocities are represented in the same frame.

Iv Multi-DoF Drift Compensator

Iv-a Representation of Drift in SE(3)

Assuming the teleoperation task comprises the complete Cartesian space, the velocities and can be defined to be body velocities [12] in  as


where and are the frames defined by the delayed master orientation and the orientation given to the slave as the reference, respectively. Moreover, and are the translational and rotational body-velocity components, respectively, whose discrete-time integral in SE(3) can be defined as



represents the skew-symmetric operator applied to


Using the definitions above, the drift present in the system at a given time step () can be represented in SE(3) by


It can be noted from (17), (19), (20), and (21) that, if the PC acts at a time step, it will affect the value of for all future time steps. In case

is not the identity matrix, there will be a drift between the delayed master pose and the pose given as reference to the slave.

Iv-B Cartesian-Space Drift Compensation

In order to compensate for the drift caused by TDPA, an additional velocity signal can be added to the delayed master velocity before it is checked by the PO. In Fig. 3 the drift compensator is represented by a current source. It can be noted that, since is applied before the point where the energies are computed, the modified velocity will be checked and corrected for passivity. This guarantees that the compensation action will only be applied when so-called “passivity gaps” appear, i.e., when from (11) would be greater than zero. Therefore, the compensator would not compromise the stability of the system. From Fig. 3, it can been seen that, when the drift compensator is added, (17) becomes


where is defined as [12]


In order to reduce the drift between master and slave devices whenever allowed by the aforementioned passivity conditions, the following law can be used


where and are the translational and rotational gains of the compensator, and and are the angle and axis components taken from an angle-axis representation of , such that


Iv-C Convergence Analysis

As mentioned in Section IV-A

, in order to keep passivity, the proposed compensator is only able to reduce the drift when energy gaps are present. During the moments when the passivity controller is acting to reduce the delayed master velocity coming from the channel, the accumulation of drift is unavoidable. For that reason, this section aims to analyze the convergence characteristics of the compensator during the moments where it is allowed to act.

At the moments where the compensation action is allowed, (22) becomes


By defining a velocity error , (27) becomes


When defining recursively as


with and represented in the frame, it follows from the compensation law (24) that, using angle axis representation, the rotational part of (29) becomes


Likewise, the translational part becomes


It can be verified that, a sufficient condition for convergence is that both, the eigenvalues of

and , should be positive and less than 2.

It is also interesting to note that, if the gain matrices are chosen to be identity matrices, the drift converges within one time step. This can be seen as the multi-DoF extension of the compensators proposed in [3] and [7]. However, in case the force peaks described in [7] are undesirable, other values within the convergent range could be chosen. In that case, the proposed compensator can be considered as an extension of the one previously presented by the authors [8].

Fig. 4: Master and slave positions, no drift compensator – ms.
Fig. 5: Master and slave forces, no drift compensator – ms.
Fig. 4: Master and slave positions, no drift compensator – ms.

V Validation Results

V-a Overview

This section provides experimental results performed using two 3-DoF devices (Section V-B), as well as numerical simulation results of teleoperation of the model of the Suspended Aerial Manipulator from Fig. 1 (Section V-C). In order to show the efficacy of the proposed compensator when both concatenated and coupled passivity controller approaches are applied, Section V-B focuses on the implementation of the former while Section V-C shows results for the latter.

Fig. 6: Master-in and slave-out energies, no drift compensator – ms.

V-B Experimental Evaluation

In order to validate the proposed compensator on multi-DoF devices, telemanipulation experiments were performed using two 3-DoF translational haptic devices. Firstly, the concatenated PO-PC alone was applied to passivate the communication channel, set to artificially add 200 ms round-trip time delays (). Subsequently, the translational part of the proposed compensation law (see (24)) was applied.

Fig. 7: Master and slave positions, drift compensator on – ms.
Fig. 8: Master and slave forces, drift compensator on – ms.
Fig. 7: Master and slave positions, drift compensator on – ms.

Figs. 5 and 5 show position and control forces, respectively, of the master and slave devices for the non-compensated case. Fig. 6 shows the master-input and slave-output energies observed on the slave side. It can be seen that, in order to ensure passivity of the channel, not only the impedance-type PC intermittently reduced the force values (Fig. 5), but also the admittance-type PC removed part of the velocity coming from the master, generating significant drift (see Fig. 5).

Fig. 9: Master-in and slave-out energies, drift compensator on – ms.

Figs. 89 show position, force, and energy values, respectively, for the case when the proposed drift compensator was applied. It can be seen that the compensator was able to completely remove the drift in the y- and z-axes (Fig. 8). However, an offset can still be observed in the x-axis. This is due to the fact that not enough passivity gaps appeared in order to compensate for the drift in a passive way. The occurrence of passivity gaps depends on the system dynamics, the task being performed and the delay of the channel.

V-C Numerical Simulation

In order to show the applicability of the proposed orientation drift compensation law, the proposed compensator was applied to the orientation of the end-effector of a simulated Suspended Aerial Manipulator when round-trip communication delays of 700 ms were present. In the results presented in this section, the coupled PO-PC implementation was used (see Section III-C).

Although being a redundant manipulator, a decoupling control law was applied so that the nullspace dynamics would not affected the Cartesian-space task (see Section II).

Figs 1113 depict the orientation, Cartesian tool-frame torques, and slave-side energy signals computed when applying coupled TDPA without drift compensation. It is important to note that Roll-Pitch-Yaw angles were used in order to facilitate the understanding of the orientation plots. However, this representation is not used in the compensation law.

From Fig. 11, significant drift caused by the admittance-type PC can be observed in all axes. In addition, the overall tracking capability of the controller was also reduced by the passivity controller.

Fig. 10: Master and slave orientation, no drift compensator – ms.
Fig. 11: Master and slave Cartesian torques in the tool-frame, no drift compensator – ms.
Fig. 10: Master and slave orientation, no drift compensator – ms.
Fig. 12: Master-in and slave-out energies, no drift compensator – ms.
Fig. 13: Master and slave orientation, drift compensator on – ms.
Fig. 12: Master-in and slave-out energies, no drift compensator – ms.

When drift compensation was added (Figs. 1315), it can be noted that the drift converged to zero in the three axes (see Fig. 13). Adding to that, it can be seen that the tracking capabilities of the robot controller was preserved even in the presence of time delay. It is also important to remark that the control torques were not increased significantly when the compensator was applied (compare Fig. 11 with Fig. 15).

Fig. 14: Master and slave Cartesian torques in the tool-frame, drift compensator on – ms.
Fig. 15: Master-in and slave-out energies, drift compensator on – ms.
Fig. 14: Master and slave Cartesian torques in the tool-frame, drift compensator on – ms.

Vi Conclusion

This paper presented an extension of the previously proposed TDPA-based drift compensators to multi-DoF Cartesian-Space teleoperation. A convergence analysis has also been provided. It has been shown that, if the gain is set within certain bounds, the proposed approach is able to reduce the drift caused by the passivity controller in case it is able to do so without violating the passivity condition. That analysis also provided an insight about the cause of force spikes, which are generated when the drift is set to converge within one time step. In addition, hardware experiments and numerical simulation results demonstrated the applicability of the proposed compensator to time-delayed bilateral teleoperation of multi-DoF devices, when using both concatenated and coupled PO-PC implementations. Future work will involve applying teleoperation methods to the nullspace of redundant manipulators.


  • [1] R. J. Anderson and M. W. Spong (1989) Bilateral control of teleoperators with time delay. IEEE Transactions on Automatic control 34 (5), pp. 494–501. Cited by: §I, §III-A.
  • [2] J. Artigas, R. Balachandran, C. Riecke, M. Stelzer, B. Weber, J. Ryu, and A. Albu-Schaeffer (2016) Kontur-2: force-feedback teleoperation from the international space station. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 1166–1173. Cited by: §I, §I.
  • [3] J. Artigas, J. H. Ryu, and C. Preusche (2010) Position drift compensation in time domain passivity based teleoperation. In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4250–4256. Cited by: §I, §I, §IV-C.
  • [4] J. Artigas, J. Ryu, C. Preusche, and G. Hirzinger (2011) Network representation and passivity of delayed teleoperation systems. In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 177–183. Cited by: §III-A.
  • [5] J. Artigas, J. Ryu, and C. Preusche (2010) Time domain passivity control for position-position teleoperation architectures. Presence: Teleoperators and Virtual Environments 19 (5), pp. 482–497. Cited by: §I.
  • [6] R. Balachandran, J. Artigas, U. Mehmood, and J. Ryu (2016) Performance comparison of wave variable transformation and time domain passivity approaches for time-delayed teleoperation: preliminary results. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 410–417. Cited by: §I.
  • [7] V. Chawda, H. Van Quang, M. K. O’Malley, and J. Ryu (2014) Compensating position drift in time domain passivity approach based teleoperation. In 2014 IEEE Haptics Symposium, pp. 195–202. Cited by: §I, §I, §IV-C.
  • [8] A. Coelho, H. Singh, T. Muskardin, R. Balachandran, and K. Kondak (2018) Smoother position-drift compensation for time domain passivity approach based teleoperation. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5525–5532. Cited by: §I, §I, §IV-C.
  • [9] B. Hannaford and J. Ryu (2002) Time-domain passivity control of haptic interfaces. IEEE Transactions on Robotics and Automation 18 (1), pp. 1–10. Cited by: §I.
  • [10] D. A. Lawrence (1993) Stability and transparency in bilateral teleoperation. IEEE transactions on robotics and automation 9 (5), pp. 624–637. Cited by: §III-A.
  • [11] N. Y. Lii, D. Leidner, P. Birkenkampf, B. Pleintinger, R. Bayer, and T. Krueger (2017) Toward scalable intuitive teleoperation of robots for space deployment with the meteron supvis justin experiment. In Proceedings of the 14th Symposium on Advanced Space Technologies for Robotics and Automation (ASTRA), Cited by: §I.
  • [12] R. M. Murray (1994) A mathematical introduction to robotic manipulation. CRC press. Cited by: §IV-A, §IV-B.
  • [13] G. Niemeyer and J. Slotine (1991) Stable adaptive teleoperation. IEEE Journal of oceanic engineering 16 (1), pp. 152–162. Cited by: §I, §III-A.
  • [14] C. Ott, J. Artigas, and C. Preusche (2011) Subspace-oriented energy distribution for the time domain passivity approach. In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 665–671. Cited by: §III-C2.
  • [15] C. Ott, A. Dietrich, and A. Albu-Schäffer (2015) Prioritized multi-task compliance control of redundant manipulators. Automatica 53, pp. 416–423. Cited by: §II, §II.
  • [16] J. Ryu, J. Artigas, and C. Preusche (2010) A passive bilateral control scheme for a teleoperator with time-varying communication delay. Mechatronics 20 (7), pp. 812–823. Cited by: §I, §I, §III-A.
  • [17] Y. S. Sarkisov, M. J. Kim, D. Bicego, D. Tsetserukou, C. Ott, A. Franchi, and K. Kondak (2019) Development of sam: cable-suspended aerial manipulator. In IEEE International Conference on Robotics and Automation (ICRA), Cited by: Fig. 1, §I.
  • [18] P. Schmaus, D. Leidner, T. Krüger, A. Schiele, B. Pleintinger, R. Bayer, and N. Y. Lii (2018) Preliminary insights from the meteron supvis justin space-robotics experiment. IEEE Robotics and Automation Letters 3 (4), pp. 3836–3843. Cited by: §I.