I Introduction
Day by day realistic robotic applications are bringing robots into human environments such as houses, hospitals, and museums where they are expected to assist us in our daily life tasks. Such humaninhabited environments are highly unstructured, dynamic and uncertain, making hardcoding the environments and related skills infeasible.
In this context, human expertise can be exploited to teach robots how to perform such tasks by transferring human skills to robots [17]. Learningfromhumandemonstrations (LfD) has been widely studied as a convenient way to transfer human skills to robots. This learning approach is aimed at extracting relevant motion patterns from human demonstrations and subsequently applying these patterns to different situations. In the past decades, several LfD based approaches have been developed such as: dynamic movement primitives (DMP) [9, 2], probabilistic movement primitives (ProMP) [13]
, Gaussian mixture models (GMM) along with Gaussian mixture regression (GMR)
[4], and more recently, kernelized movement primitives (KMP) [8, 7]. DMPs have several beneficial properties such as robustness against perturbations and ability to adapt to new requirements such as a new goal.However, many tasks in those environments require variable impedance [10, 19, 1] or high manipulability [6, 21, 12, 15]), the parameters of which are encapsulated in symmetric positive definite (SPD) matrices. Because of the structure of the manifold of SPD matrices, standard LfD approaches such as DMPs can not be directly used as they rely on Euclidean parametrization of the space.
In this paper, we a novel formulation for DMPs using Riemannian metrics such that the resulting formulation can operate with SPD data. This allows variable SPD quantities to be modeled while retaining the useful properties of standard DMPs. The main contributions are

A novel and mathematically principled framework for reformulating DMPs using Riemanian metrics, in order to learn and reproduce SPDmatricesbased robot skills.

Reformulating standard DMP goal switching to be able to handle SPDmatrixbased robot skills.
This extension of DMPs to Riemannian manifolds allows the generation of smooth trajectories for data that do not belong to the Euclidean space.
The work is inspired by quaternion and rotation matrix based formulations of DMPs [2, 20] which target specifically the problem of parametrizing the space of orientations
. Compared to the tensorbased formulation of GMM and GMR on Riemannian manifold of SPD matrices
[11] (demonstrated for manipulability transfer in [15]), the proposed approach allows adapting the SPD profile online to a new goal, similar to the difference between standard GMM/GMR and DMPs for Euclidean quantities.This paper is organized as follows: We begin by providing background about standard DMPs (Section IIA) and Riemannian manifold of SPD matrices (Section IIB). Afterwards, we exploit Riemannian manifold to derive the new formulation of DMPs (Section IIIA) followed by goal switching formulation (Section IIIB). Subsequently, we evaluate our approach through several examples (Section IV). The work is concluded in Section V.
Ii Background
In this scope we introduce a brief introduction to standard DMPs and Riemannian manifold of SPD matrices.
Iia Dynamic Movement Primitives
The basic idea of DMPs is to model movements by a system of differential equations that ensure some desired behavior, e.g. convergence to the specified attractor point [16, 9, 2]
. A DMP for a single degree of freedom trajectory
is defined by the following set of nonlinear differential equations [9](1)  
(2)  
(3) 
where is the scaled velocity, is the phase variable to avoid explicit time dependency and , and define the behavior of the 2nd–order system, is the goal of the movement, and is a nonlinear forcing term that provides a modeling of complex trajectories. Choosing a time constant along with and will make the linear part of (1) and (2) critically damped, which insures the convergence of and to a unique attractor point at and [9]. is defined as a linear combination of
nonlinear radial basis functions, which enables the robot to follow any smooth trajectory from the initial position
to the final configuration(4) 
(5) 
where
are the centers of Gaussians distributed along the phase of the movement and
their widths. For a given and setting equal to the total duration of the desired movement, we can define , and where . For each DoF, the weights should be adjusted from the measured data so that the desired behavior is achieved. For controlling a robotic system with more than one DoF, we represent the movement of every DoF with its own equation system (1)–(2), but with the common phase (3) to synchronize them.IiB Riemannian manifold of SPD matrices
Let us define as the set of
SPD matrices which cannot be considered as a vector space since it is not closed under addition and scalar product
[14], and thus the use of classical Euclidean space methods for treating and analyzing these matrices is inadequate. A compelling solution is to incorporate these matrices with a Riemannian metric, which allows the set of SPD matrices to form a Riemannian manifold [14]. Such a metric defines the geodesics, that is, the minimum length curves between two points on the manifold.A Riemannian manifold is a topological space, each point of which locally resembles a Euclidean space. For each point , there exists a tangent space which corresponds to the space of symmetric matrices for the case of the SPD manifold. The metric in the tangent space is flat, which allows the use of classical arithmetic tools. Note that the space of can be represented as the interior of a convex cone embedded in its tangent space of symmetric matrices .
To operate on the tangent spaces, a mapping system is required to switch between and . The two mapping operators are known as exponential and logarithmic maps:
The logarithmic map is a function that maps a point in the manifold to a point in the tangent space.
(6) 
The exponential map is a function that maps a point to a point , so that it lies on the geodesic starting from in the direction of .
(7) 
where and are the matrix logarithm and exponential functions.
Moving elements between different tangent spaces is performed by the parallel transport operator [18, 22]. The parallel transport is a function that transports to over the geodesic from to is given by
(8) 
where . Equation (8) has been proved to be computationally efficient [22]. This transporter is exploited whenever it is required to transport SPD matrices along geodesics in a nonlinear manifold.
In this paper, we exploit the Riemannian manifold to reformulate DMPs to be capable of encoding and reproducing SPDmatricesbased robot skills.
Iii Proposed Approach
In this section, we provide a complete formulation for DMPs in order to learn and reproduce SPDmatricesbased robot skills. For the sake of simplicity let us first recall the reinterpretation of basic standard operations in a Riemannian manifold (Table I). Define and .
Euclidean space  Riemannian manifold  

Subtraction  
Addition  
Distance  
Interpolation 
Iiia Geometryaware DMPs Formulation
Define a variable as an arbitrary SPD matrix and as the set of SPD matrices in one demonstration. In order to prepare the demonstration data for DMP, its 1st and 2ndtime derivatives are needed. The 1sttime derivative is computed as follows
(9) 
where each belongs to the corresponding tangent space . Afterwards, we use (8) to move all to a common/shared arbitrary tangent space, e.g. the tangent space of the first SPD data . The tangent space corresponds to , which allows the use of classical arithmetic tools as mentioned in section IIB. To avoid replicating information due to symmetry, we propose to reduce the space dimensionality of the data in the tangent space to using Mandel’s representation.
(10) 
where is a function that transforms a symmetric matrix into a vector using Mandel’s notation. E.g. a vectorization of a symmetric matrix is
(11) 
Now, the 2ndderivatives can be computed straight forward using standard Euclidean tools and its vectorization is denoted as . Having all necessary data , we transform the standard DMP system (1)–(2) into a geometryaware form as follows
(12)  
(13) 
where is the vectorization of . represents the goal SPD matrix. is the vectorization of the transported symmetric matrix over the geodesic from to . The forcing term can be recalculated as
(14) 
where the phase . Using (14), the weights
can be estimated by encoding any sampled SPDmatricesbased robot skills.
In the reproduction, equation (13) is integrated as follows
(15) 
where the function is the inverse of and denotes to the matricization using Mandel’s notation. represents the new SPDmatricesbased robot skills.
IiiB Goal switching
In the standard DMP formulation, in case of sudden goal switching (e.g. based on external sensory information) during the execution, Ijspeert et al. [9] proposed to add an additional equation to the dynamic system (1)–(2) in order to smoothly change the goal in (1) to a new goal as
(16) 
where is a constant. This equation transforms from being a constant to a continuous variable. Analogously, SPDbased DMP can switch the goal using
(17) 
so now is updated continually.
Iv Experiments
We evaluated the proposed imitation learning framework using simulated data. All algorithms have been implemented in MATLAB
^{®} using a workstation running Ubuntu 16.04 LTS with Intel Core i9–8950HK CPU @ 2.90GHz 12, 16 GB of RAM.Four different experiments were carried out to evaluate the proposed framework:

Learning and reproducing full stiffness matrix profiles with a 2DoF virtualmass springdamper system (MSD).

Goal switching applied to full stiffness matrix profiles.

Comparison of the resulting SPD profile between the proposed DMP and GMM/GMR proposed in [11].
Iva Learning Variable Impedance Skills
In this scope, we propose to use a simulation of MSD to evaluate our geometryaware DMPs for learning and reproducing variable impedance^{1}^{1}1Here we refer to variable impedance as variable stiffness profiles. Nevertheless, the proposed approach can also be used to learn variable damping controllers as well as any SPDmatrixbased robot skills. skills. Note that stiffness matrices belong to the space of . The MSD system starts from an initial, horizontallyaligned, stiffness ellipsoid at rest position. Afterwards, external forces are applied to stimulate the MSD system. During the stimulation, is rotating through ( is a rotation matrix) until it ends up with a verticallyaligned ellipsoid as shown in Fig. 1 in gray. This demonstration then is encoded using (12)–(13) to reproduce the ellipsoids in green . From the figure, we can see the match between the results of the SPDbased DMPs and the demonstration.
IvB Goal Switching
In this simulation we used the same MSD setup introduced in section IVA. However, here we are about to test the response of the proposed SPDbased DMP to sudden goal changing during the execution. At the middle of DMP execution we changed the goal by rotating it 90 degrees to be horizontallyaligned (red ellipsoid in Fig. 3) instead of being verticallyaligned (in gray).
Figure 3 shows the smoothness of the adaptation of the stiffness profile (in green) to the new goal (in red). The blue stiffness ellipsoid marks the instant of goal switching. It is clear from the figure that the resulting profile was following the demonstrated one until the blue ellipsoid, then started to adapt to the new goal. More clearification regarding the accuracy of the approach can be seen in Fig. 4. The blue part of the figure shows the distance before the occurrence of goal switching. However, the red part shows the distances between the SPDbased DMP results and the new goal. The figure illustrates that the system converges to the new goal.
IvC Comparison with GMM/GMR
In this scope we compare the proposed SPDbased DMP with SPDbased GMM/GMR proposed by [11]. For fair comparison, as DMP is trained using one demonstration, we used also this same one demonstration to train GMM. As number of Gaussian components influence the accuracy of GMM/GMR, we trained 1, 4, 7, and 10states GMMs.
For each GMM model, we calculated the distance error between the SPD profile obtained by GMR and the demonstration. Moreover, the distance error also has been calculated in the case of the proposed SPDbased DMP. Figure 5 shows the resulting distance error in all cases. From the figure, it is clear that the accuracy of GMM/GMR increases when the number of Gaussian components increases. However, increasing Gaussian components leads to a significant increase in the computation time as shown in Table II, while the proposed SPDbased DMP is significantly faster. Moreover, the GMM/GMR approach would not allow e.g. goal switching.
Time in seconds  

Learning  Reproduction  
1state GMM/GMR  2.8354  1.0788 
4states GMM/GMR  9.9459  3.9176 
7states GMM/GMR  16.3209  6.1992 
10states GMM/GMR  22.2517  8.3852 
Proposed SPDbased DMP  0.0941  0.4214 
V Conclusions
In this paper we successfully exploited the Riemannian manifold of to derive a new formulation of DMPs capable of direct learning and reproduction of SPDmatrixbased robot skills. This formulation avoids any prior reparametrization of such skills. Moreover, we integrated a new formulation for the goal switching that can deal directly with SPDmatrixbased robot skills.
The algorithm has been extensively validated through multiple simulation examples. Moreover, a comparison with GMM/GMR demonstrates that the proposed approach provides at least similar accuracy with a significantly lower computation cost.
In the future we propose to integrate our approach with other algorithms, e.g. iterative learning control, in order to not just reproduce SPDmatrixbased skills, but also to adapt to different situations and perform more complex tasks (e.g. forcedbased variable impedance control). Moreover, we will work on explorationbased learning methods, which will prove to be crucial when a robot needs to significantly adapt to a new situation, e.g. adapt its stiffness, in order to perform successfully in a large diversity of task situations.
Acknowledgment
This work is supported by CHISTERA project IPALM (Academy of Finland decision 326304).
References
 [1] (2018) Forcebased variable impedance learning for robotic manipulation. Robotics and Autonomous Systems 109, pp. 156–167. Cited by: §I.
 [2] (2015) Adaptation of manipulation skills in physical contact with the environment to reference force profiles. Autonomous Robots 39 (2), pp. 199–217. Cited by: §I, §I, §IIA.
 [3] (2006) Logeuclidean metrics for fast and simple calculus on diffusion tensors. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 56 (2), pp. 411–421. Cited by: §IVA.
 [4] (2014) A taskparameterized probabilistic model with minimal intervention control. In IEEE International Conference on Robotics and Automation, pp. 3339–3344. Cited by: §I.

[5]
(2011)
Efficient similarity search for covariance matrices via the jensenbregman logdet divergence.
In
2011 International Conference on Computer Vision
, pp. 2399–2406. Cited by: §IVA.  [6] (2006) Manipulability optimization for trajectory generation. In IEEE International Conference on Robotics and Automation, pp. 2017–2022. Cited by: §I.
 [7] (2019) Generalized orientation learning in robot task space. In IEEE International Conference on Robotics and Automation, pp. 2531–2537. Cited by: §I.
 [8] (2019) Kernelized movement primitives. The International Journal of Robotics Research 38 (7), pp. 833–852. Cited by: §I.
 [9] (2013) Dynamical movement primitives: learning attractor models for motor behaviors. Neural Computation 25 (2), pp. 328–373. Cited by: §I, §IIA, §IIIB.
 [10] (1995) Variable impedance control of a robot for cooperation with a human. In IEEE International Conference on Robotics and Automation, Vol. 3, Nagoya, Japan, pp. 3097–3102. Cited by: §I.
 [11] (2017) Gaussian mixture regression on symmetric positive definite matrices manifolds: application to wrist motion estimation with semg. In IEEE/RSJ International Conference on Intelligent Robots and Systems, Vancouver, Canada, pp. 59–64. Cited by: §I, 3rd item, §IVC, TABLE II.
 [12] (2016) Humanoid posture selection for reaching motion and a cooperative balancing controller. Journal of Intelligent & Robotic Systems 81 (34), pp. 301–316. Cited by: §I.
 [13] (2013) Probabilistic movement primitives. In Advances in Neural Information Processing Systems, pp. 2616–2624. Cited by: §I.
 [14] (2006) A riemannian framework for tensor computing. International Journal of Computer Vision 66 (1), pp. 41–66. Cited by: §IIB, TABLE I.
 [15] (2017) Learning manipulability ellipsoids for task compatibility in robot manipulation. In IEEE/RSJ International Conference on Intelligent Robots and Systems, Vancouver, Canada, pp. 3183–3189. Cited by: §I, §I.
 [16] (2007) Dynamics systems vs. optimal control—a unifying view. Progress in brain research 165, pp. 425–445. Cited by: §IIA.
 [17] (1999) Is imitation learning the route to humanoid robots?. Trends in Cognitive Sciences 3 (6), pp. 233–242. Cited by: §I.
 [18] (2015) Conic geometric optimization on the manifold of positive definite matrices. SIAM Journal on Optimization 25 (1), pp. 713–739. Cited by: §IIB.
 [19] (2002) Variable impedance control based on estimation of human arm stiffness for humanrobot cooperative calligraphic task. In IEEE International Conference on Robotics and Automation, Vol. 1, Washington, DC, USA, pp. 644–650. Cited by: §I.
 [20] (2014) Orientation in cartesian space dynamic movement primitives. In 2014 IEEE International Conference on Robotics and Automation, pp. 2997–3004. Cited by: §I.
 [21] (2012) Manipulability analysis. In 12th IEEE/RAS international conference on humanoid robots (humanoids), pp. 568–573. Cited by: §I.
 [22] (2019) Parallel transport on the cone manifold of spd matrices for domain adaptation. IEEE Transactions on Signal Processing 67 (7), pp. 1797–1811. Cited by: §IIB.
Comments
There are no comments yet.