For problems in grasping soft objects, modeling and simulation of deformable objects have recently been considerable interest in many robotic applications [12, 8], to manipulating soft tissue [23, 9], cloth [10, 42], and even fluids [13, 38]. Among soft structures, deformable linear objects (DLOs), including rope-like objects, strings, cables, beams, etc., are studied (cable routing , wire insertion , flexible rope , and knotting of surgical thread ). Recently, techniques involving visual servoing [50, 49], latent dynamics learning , and adaptive estimation , have all be explored for controlling DLOs. Further reviews in this space can be found in review papers on the topic [44, 37, 48].
Classical methods for modeling rope reside from Cosserat rod theory 
which involves an analytical, partial differential equation representing the rope as a continuous 3D-dimensional curve exhibiting both bending and torsion. However, the analytical dynamics are limited by the computational efficiency and stability of solving this PDE with two-point boundary conditions (given by the start and end of the rope).
Alternative to exact analytical models, a model-free method may try to identify representations of rope using purely visual descriptors from a camera without involving classical mechanics. These model-free representations include curvature and Fourier-based shapes [31, 32]
, as well as neural network features[39, 15, 40]. Instead of directly modeling physical mechanics, the state space representation is embedded in these latent space features. However, these approaches need a large quantity and variety of data for training, which is not easy to get in field robotic applications (either in simulation or real life). Meanwhile, the collected datasets are limited to the distribution of objects manipulated. Thus models that learn from simulation or limited real-world data encounter a real-to-sim transfer challenge or an over-fitting problem, respectively.
An increasing number of simulators are made available for robot learning involving rope-like objects [18, 20]. These simulators have been focused on their use for data generation and lack a method for integration in a model-based control context.In general, many existing robotics simulators (including the above, as well as Bullet ) and Gym , unable to simulate soft bodies) only support gradient calculations based on finite differences. However, differentiable physics and simulation have become critical features for the robot learning community . The works in [5, 33, 41, 22] have applied differentiable framework into rigid articulated body simulation, while others have done the same for deformable objects [16, 6, 14]. These methods provide gradients within the differentiable framework, where optimization tasks for modeling and control are natively supported. Meanwhile, the differentiability can be easily deployed into neural layers for learning-based methods, such as in [34, 26].
One simulation approach that has gained significant interest due to its speed, stability, and capability to model the spectrum from rigid to soft to liquid is Position-based dynamics (PBD) [27, 28]. Unlike the traditional Lagrangian force-based method, the geometric position constraints are solved iteratively. Simulating soft objects involves utilizing the Compliant Position-based Dynamics (XPBD) method  which has been shown for a variety of deformable solid bodies, cloth, and chains. Some researchers have directly integrated the Cosserat model into PBD simulators to define the twist and bend, stretch and shear the a the constraints, by introducing particles with orientation information to describe angular updates . Next, the authors added scale parameters for rope-like objects to preserve volume consistency . Although several works have been applied for real-to-sim tasks of soft deformable tissue [23, 9], rigid articulated robots , and fluids [13, 35], with constraint-based formulation of PBD or XPBD. However, there is a missing combination of modeling and control approaches applied to rope-like objects in literature, exploiting the capability of XPBD.
In the paper, we propose a solution to formulate a real-to-sim modeling and control framework. We extended the original position-based dynamics (PBD) for rope-like objects proposed in  with a compliance parameter by following the XPBD work in 
. Meanwhile, it is deployed differently using the automatic differentiation functionality available in PyTorch for gradient updates. The new compliant position-based dynamics (XPBD) for rope-like objects can guarantee stable forward simulation while the back-propagation of losses can be applied for modeling and control tasks and parameter identification. The contributions of this paper are as follows:
We describe rope-like objects as a compliant position-based dynamics (XPBD) model by constructing geometrical constraints defining their behaviors.
We introduced a differentiable framework for modeling and simulation that works well with auto-differentiation and algorithm configurations for constraint solving.
We defined the problem of parameter identification and manipulation control of rope-like objects in real-to-sim context (Fig. 1).
We validate our methods in real-world experiments using robot manipulators (Baxter) and surgical robots (dVRK).
Ii-a Compliant Position-Based Dynamics (XPBD) Modeling of Rope-like Object
The XPBD method will be a foundation for building the differentiable model for DLOs. We firstly discretize the DLO into a sequence of particles (Fig. 2) with Cartesian coordinates . Meanwhile, quaternions are used to describe orientations in-between adjacent particles , and will be used to solve the bending and twist deformation of the DLO. Unlike force-based methods such as Euler-Bernoulli beam or Cosserat rod theory, full strain and torsion deformations can be updated with a position-based dynamics solver. As with all simulations using PBDs, the methods starts with a list of constraints, that describe the dynamics of particles. Solving the constraints involves updating and directly using a non-linear projected Jacobi method used for constrained optimization problems. The method to solve for the gradient updates is derived below.
Constraints can be linearized by Taylor series expansion,
where the Lagrange multiplier change can be found by introducing the compliance parameter ,
where and are the weighted terms to guarantee the conservation of linear and angular momentum. Generally, it refers to the mass/inertia matrix as and . For simplification of DLOs, we use uniformed scalar weights instead of matrices for each dimensionality, which has not much impact on simulation and dynamical performance but reduces computational load. That is,
Next, we will introduce several geometrical constraints to simulate the behaviors of the DLOs.
Ii-A1 Shear and Stretch Constraint
According to Cosserat theory, the shear and stretch measures the deformation regarding the tangent direction of the rope-like object. Therefore, the stretch/compressed length should be constrained relative its rest pose, which indicates in-extensible elasticity. Simultaneously, the normal direction (i.e., the rotated from world frame noted by ) for each cross-section should be parallel to the tangent direction of object’s centerline, see Fig. 2. It measures the shear strain with respect to non-deformed states. Thus, for each pair of neighboring particles, the shear-stretch deformation can be integrated into a generalized constraint as, , which is,
where is the rotation matrix from the local frame of line segment to world frame represented using quaternion, represent the states in rest pose. According to gradient calculation in , we can obtain,
Ii-A2 Bend and Twist Constraint
In differential geometry, the Darboux vector is used to parameterize strain deformation with respect to frame rotation. According to Cosserat theory, the Darboux vector can be expressed as a quaternion by measuring the rod’s twist in the tangent direction. Thus, the difference between the current and resting configuration should be evaluated, i.e. . According to , the bend and twist constraint can be computed for each pair of two adjacent quaternions (shown in Fig. 2) by, , which is,
where is the imaginary part of the quaternion and is the conjugate quaternion. The constraint gradients can be calculated as,
Ii-A3 Distance Constraint
One property of modeling ropes is that they can be considered either extensible or inextensible based on how stiff they are, and ultimately this can be modeled as a constraint. Although the above strain deformation have considered the inextensible property implicitly, the iterative solver in Eq. 2 cannot guarantee that all the constraints will be satisfied. Therefore, we can explicitly consider the distance constraint for an enhanced inextensible chain structure using only discretized particle positions. We define by,
The gradients can be easily obtained by,
Ii-B Real-to-Sim Parameter Identification
Given the above constraints, the dynamics of a rope-like object can be represented by a set of discretized particles (shown in Fig. 2) with position and orientation evolution. However, it is just an approximation of the real model for the rope objects.
For the above shear/shear, bend/twist and distance constraints, we can introduce the additional stiffness parameter to weight the updates during iteration steps, namely by,
Which stands for shear/stretch, bend/twist, and distance constraints regarding position or orientation. Moreover, stands for position changes due to external gravity, and stands for the successive over-relaxation parameter and is applied to accelerate the convergence speed further. Thus, we can introduce the above parameters in a real-to-sim setup by inserting them into the following differentiable framework.
Ii-C Differential Framework
To perform an optimal control task, the ready-to-use gradients will be needed to compute. We introduce the compliant position-based dynamics (XPBD) inside a differentiable framework, as shown in Algorithm 1 111In this paper, we only consider the quasi-static dynamical states. Only the gravity will be regarded as without any external torques. Thus, the Euler prediction and integration of velocity and angular speed will not be involved.
. We rely on the automatic differentiation function provided by PyTorch to obtain the gradients. It can be natively integrated into optimization or learning methods, since the coding framework can be viewed as a differentiable layer which support both forward and backpropagation operations. The computing memory might be limited by the number of iterations for substep simulation. Denoting the gradient variable at timeby , we can formulate the following optimization problem as,
where , and are the position-based constraints, and
is the loss function derived from the system states.represents the selected gradient variable, which can be control states, system parameters or system states.
Iii Real-to-Sim and Shape Control Problem Setup
In this part, we used the proposed differentiable framework to conduct three different experiments both on Rethink Baxter and the da Vinci Research Kit (dVRK) robotic platforms. The Baxter represents a situation of more significant rope manipulation, whereas the dVRK represents a situation of surgical automation involving blood vessel manipulation. We considered the Baxter-rope experiment to involve an inextensible DLO, while the dVRK blood-vessel experiment to be an extensible DLO. For both setups, we looked at both a real-to-sim problem (i.e., parameter estimation based on observations from the real world), and a control problem (i.e., using the XPBD model we developed to have rope configuration reach a target configuration iteratively). The implementation of each experiment will be provided later.
Iii-a Solver Setups
In classical PBD solver , the Jacobi approach averages each constraint step changes, and keeps updating iteratively. The convergence compromises between the number of iterations and the satisfaction of other constraints. However, it was not suitable for simulation of the inextensible effects for ropes with significant axial stiffness.
Precisely, we needed to guarantee the fulfillment of the distance constraint in Eq. 9 completely. A direct linear solver was proposed  based on the tridiagonal matrix algorithm, i.e., Thomas algorithm, to preserve the inextensible characteristics of rope-like objects. We will refer to methods individually as the Jacobi XPBD and Thomas XPBD, respectively. All other constraints, such as shear/stretch and bend/twist constraints, used the Jacobi method.222Thus, in this paper, when Jacobi XPBD and Thomas XPBD are indicated, it is distinguishing how the distance constraint is solved.
Iii-B Data Preprocessing
In our experiments, we obtained both the point cloud and RGB image data using the Microsoft Azure Kinect. For the 2D images obtained, we extracted the centerline of the rope using MATLAB skeletonization function333https://www.mathworks.com/help/images/ref/bwskel.html.
Since the raw point cloud data was noisy, we projected them to a hyperplane defined by the gravity vector and two endpoints of the rope, as shown in Fig.3. The projected point cloud was constrained within a plane, which was easier for identification of bend/twist effects. We used the de-noised 2D centerline from the image and projected 3D point clouds as ground truth data for loss computation in the following section.
Iii-C Loss Functions
We defined four types of primary losses to investigate for shape matching. We looked at the combinations of these losses for ablation, resulting in 9 different aggregate loss functions for optimization. The loss functions are shown in Table I.
Point-to-Line : As shown in Fig. 4, we intended to find the minimum distance from each preprocessed real data point , to the line segments between each pair of adjacent simulated particle . Thus, the primary loss regarding point-to-line loss was
where we discretized the rope to particle nodes, along preprocessed real data points, and was the projected locations on each line segment with minimum distance.
Segment-to-Particle : We splitted the real data points (preprocessed for noises and re-ordered) into different segments along the vector formulated by two rope endpoints, i.e. , etc shown in Fig. 4. The start/endpoints (i.e., , ) of each segment were determined according to the proportional length to the whole data set. We evaluated the distance between the start/endpoints of each segment and the corresponding particle in the simulation. Then, the segment-to-particle loss was,
Segment-to-Line : As above, we already divided the real data into segments. The projection distance between all points in each segment set () and the corresponding line segment () could be used to evaluate the loss. Thus, the segment-to-line loss was,
where was the number of real data points for each segment group set, and was the projected locations on the line segment with minimum distance.
Lowest 3D Point Along Gravity Vector : The lowest point along the gravity direction of the denoised real data was compared with the one in simulation. This could compensate for lacking depth information while using the 2D centerline. The loss was defined as,
In our experiments, the point-to-line, segment-to-particle, and segment-to-line losses could be applied to either the 3D point cloud or 2D centerline. We performed 9 different combinations of types of losses as shown in Table I. The loss function , , contained 3D information only, while , , were considering 2D cases only. , , were applied to both 2D cases and the included 3D lowest point with depth information.
|Meaning||Point-to-Line for 3D and 2D||Segment-to-Particle for 3D and 2D||Segment-to-Line for 3D and 2D|
Iv Experiments And Result Analysis
Iv-a Inextensible Rope Parameter Identification
Parameter identification and position estimation of the control point were carried out on an inextensible rope manipulated by the Baxter robot. We fixed one endpoint of the rope to the environment, and the other endpoint of the rope to the manipulator end-effector (referred as the control point). We continuously moved the control point and collected 36 frames of point cloud for the rope deformation. The Microsoft Azure Kinect was used to obtain the 2D RGB image frames for centerline extraction at the same time. Since only translational movement (fixed rotation angle) of end-effector is controlled, the point cloud had occlusions and was not fully visible at the local region near the control point. We applied the keypoint-based kinematics reconstruction from  to identify the 3D position of the control point. Therefore, the ground truth data for 3D point cloud and 2D centerline, as well as control point were ready for real-to-sim transfer.
In XPBD simulation, considering the computational cost and stiffness of the rope, the number of particle were set by . was the control point and corresponding to the endpoint was fixed to the environment. According to Section II-B, there were six constraint stiffness parameters needed to estimate, as shown in Table II. Because 3D point cloud could be more informative than 2D centerline which lacked depth data, the parameters were optimized with using the 3D point-to-line loss. We then estimated the position of the control point (i.e. , see Algorithm 1) using all loss functions in Tab. I to evaluate the accuracy of parameter identification.
For parameter identification, a single gravity parameter was inferred based on Thomas solver, the comparison of simulated results before and after parameter optimization was shown in Fig. 5. After parameter optimization, the simulation states was approximately approaching to the ground truth point cloud. As for the Jacobi solver, it needed two sets of gravity stiffness to deal with different convergency of distance constraint for varied rope states (bending and tensioned), as shown in Table. III.
|0.04||0.024||position stiffness (gravity)|
|1.0||0.48||position stiffness (distance)|
|1.0||1.16||position stiffness (shear/stretch)|
|1.0||0.52||quaternion stiffness (shear/stretch)|
|1.0||1.49||quaternion stiffness (bending/twist)|
|1.0||0.61||successive over-relaxation weight|
For position estimation of control point, we made a comparison over the nine kinds of loss functions shown in Table I firstly. The 3D point cloud included the rope information in 3D space, while the 2D centerline did not contain depth information. As a result, the simulation result containing the 3D information showed the best performance, i.e., , , and as shown in the Fig. 7. For , , and , whose loss function consisted of 2D information and partial 3D information (the lowest point along gravity direction), the simulation result was improved compared to , , and , which only considered 2D information. As shown in Fig. 7 and Fig. 6 , obtained the best performance among all these losses. It simply proved the accuracy of parameter identification, and showed that the 3D data contributed more than 2D cases for control point estimation.
We also compared the performance of Thomas XPBD and Jacobi XPBD when estimating the position of the control point. The middle plot in Fig. 13 showed the rope length change of the simulated result from 36 frames. The average length deviation over the ground truth length for Thomas XPBD solver was for all structures, but for Jacobi solver, it was . Especially when the rope became tense, the error for Jacobi solver became more significant. The same phenomenon occurred on the error of the control point’s position in Fig. 9. The left and right plots in Fig. 13 indicated that the distance constraint in the Jacobi XPBD was hard to be satisfied when the rope was approaching the tensioned status. The Jacobi XPBD intuitively tried to comprise the satisfaction of each constraint solving. It iteratively struggled to maintain the stretch, twist, and distance against gravity. Thus, it resulted in a no-complete convergency of distance constraint . For Thomas XPBD, the distance constraint would be satisfied between each neighboring node simultaneously within one step solving, which was similar as the position update mechanism of the gravity. In this case, Thomas XPBD solver ensured that the length of rope was unchanged, and it was more suitable for the simulation of inextensible rope. The simulation result shown in Fig. 8 proved our conclusion.
Iv-B Extensible Rope Parameter Identification and Key Points Estimation
Parameter identification and position estimation of control point were carried out on an extensible, flexible silicone rod using the dVRK surgical robot. The silcone rod was chosen to resemble a compliant and dissected vessel, a technique that is often used in surgery to avoid damaging the vessel. The two endpoints of the rope were fixed, and the control point was in the middle of the rope. We moved the control point to collect 30 frames of point cloud data. In simulation, the number of particle nodes were . and corresponding to both the endpoints were fixed and was the control point. We did not need to preserve the inextensible behavior completely since the silicon rope had a lower axial stiffness. Thus, we only implemented the Jacobi XPBD solver for the distance constraint in this experiment. The loss function used was in Table I, and we estimated the same parameters as the ones indicated in Section II-B. After parameter estimation, we estimated the position of the control point (i.e. ) using and , i.e., point-to-line for both 3D and 2D cases. For parameter identification, we used grid-search to identify the optimized six parameters in Table III for extensible rope. Since the control point was set at the middle of the rope, manipulation could result in one side being tight and the other being loose. Thus, the effect of the gravity over two sides was different, and we needed to define different gravity weights for the two sides, as shown in Fig. 10.
|Parameter||Inextensible rope||extensible rope|
|0.26 (bending) / 0.001 (tensioned)||25 (relax) / 3 (tensioned)|
With the optimized parameters, we estimated the position of the control point and a reference point, namely by key points. Apart from the control point, there was a reference point with a red marker, shown in Fig. 10. The marker was used only for comparison between ground truth trajectory and simulation result in Fig. 12 Right and not for parameter estimation. Fig. 12 Left showed the error curve of the location of reference and control point. Even though the raw data of the extensible rope was noisier, the average error of the control point and the reference point was acceptable, which proved the robustness of our solver. In Fig. 11, we showed the original image and simulation result for different frames, and no matter how the rope deformed, our solver could get a desirable outcome. Even though Thomas solver outperformed the Jacobi solver in the previous experiment, Thomas XPBD was too tough to simulate the silicon rope. Thomas XPBD required the length of the rope to remain unchanged, which contradicted the silicon rope’s extensible property. Thus, the Thomas solver diverged for the simulation, and the Jacobi solver was more suitable.
Iv-C Inextensible Rope Shape Control
Based on the Thomas XPBD solver, the shape control task was carried out on the same Baxter setup as the Section IV-A. The ground truth shape of the rope was obtained by locating the left control point (end-effector), right fixed endpoint and the middle three Aruco Markers shown  in the Fig. 1 and Fig. 14. We fixed the simulated rope to the same right endpoint. By setting the middle three points as the shape target, we optimized the position of the control point from the differentiable Thomas XPBD simulation.
As shown in Fig. 14, the transparent rope with low opacity was our shape control target status, and the solid ones with high opacity were the result controlled by the identified parameters from Section IV-A. Our controlled shape could almost overlap with the target rope, as shown in the Top Row of Fig. 14. However, some had deviations as shown in the Bottom Row of Fig. 14. One of the reason is we only considered three shape points as the target for loss computation. The accuracy will be improved by considering more discretized segments of the ground truth data. Another reason was because we did not consider the rotational control of the end-effector when solving the inverse kinematics of the Baxter, which caused the rope not to be able to move to the desired position perfectly.
V Conclusion and Future Works
This paper used a compliant position-based framework to conduct the differential real-to-sim tasks for parameter identification and shape control tasks. Several geometrical constraints were introduced to model the rope-like objects’ coupling stretch/shear and bending/twisting effects. To inspect the inextensible and extensible impact, the Thomas solver and the Jacobi solver are proposed for the distance constraint. The experiment results on the Baxter robot and DVRK platform proved the validity and robustness of our solvers. The shape control tasks showed a novel path for real-to-sim robotic manipulation operations.
The future works will consider differentiable control with collision handling and rigid-deformable coupling. Meanwhile, more advanced tasks will be considered, such as surgical thread manipulation in field environments using the proposed inextensible solver.
Many thanks to Yutong Zhang for the rendering of the cover photo. This work was supported by NSF CAREER award 2045803 and the US Army Telemedicine and Advanced Technologies Research Center.
-  (2019) VIPER: volume invariant position-based elastic rods. CoRR abs/1906.05260. External Links: Cited by: §I.
-  (2016) Openai gym. arXiv preprint arXiv:1606.01540. Cited by: §I.
PyBullet, a python module for physics simulation for games, robotics and machine learning. Note: http://pybullet.org Cited by: §I.
-  (2021) Accelerating surgical robotics research: a review of 10 years with the da vinci research kit. IEEE Robotics and Automation Magazine. Cited by: Differentiable Robotic Manipulation of Deformable Rope-like Objects Using Compliant Position-based Dynamics.
-  (2018) End-to-end differentiable physics for learning and control. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31, pp. . External Links: Cited by: §I.
-  (2021) DiffPD: differentiable projective dynamics. External Links: Cited by: §I.
-  (2018) Forward and inverse problems in the mechanics of soft filaments. Royal Society open science 5 (6), pp. 171628. External Links: Cited by: §I.
-  (2021) Learning generalizable vision-tactile robotic grasping strategy for deformable objects via transformer. CoRR abs/2112.06374. External Links: Cited by: §I.
-  (2021) A 2d surgical simulation framework for tool-tissue interaction. External Links: Cited by: §I, §I.
-  (2020) VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation. In Robotics: Science and Systems Current file Overview 8 (RSS), Cited by: §I.
-  (2020) DiffTaichi: differentiable programming for physical simulation. External Links: Cited by: §I.
-  (2021) DefGraspSim: simulation-based grasping of 3d deformable objects. External Links: Cited by: §I.
-  (2021) Model-predictive control of blood suction for surgical hemostasis using differentiable fluid simulations. In 2021 IEEE International Conference on Robotics and Automation (ICRA), Vol. , pp. 12380–12386. External Links: Cited by: §I, §I.
-  (2021) PlasticineLab: a soft-body manipulation benchmark with differentiable physics. In International Conference on Learning Representations, External Links: Cited by: §I.
-  (2021) Keypoint-based bimanual shaping of deformable linear objects under environmental constraints using hierarchical action planning. CoRR abs/2110.08962. External Links: Cited by: §I.
-  (2021) GradSim: differentiable simulation for system identification and visuomotor control. External Links: Cited by: §I.
-  (2016-07) Position and orientation based cosserat rods. pp. . External Links: Cited by: §I, §I, §II-A1, §II-A2.
-  (2021) ReForm: a robot learning sandbox for deformable linear object manipulation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), Vol. , pp. 4717–4723. External Links: Cited by: §I.
-  (2020) Automatic shape control of deformable wires based on model-free visual servoing. IEEE Robotics and Automation Letters 5 (4), pp. 5252–5259. External Links: Cited by: §I.
SoftGym: benchmarking deep reinforcement learning for deformable object manipulation. In Conference on Robot Learning, Cited by: §I.
-  (2022) Parameter identification and motion control for articulated rigid body robots using differentiable position-based dynamics. External Links: Cited by: §I.
-  (2022) Parameter identification and motion control for articulated rigid body robots using differentiable position-based dynamics. External Links: Cited by: §I.
-  (2021) Real-to-sim registration of deformable soft tissue with position-based dynamics for surgical robot autonomy. In 2021 IEEE International Conference on Robotics and Automation (ICRA), Vol. , pp. 12328–12334. External Links: Cited by: §I, §I.
-  (2020) A learning-driven framework with spatial optimization for surgical suture thread reconstruction and autonomous grasping under multiple topologies and environmental noises. CoRR abs/2007.00920. External Links: Cited by: §I.
-  (2021) Pose estimation for robot manipulators via keypoint optimization and sim-to-real transfer. External Links: Cited by: Fig. 8, §IV-A.
DiffAqua: a differentiable computational design pipeline for soft underwater swimmers with shape interpolation. ACM Transactions on Graphics (TOG) 40 (4), pp. 132. Cited by: §I.
-  (2017) Position-based simulation methods in computer graphics. Eurographics Tutorial. Cited by: §I, §III-A.
-  (2014-07) Unified particle physics for real-time applications. ACM Trans. Graph. 33 (4). External Links: Cited by: §I.
-  (2016) XPBD: position-based simulation of compliant constrained dynamics. MIG ’16, New York, NY, USA, pp. 49–54. External Links: Cited by: §I, §I, §II-A.
-  (2020) Learning when to trust a dynamics model for planning in reduced state spaces. IEEE Robotics and Automation Letters 5 (2), pp. 3540–3547. External Links: Cited by: §I.
-  (2014) On the visual deformation servoing of compliant objects: uncalibrated control methods and experiments. The International Journal of Robotics Research 33 (11), pp. 1462–1480. External Links: Cited by: §I.
-  (2018) Fourier-based shape servoing: a new feedback method to actively deform soft objects into desired 2-d image contours. IEEE Transactions on Robotics 34 (1), pp. 272–279. External Links: Cited by: §I.
-  (2021-18–24 Jul) Efficient differentiable simulation of articulated bodies. In Proceedings of the 38th International Conference on Machine Learning, M. Meila and T. Zhang (Eds.), Proceedings of Machine Learning Research, Vol. 139, pp. 8661–8671. Cited by: §I.
-  (2021) Differentiable simulation of soft multi-body systems. In Advances in Neural Information Processing Systems, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan (Eds.), External Links: Cited by: §I.
-  (2021) Image based reconstruction of liquids from 2d surface detections. External Links: Cited by: §I.
-  (2018) Speeded up detection of squared fiducial markers. Image and Vision Computing 76, pp. 38–47. External Links: Cited by: §IV-C.
-  (2018) Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey. The International Journal of Robotics Research 37 (7), pp. 688–716. External Links: Cited by: §I.
-  (2018) SPNets: differentiable fluid dynamics for deep neural networks. In Proceedings of the Second Conference on Robot Learning (CoRL), Zurich, Switzerland. Cited by: §I.
-  (2021-07) Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies. In Proceedings of Robotics: Science and Systems, Virtual. External Links: Cited by: §I.
-  (2019-06) Learning robotic manipulation through visual planning and acting. In Proceedings of Robotics: Science and Systems, FreiburgimBreisgau, Germany. External Links: Cited by: §I.
-  (2021-07) Fast and Feature-Complete Differentiable Physics Engine for Articulated Rigid Bodies with Contact Constraints. In Proceedings of Robotics: Science and Systems, Virtual. External Links: Cited by: §I.
-  (2020-07) Learning to Manipulate Deformable Objects without Demonstrations. In Proceedings of Robotics: Science and Systems, Corvalis, Oregon, USA. External Links: Cited by: §I.
-  (2018-07-01) Real-time inextensible surgical thread simulation. International Journal of Computer Assisted Radiology and Surgery 13 (7), pp. 1019–1035. External Links: Cited by: §III-A.
-  (2021) Modeling, learning, perception, and control methods for deformable object manipulation. Science Robotics 6 (54), pp. eabd8803. External Links: Cited by: §I.
-  (2021) Shape control of deformable linear objects with offline and online learning of local linear deformation models. External Links: Cited by: §I.
-  (2021-05) Deformable linear object prediction using locally linear latent dynamics. In 2021 IEEE International Conference on Robotics and Automation (ICRA), External Links: Cited by: §I.
-  (2019) Dual-arm robotic needle insertion with active tissue deformation for autonomous suturing. IEEE Robotics and Automation Letters 4 (3), pp. 2669–2676. External Links: Cited by: §I.
-  (2021) Challenges and Outlook in Robotic Manipulation of Deformable Objects. IEEE Robotics and Automation Magazine. External Links: Cited by: §I.
-  (2021-05) Vision-based Manipulation of Deformable and Rigid Objects Using Subspace Projections of 2D Contours. Robotics and Autonomous Systems. External Links: Cited by: §I.
-  (2020-03) Vision-based robotic manipulation of deformable linear objects. Theses, Université Montpellier. External Links: Cited by: §I.