Muscle Excitation Estimation in Biomechanical Simulation Using NAF Reinforcement Learning

09/17/2018
by   Amir H. Abdi, et al.
0

Motor control is a set of time-varying muscle excitations which generate desired motions for a biomechanical system. Muscle excitations cannot be directly measured from live subjects. An alternative approach is to estimate muscle activations using inverse motion-driven simulation. In this article, we propose a deep reinforcement learning method to estimate the muscle excitations in simulated biomechanical systems. Here, we introduce a custom-made reward function which incentivizes faster point-to-point tracking of target motion. Moreover, we deploy two new techniques, namely, episode-based hard update and dual buffer experience replay, to avoid feedback training loops. The proposed method is tested in four simulated 2D and 3D environments with 6 to 24 axial muscles. The results show that the models were able to learn muscle excitations for given motions after nearly 100,000 simulated steps. Moreover, the root mean square error in point-to-point reaching of the target across experiments was less than 1 method is far from the conventional dynamic approaches as the muscle control is derived functionally by a set of distributed neurons. This can open paths for neural activity interpretation of this phenomenon.

READ FULL TEXT
research
07/16/2019

Deep Reinforcement Learning Based Robot Arm Manipulation with Efficient Training Data through Simulation

Deep reinforcement learning trains neural networks using experiences sam...
research
06/12/2019

Search on the Replay Buffer: Bridging Planning and Reinforcement Learning

The history of learning for control has been an exciting back and forth ...
research
03/13/2023

Kernel Density Bayesian Inverse Reinforcement Learning

Inverse reinforcement learning (IRL) is a powerful framework to infer an...
research
12/12/2016

Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks

We propose an inverse reinforcement learning (IRL) approach using Deep Q...
research
03/27/2023

Learning a Single Policy for Diverse Behaviors on a Quadrupedal Robot using Scalable Motion Imitation

Learning various motor skills for quadrupedal robots is a challenging pr...
research
12/08/2020

Emergence of Different Modes of Tool Use in a Reaching and Dragging Task

Tool use is an important milestone in the evolution of intelligence. In ...
research
10/28/2021

Cooperative Deep Q-learning Framework for Environments Providing Image Feedback

In this paper, we address two key challenges in deep reinforcement learn...

Please sign up or login with your details

Forgot password? Click here to reset