Simultaneous Estimation of Hand Configurations and Finger Joint Angles using Forearm Ultrasound

11/29/2022
by   Keshav Bimbraw, et al.
7

With the advancement in computing and robotics, it is necessary to develop fluent and intuitive methods for interacting with digital systems, augmented/virtual reality (AR/VR) interfaces, and physical robotic systems. Hand motion recognition is widely used to enable these interactions. Hand configuration classification and MCP joint angle detection is important for a comprehensive reconstruction of hand motion. sEMG and other technologies have been used for the detection of hand motions. Forearm ultrasound images provide a musculoskeletal visualization that can be used to understand hand motion. Recent work has shown that these ultrasound images can be classified using machine learning to estimate discrete hand configurations. Estimating both hand configuration and MCP joint angles based on forearm ultrasound has not been addressed in the literature. In this paper, we propose a CNN based deep learning pipeline for predicting the MCP joint angles. The results for the hand configuration classification were compared by using different machine learning algorithms. SVC with different kernels, MLP, and the proposed CNN have been used to classify the ultrasound images into 11 hand configurations based on activities of daily living. Forearm ultrasound images were acquired from 6 subjects instructed to move their hands according to predefined hand configurations. Motion capture data was acquired to get the finger angles corresponding to the hand movements at different speeds. Average classification accuracy of 82.7 kernels was observed on a subset of the dataset. An average RMSE of 7.35 degrees was obtained between the predicted and the true MCP joint angles. A low latency (6.25 - 9.1 Hz) pipeline has been proposed for estimating both MCP joint angles and hand configuration aimed at real-time control of human-machine interfaces.

READ FULL TEXT

page 1

page 2

page 4

page 5

page 6

page 9

page 13

research
09/23/2021

Prediction of Metacarpophalangeal joint angles and Classification of Hand configurations based on Ultrasound Imaging of the Forearm

With the advancement in computing and robotics, it is necessary to devel...
research
01/31/2023

Ultrasound Based Prosthetic Arm Control

The creation of unique control methods for a hand prosthesis is still a ...
research
09/06/2018

Sparsity Analysis of a Sonomyographic Muscle-Computer Interface

Objective: The objectives of this paper are to determine the optimal loc...
research
02/10/2022

Towards Predicting Fine Finger Motions from Ultrasound Images via Kinematic Representation

A central challenge in building robotic prostheses is the creation of a ...
research
02/21/2021

Mapping Surgeon's Hand/Finger Motion During Conventional Microsurgery to Enhance Intuitive Surgical Robot Teleoperation

Purpose: Recent developments in robotics and artificial intelligence (AI...
research
08/31/2023

Diffusion Inertial Poser: Human Motion Reconstruction from Arbitrary Sparse IMU Configurations

Motion capture from a limited number of inertial measurement units (IMUs...

Please sign up or login with your details

Forgot password? Click here to reset