Convolutional Neural Networks for Speech Controlled Prosthetic Hands

by   Mohsen Jafarzadeh, et al.

Speech recognition is one of the key topics in artificial intelligence, as it is one of the most common forms of communication in humans. Researchers have developed many speech-controlled prosthetic hands in the past decades, utilizing conventional speech recognition systems that use a combination of neural network and hidden Markov model. Recent advancements in general-purpose graphics processing units (GPGPUs) enable intelligent devices to run deep neural networks in real-time. Thus, state-of-the-art speech recognition systems have rapidly shifted from the paradigm of composite subsystems optimization to the paradigm of end-to-end optimization. However, a low-power embedded GPGPU cannot run these speech recognition systems in real-time. In this paper, we show the development of deep convolutional neural networks (CNN) for speech control of prosthetic hands that run in real-time on a NVIDIA Jetson TX2 developer kit. First, the device captures and converts speech into 2D features (like spectrogram). The CNN receives the 2D features and classifies the hand gestures. Finally, the hand gesture classes are sent to the prosthetic hand motion control system. The whole system is written in Python with Keras, a deep learning library that has a TensorFlow backend. Our experiments on the CNN demonstrate the 91 output) from speech commands, which can be used to control the prosthetic hands in real-time.



page 6

page 7


End-to-End Learning of Speech 2D Feature-Trajectory for Prosthetic Hands

Speech is one of the most common forms of communication in humans. Speec...

Deep learning approach to control of prosthetic hands with electromyography signals

Natural muscles provide mobility in response to nerve impulses. Electrom...

A.I. based Embedded Speech to Text Using Deepspeech

Deepspeech was very useful for development IoT devices that need voice r...

Successes and critical failures of neural networks in capturing human-like speech recognition

Natural and artificial audition can in principle evolve different soluti...

Nanopore Base Calling on the Edge

We developed a new base caller DeepNano-coral for nanopore sequencing, w...

NICE: Noise Injection and Clamping Estimation for Neural Network Quantization

Convolutional Neural Networks (CNN) are very popular in many fields incl...

Hand Sign to Bangla Speech: A Deep Learning in Vision based system for Recognizing Hand Sign Digits and Generating Bangla Speech

Recent advancements in the field of computer vision with the help of dee...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.