DeepAI AI Chat
Log In Sign Up

Generating and Estimating Nonverbal Alphabets for Situated and Multimodal Communications

by   Serhii Hamotskyi, et al.
Université Paris 8

In this paper, we discuss the formalized approach for generating and estimating symbols (and alphabets), which can be communicated by the wide range of non-verbal means based on specific user requirements (medium, priorities, type of information that needs to be conveyed). The short characterization of basic terms and parameters of such symbols (and alphabets) with approaches to generate them are given. Then the framework, experimental setup, and some machine learning methods to estimate usefulness and effectiveness of the nonverbal alphabets and systems are presented. The previous results demonstrate that usage of multimodal data sources (like wearable accelerometer, heart monitor, muscle movements sensors, braincomputer interface) along with machine learning approaches can provide the deeper understanding of the usefulness and effectiveness of such alphabets and systems for nonverbal and situated communication. The symbols (and alphabets) generated and estimated by such methods may be useful in various applications: from synthetic languages and constructed scripts to multimodal nonverbal and situated interaction between people and artificial intelligence systems through Human-Computer Interfaces, such as mouse gestures, touchpads, body gestures, eyetracking cameras, wearables, and brain-computing interfaces, especially in applications for elderly care and people with disabilities.


Automatized Generation of Alphabets of Symbols

In this paper, we discuss the generation of symbols (and alphabets) base...

Natural interaction with traffic control cameras through multimodal interfaces

Human-Computer Interfaces have always played a fundamental role in usabi...

Human or Machine? It Is Not What You Write, But How You Write It

Online fraud often involves identity theft. Since most security measures...

Guidelines for creating man-machine multimodal interfaces

Understanding details of human multimodal interaction can elucidate many...

Deep Learning for Fatigue Estimation on the Basis of Multimodal Human-Machine Interactions

The new method is proposed to monitor the level of current physical load...