Wearable Affective Robot

by   Min Chen, et al.

With the development of the artificial intelligence (AI), the AI applications have influenced and changed people's daily life greatly. Here, a wearable affective robot that integrates the affective robot, social robot, brain wearable, and wearable 2.0 is proposed for the first time. The proposed wearable affective robot is intended for a wide population, and we believe that it can improve the human health on the spirit level, meeting the fashion requirements at the same time. In this paper, the architecture and design of an innovative wearable affective robot, which is dubbed as Fitbot, are introduced in terms of hardware and algorithm's perspectives. In addition, the important functional component of the robot-brain wearable device is introduced from the aspect of the hardware design, EEG data acquisition and analysis, user behavior perception, and algorithm deployment, etc. Then, the EEG based cognition of user's behavior is realized. Through the continuous acquisition of the in-depth, in-breadth data, the Fitbot we present can gradually enrich user's life modeling and enable the wearable robot to recognize user's intention and further understand the behavioral motivation behind the user's emotion. The learning algorithm for the life modeling embedded in Fitbot can achieve better user's experience of affective social interaction. Finally, the application service scenarios and some challenging issues of a wearable affective robot are discussed.



page 1

page 2

page 3

page 5

page 7

page 8

page 9

page 10


Wearable Affective Life-Log System for Understanding Emotion Dynamics in Daily Life

Past research on recognizing human affect has made use of a variety of p...

OpenHealth: Open Source Platform for Wearable Health Monitoring

Movement disorders are becoming one of the leading causes of functional ...

CreativeBioMan: Brain and Body Wearable Computing based Creative Gaming System

Current artificial intelligence (AI) technology is mainly used in ration...

An Overview of Human Activity Recognition Using Wearable Sensors: Healthcare and Artificial Intelligence

With the rapid development of the internet of things (IoT) and artificia...

Preprint Extending Touch-less Interaction on Vision Based Wearable Device

This is the preprint version of our paper on IEEE Virtual Reality Confer...

Beyond Technical Motives: Perceived User Behavior in Abandoning Wearable Health & Wellness Trackers

Health trackers are widely adopted to support individuals with daily hea...

A peristaltic soft, wearable robot for compression and massage therapy

Soft robotics is attractive for wearable applications that require confo...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Artificial Intelligence (AI) is defined as the “machine intelligence imitating human behaviors and cognitive abilities” [1]

. In recent years, the AI has received continuous concerns from both academia and industry, and various countries have invested in the AI-related research and development. The data obtained from China Industrial Economic Information Network (CINIC) show that the global investment in artificial intelligence has been grown from 589 million US dollars in 2012 to more than 5 billion US dollars in 2016. It is estimated that by 2025, the market capitalization of AI applications will reach 127 billion dollars 

[2]. With the continuous development of AI technology, the AI and medical health field have been integrated, which has formed an important interdiscipline (AI-based medical treatment) closely related to the national economy and people’s livelihood [3]. According to the forecasts, by 2025, the AI-based medical treatment industry will occupy one fifth of the market scale. Although the AI and Medical treatment in China has started a little later than in some other countries, it is expected that it will reach 20 billion Chinese Yuan of the market size in China by 2018.

The AI-based medical treatment interdiscipline plays an important role in the development of the artificial intelligent-based human health diagnosis and treatment [4] [5]. In 2011, it was found by the research personnel from the Langone Health of New York University that, the analysis and matching of the pulmonary nodule images (chest CT images) based on the AI are 62% to 97% faster than the manually-annotated ones conducted by the radiologists, which could save up to 3 billion dollars every year. Another research on 379 patients in the Plastic Surgery field showed that in comparison with the independent operations performed by surgeons, the AI robot-aided technology created by Mazor Robotics reduced the surgical complications fivefold, which could reduce 21% of the patient’s post-operation hospital stay, which further could bring less pain and faster recovery to patients, providing more effective guarantee to patient’s healthy life and saving up to 40 billion dollars each year [6].

In comparison with the physiology, the psychology is a more important factor guaranteeing the human health and life happiness. When our environment is short of the experience of stability or belongingness, we may reproduce a pleasant emotion through television, movie, music, book, video game, or any other thing which can provide an immersive social world [7] [8] [9]

. The basic emotions of humans are well-founded, even in the form of the virtual artificial intelligence, such as in a virtual assistant, a traditional service robot, etc. Among them, the virtual assistant uses the Natural Language Processing (NLP) to match the user text or voice input with its executable commands, and continuously studies them by the AI technologies including the machine learning. On the other hand, the traditional service robots (without a virtual assistant), such as sweeping robots and industrial robots, only provide the mechanical services 

[10] [11] [12].

However, humans have a stronger emotional response to the tangible AI [13]. In other words, the more human-like a robot is, the stronger emotional response we will have to it. The intelligent robots are mainly divided into social robots, affective robots, and wearable robots.

  • Combining a traditional service robot with a virtual assistant, a social robot, which has the ability to imitate one or multiple cognitive competencies, such as natural language interaction, is created. The traditional social robot is a type of the social robots. Through a mobile application, it can interact with humans and other robots. A social robot owns a virtual assistant and has the mechanical ability, such as the ability to move its body parts [14] [15] [16]. Current social robots focus on four major applications: health care and treatment, education, public and work environment, and home environment [17] [18] [19]. Although the social robots have the language-based communication ability and can imitate human behaviors, they do not have strong intellectuality, such as cognition of human emotion [20].

  • On the other hand, an affective robot has the function of emotion recognition from the third-person perspective, but it cannot accompany users conveniently because its human-computer interactive mode is limited by the surrounding environment.

  • In recent years, there is the concept of a wearable robot in Cybathlon organized by ETH. However, it actually refers to the exoskeleton of a robot. Currently, its application scope is limited to the disabled people. We believe the application range of a wearable robot should be enlarged to various population.

Fig. 1: The Evolution of Wearable Affective Robot

In general, the research on social affective robots is still in the preliminary stage, and its development difficulties mainly refer to the following aspects.

  • Cost: A social affective robot currently costs about a few hundred to a few thousand dollars, which is difficult to afford.

  • Intelligence: Currently the research on the machine intelligence is still in a non-mature stage, and the cognitive competence of humans has not been achieved yet, so human emotion cannot be recognized accurately.

  • Human-computer interaction interface: The interface is not friendly enough, especially that of the hard robot shell which limits the barrier-free communication between humans and machines.

  • Movability: Users moves in a large range in the physical world, but current social affective robots do not have such a large-range movability.

  • Comfort: Current social robot or robot exoskeleton has a great disturbance to users, and cannot provide the close-fitting and comfortable experience.

  • Service object: At present, the focus is on patients or special groups without the possibility to cover the overall population. These shortcomings limit the use of social emotional robots, so the new-generation robots should have more comprehensive and powerful functions, and their application scope should cover a wider population [21] [22] [23].

Here, we present the problems that must be solved to overcome the challenges arising from the above-mentioned shortcomings.

  • The third-person perspective and first-person perspective should be switched over flexibly.

  • It should be integrated with the user comfortably without affecting the social contact and daily traveling of a user.

  • The personalization must be realized, the in-depth modeling of a user should be accomplished, and current decision-making should be influenced through a long-term cognition process on a user, so as to improve the robot intelligence continuously.

Therefore, in this work, an intelligent affective interaction robot, an intelligent haptic interaction device, and an intelligent brain wearable device are combined to realize a new Fitbot in the form of smart clothes. This is a brand new morphology of a social affective robot, which simulates the thinking process of humans based on a cognitive computing model. Moreover, it involves the utilization of data mining, pattern recognition, natural language processing, and other machine learning techniques to simulate the working mode of a human brain, which allows robots to interact with users in a more complex way. The proposed Fitbot can improve the accuracy of user’s emotion recognition by the acquisition of breadth, long-term, and in-depth data of a single-user, as well as by using the unlabeled learning, emotion recognition, and interactive algorithm. A Fitbot can significantly influence the actual life of human beings from the aspect of spiritual life, emotional care, medical rehabilitation, intelligent company, etc.

The evolution process of Fitbot is presented in Fig. 1, a virtual assistant is a pure software application, while a service robot can move autonomously, but has no language skill. Thus, a service robot can be combined with a virtual assistant forming a social robot with the ability to communicate using the language. However, a social robot is not necessarily intelligent and can only simulate human language. To further develop the intelligence of a social robot, the human feelings must be recognized, which leads to a social affective robot. However, a social affective robot does not have high comfort and personalization, so a Fitbot is proposed in this paper. By introducing a strong AI, a Fitbot is endowed with the ability to cognize the emotion, having the portable and fashionable elements, so as to serve people in a wider range. Moreover, it can improve the people “health” on the spirit level. Therefore, a Fitbot is a social affective robot whose ability to cognize emotions can be improved on its own. Table I gives the performance comparison between the Fitbot, smart clothes, social robot (taking a chatting robot as an example), wearable robot, and affective robot.

Fig. 2: The architecture and constituent parts of a Fitbot
Category Sociality
Usability Washable Accuracy Sustainability
Real-time Consumers
Wearable 2.0 No High Very easy Yes Middle Yes Complex Yes Everyone
Robot 1.0
Simple Middle Easy No Middle Yes No Yes Everyone
Robot 1.0
No Low Hard No Low Yes Simple No Deformity
Robot 1.0
Complex Middle Easy No Middle Yes No Yes Everyone
Robot 2.0
High Very easy Yes High Yes
Yes Everyone
TABLE I: Comparison of different robots

In general, the main contributions of this paper are as follows:

  • A wearable robot having the emotion-cognition ability named the Fitbot is presented. The composition and structure of the presented robot are introduced in detail from the aspect of hardware design and algorithm design.

  • The important constituent part of a Fitbot, the brain wearable device, is introduced in detail. The innovation research in the brain wearable field is introduced from the aspects of hardware design, EGG data acquisition and analysis, user behavior perception, algorithm deployment, etc.

  • Using the cognitive computing, and through the continuous data acquisition in depth and breadth, the life modeling of a user is achieved, which truly realizes a personalized intelligent Fitbot.

  • Finally, the application services and the market prospect of a Fitbot are summarized, and the future challenges are discussed.

The remaining of this paper is organized as follows. In Section II, a Fitbot architecture is introduced. In Section III, the hardware design of a Fitbot is presented. In Section IV, the algorithm design of a Fitbot is explained. In Section V, the process of life modeling is introduced in detail. In Section VI, the applications and services of a Fitbot are discussed. Lastly, in Section VII, the challenges that Fitbot faces are listed, and the paper is concluded.

Ii Fitbot Architecture and Design Issues

Ii-a Fitbot Architecture

As previously mentioned, the proposed Fitbot integrates the intelligent affective interaction robot, intelligent haptic interaction device, and intelligent brain wearable device, and represents a new-type wearable robot with the emotion-cognition ability in the way of smart clothes. The architecture and constituent parts of a Fitbot are shown in Fig. 2.

On the basis of the fashionable clothes commonly worn by people, we have designed an isolation layer for storing the smart terminal (a smartphone) according to the physiological structure of a human body and using the users’ habits, so as to facilitate its physical connection and interaction with other modules such as AIWAC smart box, AIWAC smart tactile device, brain wearable device, AIWAC robot, and cloud platform. The smart terminal can be considered as the edge server of the system [24], which has the function of affective cognitive. Different application scenarios can be realized using a smartphone. In addition, we integrate the design concept of the Wearable 2.0, i.e., the AIWAC smart tactile device. We physically connect the smart tactile device with the AIWAC device and smartphone (via USB I/O mode), which can perform simple control operations and tactile perception [25] [26], so as to realize the tactile human-computer interaction process. Inside the smart clothes, we deploy a Fitbot using the AIWAC smart box as hardware, which can provide the visual and voice interactions with users through the wireless communication of a smartphone and an external MIC device. Due to the physical connection with the smart tactile device via USB I/O mode, a Fitbot possesses rich tactile perception and interactive functions simultaneously. In addition, the AIWAC smart box integrates the local emotion recognition algorithms and has the off-line affective recognition and interaction functions. When there are relatively high requirements for the affective recognition, data access, and service quality, an AIWAC smart box may also skip the smart terminal(edge server) [27] [28], and make direct wireless communication with the cloud platform (equivalent to the system’s remote server) [29] [30] [31], so as to realize more accurate and diversified affective cognitive services. Taking emotion as a communication element to transmit between Fitbot and cloud platform, we offer a better solution to resolve the multidimensional modal emotional communication problems for Fitbot, so as to provide users with personalized emotional service.

To better percept and analyze users’ emotions in a multi-modal way [32], we integrate the intelligent brain wearable devices in the cap of the smart clothes, which can collect the users’ EEG (Electroencephalograph) data. The collected EEG data is transmitted to the background through the wireless communication, so as to realize the perception of users’ subtle expression changes. On the one hand, the detection of these subtle behaviors can ensure perception and analysis of the user’s behavior patterns; on the other hand, some high-efficiency human-computer interaction patterns may be perceived by virtue of these accurate behaviors to achieve good emotional interaction experience.

Ii-B Design Issues

Although a Fitbot can provide customized services for users, which greatly improves the user experience, it also brings many problems such as private information leakage, duration, network reliability, etc.

  • Problem of privacy invasion: Currently, this is the most concerned problem because it harms the users greatly. Since a huge amount of extensive and in-depth data of users is collected, the users’ privacy exposure will be very serious if the data leakage occurs. On the other hand, the organizations such as cybercriminals may continuously monitor the user environment and collect sensitive data of a user. If the user data are acquired by these criminals, it can also cause great harm to the user [33] [34] [35].

  • Problem of duration: Because of the movability, a Fitbot cannot be connected to the power supply continuously. Thus, when the Fitbot battery is used up, the user with a high dependence on a robot will be affected, so the product experience will be seriously impacted. Therefore, the optimization of the energy consumption is a key problem that needs to be solved urgently [36] [37] [38].

  • Problem of network reliability: This problem mainly refers to the reliability of cloud services. In the case of network interruption, the analysis algorithms and other intensive computing dependent on a social robot will not work. A possible solution to this problem is to design a flexible data processing algorithm, which can perform computing locally without a need for network connection [39] [40] [41]

    . Another feasible solution is to develop a machine learning chip and integrate that chip into small devices to realize the data processing on a device (such as the A11 bionic chip used by Apple and the Movidius neuron computing bar used by Intel).

Iii Design of Wearable Affective Robot Hardware

A Fitbot is mainly formed by three types of intelligent hardware: AIWAC smart box, brain wearable device, and AIWAC smart tactile device with the combination of the Wearable 2.0 technology. A Fitbot uses the AIWAC smart box as a hardware core for service processing to execute the data persistence functions such as service data receiving, storage, transmission, etc. A Fitbot also deploys the corresponding affective recognition algorithm relying on the service data. A Fitbot integrates the brain wearable device to achieve the perception of the users’ EEG data, which are transmitted to the cloud platform through the AIWAC smart box for processing and analysis, and the cloud platform realizes the perception of users’ affective state through the analysis of the EGG data. The AIWAC smart tactile device is flexibly integrated into a Fitbot in the PnP mode, and it allows a Fitbot to have the comfortable haptic interaction ability by combining it with the Wearable 2.0 technology, so as to enrich robot’s human-computer interaction interface.

Iii-a AIWAC Smart Box Hardware Design

The AIWAC smart box can make visual and voice interaction, providing the rich tactile perception and interaction functions. In addition, the AIWAC smart box integrates the local emotion recognition algorithms, and has the off-line affective recognition and interaction functions. Relying on the affective recognition and interaction system, by integrating the AIWAC smart box, a Fitbot can have nine types of personality characteristic including brave, steady, sincere, kind-hearted, self-confident, tenacity, forward-looking, and optimistic, and can recognize 21 different human emotions. The AIWAC smart box is the important hardware part for realizing the user emotion pacifying and health regulation, the profile of AIWAC smart box is shown in Fig. 3.

Fig. 3: The AIWAC smart box hardware design

Iii-B Brain Wearable Hardware Design

The cerebral cortex of a human can produce weak electrical signals which contain a large amount of information. However, it is a challenge to collect the weak electrical signals from the cerebral cortex while removing the numerous noise signals in the natural environment. After the profound investigation, as shown in Fig. 4, we design a 1-Channel EEG device for data acquisition, which consists of three electrodes connected in series, the IN1P EEG signal collecting electrode, refer reference electrical signal electrode, and BIAS1 paranoid drive electrode.

Fig. 4: The picture of the EEG device

Iii-C AIWAC Smart Tactile Device

To realize a better interaction of a Fitbot with the users, we have designed a haptic extension interaction using the AIWAC smart tactile device, which is not limited to the traditional voice interaction mode. We design the PnP mode to integrate the AIWAC smart tactile device into a Fitbot or other intelligent devices, so as to carry out the basic interactive control operations by using the AIWAC smart tactile device, and further achieve the convenient human-computer interaction using a Fitbot.

Iii-D Wearable Affective Robot prototype

Fig. 5: The prototype of a wearable affective robot

A Fitbot integrates the AIWAC smart box, AIWAC smart tactile device, and AIWAC brain wearable device, and its prototype is shown in Fig.  5, where position 1 denotes the voice interaction module of the AIWAC smart box, position 2 denotes the hardware core of the AIWAC smart box, and position 3 denotes the AIWAC smart tactile device integrated into the Fitbot. As shown in position 5, the brain wearable device is integrated into the cap position of a Fitbot, and its mainboard is connected with the rest of a Fitbot via USB communication, which is shown in position 6. A Fitbot has the rich means of perception and interaction, such as voice perception, tactile perception, EEG perception, and etc. On this basis, it achieves the multi-modal perception and analysis of users’ emotions and achieves good affective interaction experience.

We have built a testbed platform of Fitbot to validate emotional communication mechanism, which includes smart phone terminal(edge server), Fitbot, and remote cloud with large data centers. The emotional data collected in the experiment are mainly voice emotion data, and Fitbot has a voice acquisition module. After the robot has collected a large number of users’ emotional data, the robot transmits them to the edge server to be processed and annotated as unlabeled data, and then AI algorithm for emotion recognition is achieved in the remote cloud. According to the result of emotion recognition, as shown in Fig. 6, robot and user interact with each other. Due to the dynamic change of the user’s emotional state, the real-time emotional interaction is necessary, which requires that the communication link must have ultra-low latency. Another key element of emotional communication is that the AI emotion analysis algorithm deployed on the remote cloud(or in Fitbot) must have a high accuracy rate of user emotion recognition.

Fig. 6: Results of Emotion Recognition

Iv Wearable Affective Robot Algorithm

Iv-a Affective Recognition Algorithm based on AIWAC Smart Box

The AIWAC smart box in a Fitbot has the ability of accurate voice affective cognition. Through daily voice interactions with a user, a Fitbot collects user’s voice data and realizes voice affective analysis by using the attention based recurrent neural network algorithm. A recurrent neural network (RNN) can effectively remember the relevant feature information in a long context. By introducing an attention mechanism into the RNN algorithm framework, a new weight pooling strategy is introduced into the network, which puts special attention on the part expressing strong affective characteristics of the voice. The RNN architecture is shown in Fig. 


Fig. 7: The RNN algorithm for Fitbot Emotion Sensing

Iv-B User Behavior Perception Based on Brain Wearable Device

The brain wearable device can realize the perception of user’s subtle expression change. On the one hand, the detection of these subtle behaviors enables the perception and analysis of user’s behavior patterns; on the other hand, some high-efficiency human-computer interaction patterns may be perceived by virtue of these accurate behaviors.

The perception of user behaviors by a brain wearable device can be introduced by a blink detection example. We design an algorithm to realize such a detection.

Algorithm: Realization of blink detection based on the amplitude difference. The general process of Algorithm 1 is to determine the first order difference in the original EGG signal, then, the amplitude smoothing is performed, and lastly, the judgment on the smoothing result is conducted. The Algorithm 1 uses the time-domain data and draws the signal waveform diagram as shown in Fig. 8, wherein it can be seen that such a signal experiences a significant amplitude change during the blinking, which can be directly used to judge whether a user blinks or not. First, the first order difference in the original signal is determined to obtain the change rate information of the time-domain signal. To amplify the peak feature, the amplitude smoothing is used, i.e., the amplitude values lower than 150 are set to zero. By comparing the original signal with the result of the actual blink test and discrimination, it can be seen that in 20 blink tests, in 17 tests the blinks were detected accurately, as shown in Fig. 9.

Fig. 8: Time-domain EEG Signal
Fig. 9: Comparison between the actual blinks and detection results
Scenes Items Description
Home Smart Home Fitbot can accurately recognize the emotion of the people in an enclosed environment, so as to prevent the mental anomaly and safety problem under no supervision, which is like a butler tracking and accompanying a user 24/7.
Stay-at-home children
Empty-nest elderly
Patient living alone
Hospital Outpatient clinic, emergency treatment Fitbot reduces the occurrence of sudden diseases or medical troubles by recognizing and effectively intervening the emotions of patients and timely notifying the doctors and patient families. Fitbot is a doctor’s intelligent assistant and the patient’s guardian angel who not only can create a more humanized medical environment but also can reduce the tension between doctors and patients.
Operating Room
Inpatient Department
Education System Monitoring in the examination rooms Fitbot can accurately recognize the emotions of students, improving the education and teaching efficiency and reducing the occurrence of accidental injuries in school. Fitbot aims to realize three major purposes: expanding the education opportunities, improving the education quality, and reducing the education cost. It realizes the in-depth integration of information technology and education and teaching, forming a new talent-training mode which is multi-media, interactive, personalized, self-adaptive, and learner-centered.
Campus security monitoring
Remote teaching
Other public place Tourist Areas Aiming at other high-risk industries, crowd-intensive places, and areas with high incidence of crimes, in combination with the video big data collected by the security monitoring system, a Fitbot provides an intelligent emotion recognition algorithm and an intervention strategy to improve the security efficiency and reduce the occurrence of accidents. Good environment security management of the public places represents the civilization degree of a city or even a country.
Chain stores
Entertainment and business places
TABLE II: Fitbot Application Service Scenarios

V Life Modeling of Wearable Affective Robot

Through continuous in-depth and in-breadth data collection, a Fitbot builds the life modeling for users and conducts the long-term analysis and modeling of users’ behaviors and emotions, which makes Fitbot have a deeper cognitive level on users and endows Fitbot with better user’s experience of affective social interaction. The process of life modeling from three aspects is introduced below.

V-a Dataset Labeling and Processing

In the process of data collection, the labeled data such as the user’s physical examination reports and doctor’s diagnostic records cannot be obtained in most cases. Therefore, a large amount of unlabeled data is collected in the unconscious interaction between the user and system. These data should be processed without disturbing a user.

After Fitbot collects the breadth and depth data of the user, it uses AI emotional communication to pass on data to the edge cloud for labeling and processing of data set [42] [43], and then it further transmits to the distal cloud and uses AI technology to have modeling analysis of the life modeling of the user, and feeds back to the user, so as to improve its cognition to the user’s life modeling.

In the case of a small amount of data, the mathematical modeling is used to judge whether the unlabeled data should be added to the dataset. The labelless learning is used to decide whether the unlabeled data are added to the dataset or not based on the similarity measurement and taking into account the effect of data on the dataset after such data are added to the dataset. If such data influence positive to the entire dataset, the system will consider adding such data to the dataset. In addition, we also needs to consider the purity of the data. With the aim to improve the overall purity of the dataset, some unreliable data must be excluded because the ambiguous and low-value data will cause the spread of the error.

V-B Multi-dimensional Data Integration Modeling

By using the hardware, the embedded control, the cloud platform of Big Data, and various deep learning algorithms, the various data are collected continuously through the IoT (Internet of Things) terminal devices, including the affective interaction robots, mobile phones (Android and iOS applications), etc. Through users’ interaction with a social robot and his playing the affective cognitive games on a mobile phone, a lot of user data can be collected, including the pictures, environment background, voice, and text description of things to be done.

The text data are processed by the convolutional neural network, and the deep network is established mainly to extract the features from such unstructured data. In the case of the structured data, it is relatively easy to use the machine learning method directly, integrating multiple machine learning models including the decision-making tree, random forest, clustering algorithm, and others to obtain the robust models. These models can be used to predict users’ emotions, preferences, and behaviors.

V-C Modeling of Associated Data Scenarios

Since the dimension of user data is high and diversified, the inference of user data is not a simple one-to-one inference. There are many classes of image data, such as identity data. Therefore, after the person identity is determined, it is also needed to make a further inference from the data of the surrounding environment of a user. The data such as “who”, “where”, and “when”, and even the sports data, denote the elements needed for inference. Since the information data are associated with the emotion in other aspects, the emotion cannot be inferred separately. Instead, the emotion should be inferred according to the information data in various aspects of a circumstance, and the reason for giving rise to such emotion should be inferred in a high degree of confidence. Therefore, in the modeling of associated data scenario, according to the behavioral fragments of a user, the events associated with emotion are extracted, and their associations with similar users and with other different users are explored.

In modeling, the key elements of multi-modal unstructured data are first extracted through the simplified mode. At this point, the emotions of a user are labeled in a multi-dimensional and multi-perspective way. Then, for the same user, many other users in different time periods and locations are labeled, and then the similarity of these users can be further explored.

The modeling of the associated data scenario not only can provide simple affective classification, but also can realize more complex scenario analysis to analyze the user’s mental experiences, including the change and transformation of the user’s emotional state in a certain situation, which makes a Fitbot more intelligent.

Vi Application

Vi-a Characteristics of Wearable Affective Robot

  • Intelligence and Autonomy: A real intelligent robot should have the intelligence and autonomy and should be able to refuse an unhealthy or an immoral request from a user. For instance, if a user requests from an affective robot to buy a junk food, the affective robot should refuse that request and explain the harmful effects resulting from the consumption of such a junk food, which will promote the establishment of trust between the affective robot and user, and improve the health surveillance effect on the user.

  • Comfort and Portability: The entity robot other than the virtual robot basically has limited movement ability, so it can accompany user with a space limitation. Its operation scenes are limited, and a user is often required to spend extra care to carry or maintain such an interactive robot. By virtue of the Wearable 2.0, a Fitbot is integrated into a wearable device in an natural way. The robot accompanies a user as a part of user’s clothes, providing good portability to the robot. Meanwhile, a Fitbot provides convenient and various human-computer interaction modes, which make the user have good comfort.

  • Diversified Human-Computer Interaction: Current social robots are mainly based on voice interaction, while a Fitbot conveniently integrates tactile interaction on the basis of voice interaction, which expands the form of human-computer interaction, and provides available human-computer interaction means for some specific groups (such as the deaf-mute).

  • Context Awareness and Personalized Tracking Service: The collection of context data is conducive to the personalized services for a user. Generally, users having the social affective robots are also equipped with a smartphone. The context awareness data collected by a smartphone is closely related to a user and applies only to himself. A Fitbot can collect the context awareness data from the third-person (spectator) perspective, and can achieve more accurate identification of user activities. Such user data can be shared with multiple users such as family members or friends who are in the same room.

In addition, an affective robot can collect data from the first-person perspective and communicate with others on behalf of a user. Moreover, it can present the user’s emotions in the form of a voice, a text, or an image, and convey others’ information so as to make the interactions.

Vi-B Application Range

On the basis of the affective recognition and interaction, a Fitbot imitates human behaviors and interacts with users through the voice interface. As the understanding on a user becomes deeper, a Fitbot will grow with the user and provide a user with a series of services. In Table II, the application services provided by a Fitbot in four fields (home environment, health care, education, and public places) are listed.

Vii Conclusion

In this paper, a wearable affective robot equipped with cognitive computing named Fitbot is proposed. Joining the AIWAC smart box, smart tactile device, and brain wearable device, the complete system of a Fitbot is built. A Fitbot can realize the multi-modal data perception. Based on the perceived multi-modal data of a user, a Fitbot can cognize the user’s emotions. Human emotions denote the external manifestation of a user’s intentions, which enables a Fitbot to understand the behavioral motivations that are behind the user’s emotions. Finally, the application service scenarios and some open questions of a Fitbot are discussed.


This work was supported by the National Natural Science Foundation of China (under Grant No. 61875064, U1705261, 61572220), the National Key R&D Program of China (2018YFC1314600), Director Fund of WNLO, the Fundamental Research Funds for the Central Universities (HUST: 2018KFYXKJC045), the National 1000 Talents Program, China, the Hubei Provincial Key Project under grant 2017CFA051, and the Applied Basic Research Program through Wuhan Science and Technology Bureau under Grant 2017010201010118.


  • [1] Cisco Visual Networking Index: Global mobile data traffic forecast update,2014-2019, Feb. 2014.
  • [2] M. Chen, Y. Hao, M. Qiu, and et al., “Mobility-aware caching and computation offloading in 5g ultra-dense cellular networks,” Sensors, vol. 16, no. 7, pp.974-987, 2016.
  • [3] M. Chen, J. Yang, Y. Hao, and et al., “A 5G Cognitive System for Healthcare,” Big Data and Cognitive Computing, Vol. 1, No. 1, DOI:10.3390/bdcc1010002, 2017.
  • [4] Y. Zheng, D. Wu, Y. Ke, and et al., “Online Cloud Transcoding and Distribution for Crowdsourced Live Game Video Streaming,” In IEEE Transactions on Circuits and Systems for Video Technology (IEEE TCSVT), vol. 27, no. 8, pp. 1777-1789, 2017.
  • [5] M. Chen, W. Li, Y. Hao, and et al., “Edge Cognitive Computing based Smart Healthcare System,” Future Generation Computing System, vol. 86, pp. 403-411, 2018.
  • [6] M. Chen, Y. Zhang, M. Qiu, and et al., “SPHA: Smart Personal Health Advisor Based on Deep Analytics,” IEEE Communications, vol. 56, no. 3, pp. 164-169, 2018.
  • [7] H. Ghasemzadeh, P. Panuccio, S. Trovato, and et al., “Power-Aware Activity Monitoring Using Distributed Wearable Sensors,” IEEE Trans. Human-Machine Systems, vol. 44, no. 4, pp.537-544, 2014.
  • [8] Y. Zhang, R. Yu, M. Nekovee, and et al., “Cognitive Machine-to-Machine Communications: Visions and Potentials for the Smart Grid,” IEEE Network Magazine, vol.26, no.3, pp.6-13, 2012.
  • [9] X. Ge, Y. Sun, H. Gharavi, and et al., “Joint Optimization of Computation and Communication Power in Multi-user Massive MIMO Systems,” IEEE Transactions on Wireless Communications, vol. 17, no. 6, pp. 4051-4063, 2018.
  • [10] L. Lei, Y. Kuang, N. Cheng, and et al., “Delay-optimal dynamic mode selection and resource allocation in device-to-device communications Part I: Optimal policy,” IEEE Transactions on Vehicular Technology, vol. 65, no. 5, pp. 3474-3490, 2016.
  • [11] B. Yang, G. Mao, M. Ding, and et al., “Dense Small Cell Networks: From Noise-Limited to Dense Interference-Limited,” IEEE Transactions on Vehicular Technology, vol. 67, no. 5, pp. 4262-4277, 2018.
  • [12] K. Zheng, L. Hou, H. Meng, and et al., “Soft-defined heterogeneous vehicular network: architecture and challenges,” IEEE Network, vol. 30, no. 4, pp. 72-80, 2016.
  • [13] M. Chen, P. Zhou, G. Fortino, “Emotion Communication System,” IEEE Access, vol 5, pp.326-337, 2017.
  • [14] Y. Zhou, X. Gu, D. Wu, and et al., “Statistical Study of View Preferences for Online Videos with Cross-platform Information,” In IEEE Transactions on Multimedia (IEEE TMM), vol. 20, no 6, 1512-1524, 2018.
  • [15] S. Baker, W. Xiang, and I. Atkinson, “Internet of Things for smart healthcare: Technologies, challenges and opportunities,” IEEE Access, vol. 5, pp.26521-26544, 2017.
  • [16] W. Xiang, Y. Zhou, and N. Wang, “An energy-efficient routing algorithm for software-defined wireless sensor network,” IEEE Sensors Journal, vol. 16, no. 20, pp.7393-7400, 2016.
  • [17] Y. Zhang, R. Yu, W. Yao, and et al., “Home M2M Networks: Architectures, Standards, and QoS Improvement,” IEEE Communications Magazine, vol.49, no.4, pp.44-52, 2011.
  • [18] H. Jiang, C. Cai, X. Ma, and et al., “Smart Home Based on WiFi Sensing: A Survey,” IEEE Access, vol. 6, pp.13317-13325, 2018.
  • [19] M. Chen , J. Yang , X. Zhu, and et al., “Smart Home 2.0: Innovative Smart Home System Powered by Botanical IoT and Emotion Detection,” Mobile Networks and Applications, vol. 22, pp. 1159-1169, 2017.
  • [20] R. Gravina, G. Fortino, “Automatic Methods for the Detection of Accelerative Cardiac Defense Response,” IEEE Trans. Affective Computing, vol. 7, no. 3, pp.286-298, 2016.
  • [21] K. Zheng, H. Meng, P. Chatzimisios, and et al., “An SMDP-based resource allocation in vehicular cloud computing systems,” IEEE Transactions on Industrial Electronics, vol. 62, no. 12, pp. 7920-7928, 2015.
  • [22] G. Fortino, A. Guerrieri, F. Bellifemine, and et al., “SPINE2: developing BSN applications on heterogeneous sensor nodes,” SIES 2009, pp. 128-131, 2009.
  • [23] R. Gravina, C. Ma, P. Pace, and et al., “Cloud-based Activity-aaService cyber-physical framework for human activity monitoring in mobility,” Future Generation Comp. Syst, vol. 75, pp.158-171, 2017.
  • [24] X. Chen, L. Jiao, W. Li, and X. Fu, “Efficient multi-user computation offloading for mobile-edge cloud computing,” IEEE/ACM Transactions on Networking, vol. 24, no. 5, 2016.
  • [25] G. Tao, A.M. Stolyarov, and A. F. Abouraddy, “Multimaterial fibers,” I. J. Appl. Glass Science, vol. 3, pp. 349-368, 2012.
  • [26] G. Tao, J. J. Kaufman, S. Shabahang, and et al., “Digital design of multimaterial photonic particles,” Proc. Natl. Acad. Sci. USA, vol. 113, no. 25, pp.6839-6844, 2016.
  • [27] C. Wang, Y. Li, and D. Jin, “Mobility-assisted opportunistic computation offloading,” IEEE Communications Letters, vol. 18, no. 10, pp. 1779-1782, 2014.
  • [28] D. Wang, Y. Peng, X. Ma, and et al., “Adaptive Wireless Video Streaming based on Edge Computing: Opportunities and Approaches,” IEEE Transactions on Services Computing, doi: 10.1109/TSC.2018.2828426, 2018.
  • [29] M. Chen, Y. Miao, Y. Hao, and et al., “Narrow Band Internet of Things,” IEEE Access, vol. 5, pp. 20557-20577, 2017.
  • [30] M. Chen, Y. Tian, G. Fortino, and et al., “Cognitive Internet of Vehicles,” Computer Communications, vol. 120, pp. 58-70, 2018.
  • [31] Y. Zhang, M. Chen , N. Guizani, and et al., “SOVCAN: Safety-Oriented Vehicular Controller Area Network,” IEEE Communications, vol. 55, no. 8, pp. 94-99, 2017.
  • [32] M. Chen, X. Shi, Y. Zhang, and et al., “Deep Features Learning for Medical Image Analysis with Convolutional Autoencoder Neural Network,” IEEE Trans. Big Data, DOI:10.1109/TBDATA.2017.2717439, 2017.
  • [33] F. Cicirelli, G. Fortino, A. Giordano, and et al., “On the Design of Smart Homes: A Framework for Activity Recognition in Home Environment,” J. Medical Systems, vol. 40, no. 9, pp.2001-2017, 2016.
  • [34] P. Zhao, J. Li, F. Zeng, and et al., “ILLIA: Enabling -Anonymity-Based Privacy Preserving Against Location Injection Attacks in Continuous LBS Queries,” IEEE Internet of Things Journal, vol. 5, no. 2, pp.1033-1042, 2018.
  • [35] M. Chen, Y. Hao, Y. Li, and et al., “On the computation offloading at ad hoc cloudlet: architecture and service modes,” IEEE Communications Magazine, vol. 53, no. 6, pp.18-24, 2015.
  • [36] G. Xue, Q. Chen, S. Lin, and et al., “Highly Efficient Water Harvesting with Optimized Solar Thermal Membrane Distillation Device,” advanced science news, vol. 2, no. 5, pp.1-1, 2018.
  • [37] M. Chen, Y. Hao, L. Hu, and et al., “Edge-CoCaCo: Towards Joint Optimization of Computation, Caching and Communication on Edge Cloud,” IEEE Wireless Communications, vol. 25, no. 3, pp. 21-27, 2018.
  • [38] S. Maharjan, Q. Zhu, Y. Zhang, and et al., “Dependable Demand Response Management in the Smart Grid: A Stackelberg Game Approach,” IEEE Transactions on Smart Grid, vol.4, no.1, pp.120-132, 2013.
  • [39] S. Zhang, N. Zhang, S. Zhou, and et al., “Energyaware traffic offloading for green heterogeneous networks,” IEEE Journal on Selected Areas in Communications, vol. 34, no. 5, pp.1116-1129, 2016.
  • [40] M. Chen, Y. Hao, “Task Offloading for Mobile Edge Computing in Software Defined Ultra-dense Network,” IEEE Journal on Selected Areas in Communications, vol. 36, no. 3, pp. 587-597, 2018.
  • [41] Y. Liu, D. Niu, and B. Li, “Delay-optimized video traffic routing in software-defined interdatacenter networks,” IEEE Transactions on Multimedia, vol. 18, no. 5, pp.865-878, 2016.
  • [42] T. Touzri, M. B. Ghorbel, B. Hamdaoui, and et al., “Efficient usage of renewable energy in communication systems using dynamic spectrum allocation and collaborative hybrid powering,” IEEE Transactions on Wireless Communications, vol. 15, no. 5, pp.3327-3338, 2016.
  • [43] J. Duan, W. Xie, P. Yang, and et al., “Tough hydrogel diodes with tunable interfacial adhesion for safe and durable wearable batteries,” Nano Energy, vol. 48, pp.569-574, 2018.