Log In Sign Up

Personalized Machine Learning for Robot Perception of Affect and Engagement in Autism Therapy

by   Ognjen Rudovic, et al.

Robots have great potential to facilitate future therapies for children on the autism spectrum. However, existing robots lack the ability to automatically perceive and respond to human affect, which is necessary for establishing and maintaining engaging interactions. Moreover, their inference challenge is made harder by the fact that many individuals with autism have atypical and unusually diverse styles of expressing their affective-cognitive states. To tackle the heterogeneity in behavioral cues of children with autism, we use the latest advances in deep learning to formulate a personalized machine learning (ML) framework for automatic perception of the childrens affective states and engagement during robot-assisted autism therapy. The key to our approach is a novel shift from the traditional ML paradigm - instead of using 'one-size-fits-all' ML models, our personalized ML framework is optimized for each child by leveraging relevant contextual information (demographics and behavioral assessment scores) and individual characteristics of each child. We designed and evaluated this framework using a dataset of multi-modal audio, video and autonomic physiology data of 35 children with autism (age 3-13) and from 2 cultures (Asia and Europe), participating in a 25-minute child-robot interaction ( 500k datapoints). Our experiments confirm the feasibility of the robot perception of affect and engagement, showing clear improvements due to the model personalization. The proposed approach has potential to improve existing therapies for autism by offering more efficient monitoring and summarization of the therapy progress.


page 6

page 8

page 10

page 13

page 23


A Two-stage Multi-modal Affect Analysis Framework for Children with Autism Spectrum Disorder

Autism spectrum disorder (ASD) is a developmental disorder that influenc...

Personalizing User Engagement Dynamics in a Non-Verbal Communication Game for Cerebral Palsy

Children and adults with cerebral palsy (CP) can have involuntary upper ...

A Conceptual Framework for Using Machine Learning to Support Child Welfare Decisions

Human services systems make key decisions that impact individuals in the...

A Deep Learning Approach for Multi-View Engagement Estimation of Children in a Child-Robot Joint Attention task

In this work we tackle the problem of child engagement estimation while ...

Detecting Behavioral Engagement of Students in the Wild Based on Contextual and Visual Data

To investigate the detection of students' behavioral engagement (On-Task...