Let me join you! Real-time F-formation recognition by a socially aware robot

by   Hrishav Bakul Barua, et al.

This paper presents a novel architecture to detect social groups in real-time from a continuous image stream of an ego-vision camera. F-formation defines social orientations in space where two or more person tends to communicate in a social place. Thus, essentially, we detect F-formations in social gatherings such as meetings, discussions, etc. and predict the robot's approach angle if it wants to join the social group. Additionally, we also detect outliers, i.e., the persons who are not part of the group under consideration. Our proposed pipeline consists of – a) a skeletal key points estimator (a total of 17) for the detected human in the scene, b) a learning model (using a feature vector based on the skeletal points) using CRF to detect groups of people and outlier person in a scene, and c) a separate learning model using a multi-class Support Vector Machine (SVM) to predict the exact F-formation of the group of people in the current scene and the angle of approach for the viewing robot. The system is evaluated using two data-sets. The results show that the group and outlier detection in a scene using our method establishes an accuracy of 91 made rigorous comparisons of our systems with a state-of-the-art F-formation detection system and found that it outperforms the state-of-the-art by 29 formation detection and 55 approach angle.



page 1

page 6


F-formation Detection: Individuating Free-standing Conversational Groups in Images

Detection of groups of interacting people is a very interesting and usef...

Outlier Cluster Formation in Spectral Clustering

Outlier detection and cluster number estimation is an important issue fo...

Social nucleation: Group formation as a phase transition

The spontaneous formation and subsequent growth, dissolution, merger and...

Self-supervised Social Relation Representation for Human Group Detection

Human group detection, which splits crowd of people into groups, is an i...

Automatic Group Cohesiveness Detection With Multi-modal Features

Group cohesiveness is a compelling and often studied composition in grou...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.