SwarmPaint: Human-Swarm Interaction for Trajectory Generation and Formation Control by DNN-based Gesture Interface

by   Valerii Serpiva, et al.

Teleoperation tasks with multi-agent systems have a high potential in supporting human-swarm collaborative teams in exploration and rescue operations. However, it requires an intuitive and adaptive control approach to ensure swarm stability in a cluttered and dynamically shifting environment. We propose a novel human-swarm interaction system, allowing the user to control swarm position and formation by either direct hand motion or by trajectory drawing with a hand gesture interface based on the DNN gesture recognition. The key technology of the SwarmPaint is the user's ability to perform various tasks with the swarm without additional devices by switching between interaction modes. Two types of interaction were proposed and developed to adjust a swarm behavior: free-form trajectory generation control and shaped formation control. Two preliminary user studies were conducted to explore user's performance and subjective experience from human-swarm interaction through the developed control modes. The experimental results revealed a sufficient accuracy in the trajectory tracing task (mean error of 5.6 cm by gesture draw and 3.1 cm by mouse draw with the pattern of dimension 1 m by 1 m) over three evaluated trajectory patterns and up to 7.3 cm accuracy in targeting task with two target patterns of 1 m achieved by SwarmPaint interface. Moreover, the participants evaluated the trajectory drawing interface as more intuitive (12.9 requiring less effort to utilize (22.7 by gestures, although its physical workload and failure in performance were presumed as more significant (by 9.1



There are no comments yet.


page 1

page 3

page 4

page 5


DronePaint: Swarm Light Painting with DNN-based Gesture Recognition

We propose a novel human-swarm interaction system, allowing the user to ...

Gesture based Human-Swarm Interactions for Formation Control using interpreters

We propose a novel Human-Swarm Interaction (HSI) framework which enables...

SwarmTouch: Tactile Interaction of Human with Impedance Controlled Swarm of Nano-Quadrotors

We propose a novel interaction strategy for a human-swarm communication ...

Towards Decentralized Human-Swarm Interaction by Means of Sequential Hand Gesture Recognition

In this work, we present preliminary work on a novel method for Human-Sw...

A wearable general-purpose solution for Human-Swarm Interaction

Swarms of robots will revolutionize many industrial applications, from t...

Mouse Simulation Using Two Coloured Tapes

In this paper, we present a novel approach for Human Computer Interactio...

User interface for in-vehicle systems with on-wheel finger spreading gestures and head-up displays

Interacting with an in-vehicle system through a central console is known...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.