DeepAI AI Chat
Log In Sign Up

Enhancing Human-in-the-Loop Adaptive Systems through Digital Twins and VR Interfaces

by   Enes Yigitbas, et al.

Self-adaptation approaches usually rely on closed-loop controllers that avoid human intervention from adaptation. While such fully automated approaches have proven successful in many application domains, there are situations where human involvement in the adaptation process is beneficial or even necessary. For such "human-in-the-loop" adaptive systems, two major challenges, namely transparency and controllability, have to be addressed to include the human in the self-adaptation loop. Transparency means that relevant context information about the adaptive systems and its context is represented based on a digital twin enabling the human an immersive and realistic view. Concerning controllability, the decision-making and adaptation operations should be managed in a natural and interactive way. As existing human-in-the-loop adaptation approaches do not fully cover these aspects, we investigate alternative human-in-the-loop strategies by using a combination of digital twins and virtual reality (VR) interfaces. Based on the concept of the digital twin, we represent a self-adaptive system and its respective context in a virtual environment. With the help of a VR interface, we support an immersive and realistic human involvement in the self-adaptation loop by mirroring the physical entities of the real world to the VR interface. For integrating the human in the decision-making and adaptation process, we have implemented and analyzed two different human-in-the-loop strategies in VR: a procedural control where the human can control the decision making-process and adaptations through VR interactions (human-controlled) and a declarative control where the human specifies the goal state and the configuration is delegated to an AI planner (mixed-initiative). We illustrate and evaluate our approach based on an autonomic robot system that is accessible and controlled through a VR interface.


page 1

page 5

page 6


Semi-Autonomous Planning and Visualization in Virtual Reality

Virtual reality (VR) interfaces for robots provide a three-dimensional (...

Sonic Interactions in Virtual Environments: the Egocentric Audio Perspective of the Digital Twin

The relationships between the listener, physical world and virtual envir...

Self-Adaptive Digital Assistance Systems for Work 4.0

In the era of digital transformation, new technological foundations and ...

3D Reconstruction of Sculptures from Single Images via Unsupervised Domain Adaptation on Implicit Models

Acquiring the virtual equivalent of exhibits, such as sculptures, in vir...

Improving the Cybersecurity of Critical National Infrastructure using Modelling and Simulation

The UK Critical National Infrastructure is critically dependent on digit...

Transparency's Influence on Human-Collective Interactions

Collective robotic systems are biologically inspired and advantageous du...