Adaptive Autonomy in Human-on-the-Loop Vision-Based Robotics Systems

03/28/2021
by   Sophia Abraham, et al.
5

Computer vision approaches are widely used by autonomous robotic systems to sense the world around them and to guide their decision making as they perform diverse tasks such as collision avoidance, search and rescue, and object manipulation. High accuracy is critical, particularly for Human-on-the-loop (HoTL) systems where decisions are made autonomously by the system, and humans play only a supervisory role. Failures of the vision model can lead to erroneous decisions with potentially life or death consequences. In this paper, we propose a solution based upon adaptive autonomy levels, whereby the system detects loss of reliability of these models and responds by temporarily lowering its own autonomy levels and increasing engagement of the human in the decision-making process. Our solution is applicable for vision-based tasks in which humans have time to react and provide guidance. When implemented, our approach would estimate the reliability of the vision task by considering uncertainty in its model, and by performing covariate analysis to determine when the current operating environment is ill-matched to the model's training data. We provide examples from DroneResponse, in which small Unmanned Aerial Systems are deployed for Emergency Response missions, and show how the vision model's reliability would be used in addition to confidence scores to drive and specify the behavior and adaptation of the system's autonomy. This workshop paper outlines our proposed approach and describes open challenges at the intersection of Computer Vision and Software Engineering for the safe and reliable deployment of vision models in the decision making of autonomous systems.

READ FULL TEXT

page 1

page 2

page 4

page 5

research
10/04/2021

Emotionally-Informed Decisions: Bringing Gut's Feelings into Self-adaptive and Co-adaptive Software Systems

Software systems now complement an incredibly vast number of human activ...
research
08/18/2019

Long-Duration Fully Autonomous Operation of Rotorcraft Unmanned Aerial Systems for Remote-Sensing Data Acquisition

Recent applications of unmanned aerial systems (UAS) to precision agricu...
research
05/31/2023

Designing Closed-Loop Models for Task Allocation

Automatically assigning tasks to people is challenging because human per...
research
11/27/2022

Attribution-based XAI Methods in Computer Vision: A Review

The advancements in deep learning-based methods for visual perception ta...
research
09/20/2022

Evaluation Framework for Computer Vision-Based Guidance of the Visually Impaired

Visually impaired persons have significant problems in their everyday mo...
research
09/18/2023

NOMAD: A Natural, Occluded, Multi-scale Aerial Dataset, for Emergency Response Scenarios

With the increasing reliance on small Unmanned Aerial Systems (sUAS) for...
research
03/28/2018

Making Sense of the World: Models for Reliable Sensor-Driven Systems

Sensor-driven systems are increasingly ubiquitous: they provide both dat...

Please sign up or login with your details

Forgot password? Click here to reset