Exploiting Natural Language for Efficient Risk-Aware Multi-robot SaR Planning

04/08/2021
by   Vikram Shree, et al.
0

The ability to develop a high-level understanding of a scene, such as perceiving danger levels, can prove valuable in planning multi-robot search and rescue (SaR) missions. In this work, we propose to uniquely leverage natural language descriptions from the mission commander in chief and image data captured by robots to estimate scene danger. Given a description and an image, a state-of-the-art deep neural network is used to assess a corresponding similarity score, which is then converted into a probabilistic distribution of danger levels. Because commonly used visio-linguistic datasets do not represent SaR missions well, we collect a large-scale image-description dataset from synthetic images taken from realistic disaster scenes and use it to train our machine learning model. A risk-aware variant of the Multi-robot Efficient Search Path Planning (MESPP) problem is then formulated to use the danger estimates in order to account for high-risk locations in the environment when planning the searchers' paths. The problem is solved via a distributed approach based on Mixed-Integer Linear Programming. Our experiments demonstrate that our framework allows to plan safer yet highly successful search missions, abiding to the two most important aspects of SaR missions: to ensure both searchers' and victim safety.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 8

11/25/2020

Mixed-Integer Linear Programming Models for Multi-Robot Non-Adversarial Search

In this letter, we consider the Multi-Robot Efficient Search Path Planni...
07/21/2020

Balance Scene Learning Mechanism for Offshore and Inshore Ship Detection in SAR Images

This letter proposes a novel Balance Scene Learning Mechanism (BSLM) for...
02/01/2021

Exploiting multi-temporal information for improved speckle reduction of Sentinel-1 SAR images by deep learning

Deep learning approaches show unprecedented results for speckle reductio...
03/25/2020

Risk-Aware Planning and Assignment for Ground Vehicles using Uncertain Perception from Aerial Vehicles

We propose a risk-aware framework for multi-robot, multi-demand assignme...
08/28/2020

Collaborative Multi-Robot Systems for Search and Rescue: Coordination and Perception

Autonomous or teleoperated robots have been playing increasingly importa...
03/23/2018

Counterexamples for Robotic Planning Explained in Structured Language

Automated techniques such as model checking have been used to verify mod...
05/10/2021

An Autonomous Drone for Search and Rescue in Forests using Airborne Optical Sectioning

Drones will play an essential role in human-machine teaming in future se...

Code Repositories

DISC-L

Image-description dataset for multimodal scene perception.


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.