PlanT: Explainable Planning Transformers via Object-Level Representations

10/25/2022
by   Katrin Renz, et al.
0

Planning an optimal route in a complex environment requires efficient reasoning about the surrounding scene. While human drivers prioritize important objects and ignore details not relevant to the decision, learning-based planners typically extract features from dense, high-dimensional grid representations containing all vehicle and road context information. In this paper, we propose PlanT, a novel approach for planning in the context of self-driving that uses a standard transformer architecture. PlanT is based on imitation learning with a compact object-level input representation. On the Longest6 benchmark for CARLA, PlanT outperforms all prior methods (matching the driving score of the expert) while being 5.3x faster than equivalent pixel-based planning baselines during inference. Combining PlanT with an off-the-shelf perception module provides a sensor-based driving system that is more than 10 points better in terms of driving score than the existing state of the art. Furthermore, we propose an evaluation protocol to quantify the ability of planners to identify relevant objects, providing insights regarding their decision-making. Our results indicate that PlanT can focus on the most relevant object in the scene, even when this object is geometrically distant.

READ FULL TEXT

page 2

page 8

research
05/31/2022

TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving

How should we integrate representations from complementary sensors for a...
research
08/02/2023

Interpretable End-to-End Driving Model for Implicit Scene Understanding

Driving scene understanding is to obtain comprehensive scene information...
research
09/09/2021

NEAT: Neural Attention Fields for End-to-End Autonomous Driving

Efficient reasoning about the semantic, spatial, and temporal structure ...
research
06/16/2023

Efficient Search and Detection of Relevant Plant Parts using Semantics-Aware Active Vision

To automate harvesting and de-leafing of tomato plants using robots, it ...
research
08/30/2019

Context Aware Road-user Importance Estimation (iCARE)

Road-users are a critical part of decision-making for both self-driving ...
research
06/19/2018

Translating MFM into FOL: towards plant operation planning

This paper proposes a method to translate multilevel flow modeling (MFM)...
research
06/21/2022

Attention-driven Active Vision for Efficient Reconstruction of Plants and Targeted Plant Parts

Visual reconstruction of tomato plants by a robot is extremely challengi...

Please sign up or login with your details

Forgot password? Click here to reset