DeepAI AI Chat
Log In Sign Up

Moving Forward by Moving Backward: Embedding Action Impact over Action Semantics

by   Kuo-Hao Zeng, et al.

A common assumption when training embodied agents is that the impact of taking an action is stable; for instance, executing the "move ahead" action will always move the agent forward by a fixed distance, perhaps with some small amount of actuator-induced noise. This assumption is limiting; an agent may encounter settings that dramatically alter the impact of actions: a move ahead action on a wet floor may send the agent twice as far as it expects and using the same action with a broken wheel might transform the expected translation into a rotation. Instead of relying that the impact of an action stably reflects its pre-defined semantic meaning, we propose to model the impact of actions on-the-fly using latent embeddings. By combining these latent action embeddings with a novel, transformer-based, policy head, we design an Action Adaptive Policy (AAP). We evaluate our AAP on two challenging visual navigation tasks in the AI2-THOR and Habitat environments and show that our AAP is highly performant even when faced, at inference-time with missing actions and, previously unseen, perturbed action space. Moreover, we observe significant improvement in robustness against these actions when evaluating in real-world scenarios.


page 2

page 5

page 8

page 18


Learning Action Embeddings for Off-Policy Evaluation

Off-policy evaluation (OPE) methods allow us to compute the expected rew...

Pushing it out of the Way: Interactive Visual Navigation

We have observed significant progress in visual navigation for embodied ...

Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translation

Simultaneous machine translation (SiMT) starts its translation before re...

Utilizing Skipped Frames in Action Repeats via Pseudo-Actions

In many deep reinforcement learning settings, when an agent takes an act...

Efficient Empowerment

Empowerment quantifies the influence an agent has on its environment. Th...

Conservative Exploration using Interleaving

In many practical problems, a learning agent may want to learn the best ...

Off-Policy Evaluation in Embedded Spaces

Off-policy evaluation methods are important in recommendation systems an...