Scene Dynamics: Counterfactual Critic Multi-Agent Training for Scene Graph Generation

by   Long Chen, et al.

Scene graphs -- objects as nodes and visual relationships as edges -- describe the whereabouts and interactions of the things and stuff in an image for comprehensive scene understanding. To generate coherent scene graphs, almost all existing methods exploit the fruitful visual context by modeling message passing among objects, fitting the dynamic nature of reasoning with visual context, eg, "person" on "bike" can help determine the relationship "ride", which in turn contributes to the category confidence of the two objects. However, we argue that the scene dynamics is not properly learned by using the prevailing cross-entropy based supervised learning paradigm, which is not sensitive to graph inconsistency: errors at the hub or non-hub nodes are unfortunately penalized equally. To this end, we propose a Counterfactual critic Multi-Agent Training (CMAT) approach to resolve the mismatch. CMAT is a multi-agent policy gradient method that frames objects as cooperative agents, and then directly maximizes a graph-level metric as the reward. In particular, to assign the reward properly to each agent, CMAT uses a counterfactual baseline that disentangles the agent-specific reward by fixing the dynamics of other agents. Extensive validations on the challenging Visual Genome benchmark show that CMAT achieves a state-of-the-art by significant performance gains under various settings and metrics.


page 6

page 14

page 15


Counterfactual Multi-Agent Policy Gradients

Cooperative multi-agent systems can be naturally used to model many real...

Learning to Compose Dynamic Tree Structures for Visual Contexts

We propose to compose dynamic tree structures that place the objects in ...

Counterfactual Multi-Agent Reinforcement Learning with Graph Convolution Communication

We consider a fully cooperative multi-agent system where agents cooperat...

Prioritized Guidance for Efficient Multi-Agent Reinforcement Learning Exploration

Exploration efficiency is a challenging problem in multi-agent reinforce...

An End-to-End Network for Generating Social Relationship Graphs

Socially-intelligent agents are of growing interest in artificial intell...

GEMS: Scene Expansion using Generative Models of Graphs

Applications based on image retrieval require editing and associating in...

Object-oriented state editing for HRL

We introduce agents that use object-oriented reasoning to consider alter...