Neurosymbolic AI for Situated Language Understanding

12/05/2020
by   Nikhil Krishnaswamy, et al.
16

In recent years, data-intensive AI, particularly the domain of natural language processing and understanding, has seen significant progress driven by the advent of large datasets and deep neural networks that have sidelined more classic AI approaches to the field. These systems can apparently demonstrate sophisticated linguistic understanding or generation capabilities, but often fail to transfer their skills to situations they have not encountered before. We argue that computational situated grounding provides a solution to some of these learning challenges by creating situational representations that both serve as a formal model of the salient phenomena, and contain rich amounts of exploitable, task-appropriate data for training new, flexible computational models. Our model reincorporates some ideas of classic AI into a framework of neurosymbolic intelligence, using multimodal contextual modeling of interactive situations, events, and object properties. We discuss how situated grounding provides diverse data and multiple levels of modeling for a variety of AI learning challenges, including learning how to interact with object affordances, learning semantics for novel structures and configurations, and transferring such learned knowledge to new objects and situations.

READ FULL TEXT

page 8

page 9

page 10

page 11

page 12

page 13

page 14

page 17

research
03/01/2023

Understanding Natural Language Understanding Systems. A Critical Analysis

The development of machines that «talk like us», also known as Natural L...
research
05/24/2023

Exploring the Grounding Issues in Image Caption

This paper explores the grounding issue concerning multimodal semantic r...
research
02/05/2019

Situational Grounding within Multimodal Simulations

In this paper, we argue that simulation platforms enable a novel type of...
research
04/21/2020

Experience Grounds Language

Successful linguistic communication relies on a shared experience of the...
research
11/11/2021

An Enactivist account of Mind Reading in Natural Language Understanding

In this paper we apply our understanding of the radical enactivist agend...
research
06/14/2022

ProcTHOR: Large-Scale Embodied AI Using Procedural Generation

Massive datasets and high-capacity models have driven many recent advanc...
research
02/16/2023

What A Situated Language-Using Agent Must be Able to Do: A Top-Down Analysis

Even in our increasingly text-intensive times, the primary site of langu...

Please sign up or login with your details

Forgot password? Click here to reset