FLIN: A Flexible Natural Language Interface for Web Navigation

10/24/2020
by   Sahisnu Mazumder, et al.
0

AI assistants have started carrying out tasks on a user's behalf by interacting directly with the web. However, training an interface that maps natural language (NL) commands to web actions is challenging for existing semantic parsing approaches due to the variable and unknown set of actions that characterize websites. We propose FLIN, a natural language interface for web navigation that maps NL commands to concept-level actions rather than low-level UI interactions, thus being able to flexibly adapt to different websites and handle their transient nature. We frame this as a ranking problem where, given a user command and a webpage, FLIN learns to score the most appropriate navigation instruction (involving action and parameter values). To train and evaluate FLIN, we collect a dataset using nine popular websites from three different domains. Quantitative results show that FLIN is capable of adapting to new websites in a given domain.

READ FULL TEXT
research
03/30/2021

Grounding Open-Domain Instructions to Automate Web Support Tasks

Grounding natural language instructions on the web to perform previously...
research
01/16/2020

Schema2QA: Answering Complex Queries on the Structured Web with a Neural Model

Virtual assistants today require every website to submit skills individu...
research
04/26/2019

Characterizing web pornography consumption from passive measurements

Web pornography represents a large fraction of the Internet traffic, wit...
research
06/09/2023

Mind2Web: Towards a Generalist Agent for the Web

We introduce Mind2Web, the first dataset for developing and evaluating g...
research
03/08/2021

Modeling Web Browsing Behavior across Tabs and Websites with Tracking and Prediction on the Client Side

Clickstreams on individual websites have been studied for decades to gai...
research
08/19/2020

Automatic Generation of Chatbots for Conversational Web Browsing

In this paper, we describe the foundations for generating a chatbot out ...
research
07/23/2016

Processing Natural Language About Ongoing Actions

Actions may not proceed as planned; they may be interrupted, resumed or ...

Please sign up or login with your details

Forgot password? Click here to reset