Truth-Conditional Captioning of Time Series Data

10/05/2021
by   Harsh Jhamtani, et al.
0

In this paper, we explore the task of automatically generating natural language descriptions of salient patterns in a time series, such as stock prices of a company over a week. A model for this task should be able to extract high-level patterns such as presence of a peak or a dip. While typical contemporary neural models with attention mechanisms can generate fluent output descriptions for this task, they often generate factually incorrect descriptions. We propose a computational model with a truth-conditional architecture which first runs small learned programs on the input time series, then identifies the programs/patterns which hold true for the given input, and finally conditions on only the chosen valid program (rather than the input time series) to generate the output text description. A program in our model is constructed from modules, which are small neural networks that are designed to capture numerical patterns and temporal information. The modules are shared across multiple programs, enabling compositionality as well as efficient learning of module parameters. The modules, as well as the composition of the modules, are unobserved in data, and we learn them in an end-to-end fashion with the only training signal coming from the accompanying natural language text descriptions. We find that the proposed model is able to generate high-precision captions even though we consider a small and simple space of module types.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/16/2020

Human-like Time Series Summaries via Trend Utility Estimation

In many scenarios, humans prefer a text-based representation of quantita...
research
07/14/2020

Modeling Financial Time Series using LSTM with Trainable Initial Hidden States

Extracting previously unknown patterns and information in time series is...
research
11/07/2016

Latent Attention For If-Then Program Synthesis

Automatic translation from natural language descriptions into programs i...
research
04/10/2016

TGIF: A New Dataset and Benchmark on Animated GIF Description

With the recent popularity of animated GIFs on social media, there is ne...
research
04/22/2021

SBNet: Segmentation-based Network for Natural Language-based Vehicle Search

Natural language-based vehicle retrieval is a task to find a target vehi...
research
01/28/2022

Summarizing Differences between Text Distributions with Natural Language

How do two distributions of texts differ? Humans are slow at answering t...
research
06/19/2018

VirtualHome: Simulating Household Activities via Programs

In this paper, we are interested in modeling complex activities that occ...

Please sign up or login with your details

Forgot password? Click here to reset