Using Large Language Models to Generate Engaging Captions for Data Visualizations

12/27/2022
by   Ashley Liew, et al.
0

Creating compelling captions for data visualizations has been a longstanding challenge. Visualization researchers are typically untrained in journalistic reporting and hence the captions that are placed below data visualizations tend to be not overly engaging and rather just stick to basic observations about the data. In this work we explore the opportunities offered by the newly emerging crop of large language models (LLM) which use sophisticated deep learning technology to produce human-like prose. We ask, can these powerful software devices be purposed to produce engaging captions for generic data visualizations like a scatterplot. It turns out that the key challenge lies in designing the most effective prompt for the LLM, a task called prompt engineering. We report on first experiments using the popular LLM GPT-3 and deliver some promising results.

READ FULL TEXT

page 3

page 4

page 6

research
06/01/2023

CapText: Large Language Model-based Caption Generation From Image Context and Description

While deep-learning models have been shown to perform well on image-to-t...
research
05/03/2022

How Do Captions Affect Visualization Reading?

Captions help readers better understand visualizations. However, if the ...
research
07/21/2018

What is not where: the challenge of integrating spatial representations into deep learning architectures

This paper examines to what degree current deep learning architectures f...
research
01/20/2021

Towards Understanding How Readers Integrate Charts and Captions: A Case Study with Line Charts

Charts often contain visually prominent features that draw attention to ...
research
04/17/2018

Learning to Color from Language

Automatic colorization is the process of adding color to greyscale image...
research
07/10/2019

Hillview: A trillion-cell spreadsheet for big data

Hillview is a distributed spreadsheet for browsing very large datasets t...

Please sign up or login with your details

Forgot password? Click here to reset