"What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models

by   Michael Xieyang Liu, et al.

Code-generating large language models translate natural language into code. However, only a small portion of the infinite space of naturalistic utterances is effective at guiding code generation. For non-expert end-user programmers, learning this is the challenge of abstraction matching. We examine this challenge in the specific context of data analysis in spreadsheets, in a system that maps the users natural language query to Python code using the Codex generator, executes the code, and shows the result. We propose grounded abstraction matching, which bridges the abstraction gap by translating the code back into a systematic and predictable naturalistic utterance. In a between-subjects, think-aloud study (n=24), we compare grounded abstraction matching to an ungrounded alternative based on previously established query framing principles. We find that the grounded approach improves end-users' understanding of the scope and capabilities of the code-generating model, and the kind of language needed to use it effectively.


page 5

page 6


UGIF: UI Grounded Instruction Following

New smartphone users have difficulty engaging with it and often use only...

"How to best say it?" : Translating Directives in Machine Language into Natural Language in the Blocks World

We propose a method to generate optimal natural language for block place...

SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models

Computer end users have spent billions of hours completing daily tasks l...

A Mathematical Abstraction for Balancing the Trade-off Between Creativity and Reality in Large Language Models

Large Language Models have become popular for their remarkable capabilit...

AmadeusGPT: a natural language interface for interactive animal behavioral analysis

The process of quantifying and analyzing animal behavior involves transl...

From Words to Wires: Generating Functioning Electronic Devices from Natural Language Descriptions

In this work, we show that contemporary language models have a previousl...

Please sign up or login with your details

Forgot password? Click here to reset