Exploring the Feasibility of ChatGPT for Event Extraction

by   Jun Gao, et al.

Event extraction is a fundamental task in natural language processing that involves identifying and extracting information about events mentioned in text. However, it is a challenging task due to the lack of annotated data, which is expensive and time-consuming to obtain. The emergence of large language models (LLMs) such as ChatGPT provides an opportunity to solve language tasks with simple prompts without the need for task-specific datasets and fine-tuning. While ChatGPT has demonstrated impressive results in tasks like machine translation, text summarization, and question answering, it presents challenges when used for complex tasks like event extraction. Unlike other tasks, event extraction requires the model to be provided with a complex set of instructions defining all event types and their schemas. To explore the feasibility of ChatGPT for event extraction and the challenges it poses, we conducted a series of experiments. Our results show that ChatGPT has, on average, only 51.04 the performance of a task-specific model such as EEQA in long-tail and complex scenarios. Our usability testing experiments indicate that ChatGPT is not robust enough, and continuous refinement of the prompt does not lead to stable performance improvements, which can result in a poor user experience. Besides, ChatGPT is highly sensitive to different prompt styles.


page 1

page 2

page 3

page 4


Event Extraction: A Survey

Extracting the reported events from text is one of the key research them...

Simultaneous Machine Translation with Large Language Models

Large language models (LLM) have demonstrated their abilities to solve v...

Salience-Aware Event Chain Modeling for Narrative Understanding

Storytelling, whether via fables, news reports, documentaries, or memoir...

Extracting Victim Counts from Text

Decision-makers in the humanitarian sector rely on timely and exact info...

STAR: Boosting Low-Resource Event Extraction by Structure-to-Text Data Generation with Large Language Models

Structure prediction tasks such as event extraction require an in-depth ...

Quantifying the Task-Specific Information in Text-Based Classifications

Recently, neural natural language models have attained state-of-the-art ...

COVID-19 event extraction from Twitter via extractive question answering with continuous prompts

As COVID-19 ravages the world, social media analytics could augment trad...

Please sign up or login with your details

Forgot password? Click here to reset