A Drop of Ink may Make a Million Think: The Spread of False Information in Large Language Models

by   Ning Bian, et al.

Large language models (LLMs) like ChatGPT have gained increasing prominence in artificial intelligence, making a profound impact on society and various industries like business and science. However, the presence of false information on the internet and in text corpus poses a significant risk to the reliability and safety of LLMs, underscoring the urgent need to understand the mechanisms of how false information impacts and spreads in LLMs. In this paper, we investigate how false information spreads in LLMs and affects related responses by conducting a series of experiments on the effects of source authority, injection paradigm, and information relevance. Specifically, we compare four authority levels of information sources (Twitter, web blogs, news reports, and research papers), two common knowledge injection paradigms (in-context injection and learning-based injection), and three degrees of information relevance (direct, indirect, and peripheral). The experimental results show that (1) False information will spread and contaminate related memories in LLMs via a semantic diffusion process, i.e., false information has global detrimental effects beyond its direct impact. (2) Current LLMs are susceptible to authority bias, i.e., LLMs are more likely to follow false information presented in a trustworthy style like news or research papers, which usually causes deeper and wider pollution of information. (3) Current LLMs are more sensitive to false information through in-context injection than through learning-based injection, which severely challenges the reliability and safety of LLMs even if all training data are trusty and correct. The above findings raise the need for new false information defense algorithms to address the global impact of false information, and new alignment algorithms to unbiasedly lead LLMs to follow internal human values rather than superficial patterns.


page 1

page 2

page 3

page 4


Combating fake news by empowering fact-checked news spread via topology-based interventions

Rapid information diffusion and large-scaled information cascades can en...

Online search is more likely to lead students to validate true news than to refute false ones

With the spread of high-speed Internet and portable smart devices, the w...

Mitigating Adversarial Effects of False Data Injection Attacks in Power Grid

Deep Neural Networks have proven to be highly accurate at a variety of t...

An Information Diffusion Approach to Rumor Propagation and Identification on Twitter

With the increasing use of online social networks as a source of news an...

Estudo Abordando o Contexto de Notícias Falsas em Países de Língua Portuguesa (Fake News)

This work consists of a study that addresses the context of false news i...

On the Effects of Data Distortion on Model Analysis and Training

Data modification can introduce artificial information. It is often assu...

Streamline-Based Simulation of Carbon Dioxide Sequestration in Saline Aquifers

Subsurface sequestration of CO2 has received attention from the global s...

Please sign up or login with your details

Forgot password? Click here to reset