A network perspective on intermedia agenda-setting

by   Samuel Stern, et al.

In Communication Theory, intermedia agenda-setting refers to the influence that different news sources may have on each other, and how this subsequently affects the breadth of information that is presented to the public. Several studies have attempted to quantify the impact of intermedia agenda-setting in specific countries or contexts, but a large-scale, data-driven investigation is still lacking. Here, we operationalise intermedia agenda-setting by putting forward a methodology to infer networks of influence between different news sources on a given topic, and apply it on a large dataset of news articles published by globally and locally prominent news organisations in 2016. We find influence to be significantly topic-dependent, with the same news sources acting as agenda-setters (i.e., central nodes) with respect to certain topics and as followers (i.e., peripheral nodes) with respect to others. At the same time, we find that the influence networks associated to most topics exhibit small world properties, which we find to play a significant role towards the overall diversity of sentiment expressed about the topic by the news sources in the network. In particular, we find clustering and density of influence networks to act as competing forces in this respect, with the former increasing and the latter reducing diversity.



There are no comments yet.


page 1

page 2

page 3

page 4


Detecting Polarized Topics in COVID-19 News Using Partisanship-aware Contextualized Topic Embeddings

Growing polarization of the news media has been blamed for fanning disag...

Framing Matters: Predicting Framing Changes and Legislation from Topic News Patterns

News has traditionally been well researched, with studies ranging from s...

A Multi-Platform Analysis of Political News Discussion and Sharing on Web Communities

The news ecosystem has become increasingly complex, encompassing a wide ...

An unsupervised framework for tracing textual sources of moral change

Morality plays an important role in social well-being, but people's mora...

MediaRank: Computational Ranking of Online News Sources

In the recent political climate, the topic of news quality has drawn att...

Characterizing the public perception of WhatsApp through the lens of media

WhatsApp is, as of 2018, a significant component of the global informati...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The news media is an important information source for much of the world’s population, as it feeds into opinions and choices [1, 2, 3]. Frequently referred to as the ‘gatekeepers of information’, journalists have been shown to play a powerful role in influencing public opinion through determining what stories (or elements thereof) are presented to the public, how content is framed, which elements are emphasized, and how the public forms associations between topics covered by the news [4, 3]. While much research has been done on identifying bias in the mainstream media [3], and has more recently focused on ‘fake news’ and misinformation (e.g., [5, 6, 7]), little research has thus far looked at studying the dynamics underpinning the formation of opinions within the media. News organisations do not construct narratives in isolation but are instead influenced by each other. This phenomenon is the subject of intermedia agenda-setting theory [8], which explains the ‘co-orientation’ of the narratives in the media as the result of exposure to both economic and local social factors [4]. The topics that the media covers and the context in which headlines are presented have been shown to impact observable phenomena ranging from voter behaviour [2] to macroeconomic indicators [9]. Affective attributes of the news content, such as sentiment and tone, have been shown to affect voters’ judgment on political issues [10, 11] and people’s perception of corporations [12]. Because of its far-reaching impact, it is valuable to understand the opinion dynamics of the media at the level of individual news organisations as it ultimately determines what content is made available to the public as well as how it is presented.

The media continues to play a central role in directing what information the public attends to, how it is presented, and with what they associate that information. It is therefore important to have a stable, balanced and diverse set of opinions being presented to the public. Research on agenda-setting theories of the media [1] generally either aggregate the media into a single representative agent or treat the media as a collection of independent actors. Here, we demonstrate an alternative representation in which news sources can be understood as nodes in a social network whose actions are influenced by the other nodes in the network. By framing the system as a social network, we are able to exploit the tools provided by network science to aid in our understanding of how such a system evolves. Numerous studies have demonstrated how network structure impacts signal propagation [13, 14]. However, we are not aware of any research to date that has employed a network approach to explore the propagation of information within the media.

In many real-world social networks, connections and interactions between agents can be explicitly gathered from the data, such as through citation in academia [15], or liking and following in social media [16, 17]. In the domain of the news media, whilst there are naturally interactions between agents, these are latent and network structures must first be inferred from the available data before one can study the dissemination and propagation of news and sentiment through them. Previous work on studying the dissemination of news data from a social network modelling approach has explored what content the media presents [18] and how breaking news stories are picked up and propagated through the media [19], but not how the network of interactions of the news sources impacts how content is framed.

In this work, we tackle the above by presenting a method to operationalise intermedia agenda-setting theory in which each news source is a potential node in a directed network and edges exist between news sources when there is empirical evidence that the sentiment expressed by one source on a topic is Granger-caused by that of another. We implement the proposed method and employ it to answer the following research questions (RQs):

  • RQ1: Can ‘bellwether’ news sources be identified? That is to say, sources who have influence over the way in which the rest of the media portray a story. Moreover, are such bellwethers more central in networks that capture intermedia influence?

  • RQ2: Is there emergent structure in the networks that can potentially impact how a news story is depicted and propagated across the media?

  • RQ3: How does the structure of these networks impact the diversity of sentiment that exist across the media on a given topic?

We propose and implement the aforementioned method, which constitutes our first contribution. We perform topic modelling on a collection of news articles to identify 200 topics discussed in the news media between May and December 2016. Sentiment analysis is then used to model the views adopted by each news source and how these vary over time. Lead-lag relationships between the sentiment expressed across news sources are identified to construct networks representing how sentiment is transmitted and adopted by the media. Our second contribution is then in applying this method to explore the diffusion of sentiment in the media. Our final contribution is towards the discussion of the ability of the media to self-regulate and provide a varied perspective for public opinion formation.

We find:

  1. There exist significant lead-lag relationships in which specific news sources can influence the sentiment with which a story is presented.

  2. Intermedia influence networks are able to capture characteristics that are consistent with intermedia agenda-setting theory, such that certain elite news sources sit more centrally.

  3. Influence is distributed across media sources and bellwether behaviour varies depending on the topic.

  4. Influence networks exhibit small-world characteristics with communities of densely connected subgroups.

  5. The overall diversity in sentiment is impacted by the structural features of intermedia networks. In particular, the density of such intermedia networks has a negative impact on the sentiment diversity on a topic whereas the strength of the network clustering has a positive impact on it.

The remainder of this paper is organised as follows: in section 2 we provide a background and overview of related work on influence analysis in social networks. In section 3 we outline our methodology, and in section 4 we provide a detailed analysis of our results and discuss their implications. Finally, section 5 provides some conclusive remarks and proposes potential further work.

2 Related Work

Since the beginning of the 20 century, numerous theories have emerged that try to explain the role of the media in society and the impact that it has on public opinion (and vice versa). Arguably the most broadly accepted theory today is based on agenda-setting theory [1] and extensions thereof. It argues that the media does not regulate what audiences think, but rather it tells them what to think about by controlling what content is accessible. It paints the media as ‘gatekeepers’ of information about reality by selecting, omitting, and framing what issues are reported on, and to what extent. This then prompts the public to perceive selected issues as being more/less important than others, or drawing links between events, which ultimately changes how the audience engages with the issue and thereby influences the public’s narrative [20].

Beyond looking at the relationship between the media and the public, intermedia agenda-setting research concerns itself with measuring the extent to which news content transfers between media. The highly regarded media have been shown to exert influence over their peers [18, 8]. This “co-orientation” within the media can be attributed to several factors. One explanation is that it is the result of ‘churnalism’ whereby media sources re-hash the content of their peers without having to expend the resources that are required to construct the narrative [8, 21]. Another theory is that co-orientation is sociopsychological in origin, whereby the major news sources are regarded as better judges of what is (subjectively) newsworthy [8, 22]. Recently intermedia agenda-setting has also considered the transfer of content between different types of media, such as whether, and when, the mainstream media influences social media, and vice versa. It has been shown, for instance, that the online news media influences the content of politically aligned Twitter accounts [8].

So far, intermedia agenda-setting research has focused on testing a priori hypotheses about the expected influence exerted by certain news sources. For example [18] finds that The New York Times and The Washington Post have the ability to set the agenda of others. In [23], the authors find a similar effect for Algemeen Nederlands Persbureau (ANP) in the Dutch press, while [21] identifies intermedia agenda-setting effects of Chosun.com and Donga.com on other South Korean online media websites.

Here, by operationalising intermedia agenda-setting in network terms we circumvent the need for a priori assumptions about which sources may act as agenda-setters. A core component in this is to automatically capture and interpret how the agents in the news media influence one another. As will be discussed in the next section, the concept of ‘influence’, and how it is measured, has emerged as a popular field of research in applied mathematics and computer science.

2.1 Social Network Influence

There is no single mathematical definition of social network influence. This is largely because it varies between contexts, and even within the same domain, it can be unclear how influence is defined. Peng et al. [24] describe four basic components that define influence, regardless of the domain: (1) social influence exists between two agents, the influencer and the influencee (aka. the influenced); (2) influence is a function of uncertainty, with zero uncertainty if the influencer has no doubt that the influencee will perform an action and the maximum amount of uncertainty when the influencer has no influence over the influencee’s actions; (3) the level of influence can be represented by a real number which describes the uncertainty of the influencer’s ability to dictate the influencee’s action; (4) influence does not need to be symmetric.

Consistent with the above criteria, numerous evaluation metrics have been proposed to assign influence values to agents in social networks, some based purely on centrality and network topology, and others on entropy measures. Degree, closeness, PageRank, HITS and Katz centralities have all been used in past studies

[25, 24, 26, 27].

While influence measures based on network topology are convenient because they may be calculated for an arbitrary network, they ignore potentially valuable information about the content of the signals being passed through networks. Entropy measures of influence are one method for including signal content. Ver Steeg and Galstyan [28], for example, propose the use of transfer entropy, a measure for determining how much better the sequence of signals emitted by one agent can be predicted by including the signal history from another agent. Other methods have focused on whether the signals transmitted by one agent Granger-cause those of the other agents within the same network [18, 29]. We will build networks of influence based on the latter approach.

2.2 Opinion and Emotion Influence

Prior research has approached the question of how actors influence one another through emotion from two differing disciplines. The idea of how emotions and opinions are influenced has been studied extensively in the fields of psychology, cognitive science and sociology [30, 31, 32]. More recently though, computational and mathematical models have explored how sentiment and opinions spread through social networks. The addition of emotion in the context of influence within social networks has been examined in Twitter [33, 34, 35], Facebook [36], and blog posts [37, 38]. Semantic information expressed by actors in social media has been proposed as a method to augment influencer detection beyond what is possible using only network topology [38, 39]. Inversely, incorporating information about a user’s influence based on network topology has been used to augment sentiment analysis [40, 37]. Furthermore, contagion models have been developed to study how emotion and emotive content propagates and diffuses throughout social networks; sentiment in social media-based content has been shown to correlate with both the speed and quantity of information sharing [41, 42, 33, 35].

3 Methodology

3.1 Identifying Sentiment-Loaded Topics

Whereas prior related studies have looked at how news stories are picked up on, we are concerned not only with what is discussed but also with capturing how it is presented. In other words, understanding from what angle the story or ‘theme’ is depicted as well as how this varies across different news sources and different topics. Therefore, our goal is to construct time-series that encapsulate two factors: (2) the prominence that a source assigns to a recurring topic, and (2) the sentiment with which it covers it. We do this by performing topic modelling on news articles to identify what the stories are about; we then scale the topic assignments according to the sentiment of the articles. This gives us an indicator of the affinity between the source of the articles and the topics the article covers.

Our method produces a tensor, where is the number of topics, is the number of news sources, and is the length of our time-series. In other words, each news source, has distinct time-series that describe how their sentiment towards each of the topics evolves through time. Topic extraction is performed by training a Latent Dirichlet Allocation (LDA) topic model [43] with

topics on a corpus containing news articles from a range of different sources. LDA is an unsupervised learning technique that models documents as a multinomial distribution over

topics, and topics as a multinomial distribution over words. It has been shown to successfully capture semantically coherent clusters of topics in medium-length texts such as news articles, blogs and journals [43]. Using the LDA model, each article in the corpus is mapped to a multinomial topic distribution that encodes how ‘much’ of each of the topics is entailed within the article. Topics are modelled as being static over time (i.e., the per-topic word distributions are independent of time) because long-term media coverage has been shown to have a greater effect on policy-making than topics that are covered over the short- or medium-term through repetition and cumulative exposure [44].

The sentiment for each article is subsequently measured following [45], i.e., counting the relative frequency of ’excitement’ and ‘anxiety’ keywords that have been shown to capture the affective impact that articles may have on a reader. That is, the sentiment value of an article is the number of excitement keywords minus the number of anxiety keywords, all as a fraction of the length of the article. Despite there being more sophisticated methods of capturing the sentiment of a document, using pre-defined keywords allows us to explicitly remove those words from the vocabulary that constitutes the topic distribution of words in the LDA model. This ensures that when the sentiment dynamics of each of the topics are later analysed, they are not biased by intrinsically ‘positive’ or ‘negative’ topics.

To capture how a particular news source ‘feels’ about a topic on any given day, each of that source’s documents, from that day, are assigned a topic distribution using the LDA model. The topic probabilities for each of that source’s articles are scaled by the sentiment score of that article and then summed together across articles. Repeating this process daily for each news source allows us to populate a tensor

where is the sentiment of a news source on topic on a day . The steps for assigning topic-sentiment time series are presented in algorithm 1.

input : Set of articles
output : Tensor of size
dates of articles in ;
set of topics;
set of news sources;
for  do
       for  do
             source of document ;
             multinomial topic distribution of document ;
             sentiment of document ;
             for   do
             end for
       end for
end for
Algorithm 1 Topic-Sentiment Assignment

The sentiment score assigned to a given news source on a given topic on a given day is influenced by three factors: the strength of the emotion in the content they published, the topic entropy of their articles (i.e., the probability mass of the articles that are attributed to a topic), and amount that they invest in a topic (i.e., the number of articles that have non-zero probability mass assigned to the topic). In other words, an a source’s score will be high if there is substantial positive emotion expressed in their content and/or their writing is about a specific topic and/or there is a proportionately large number of their articles about said topic.

3.2 Topic-Level Influence Networks

There have been many proposed definitions of influence, and metrics vary widely depending on the discipline, application area and data. The definition used here follows that of [18, 29] in which, for two agents and , is considered to influence the opinion of on a given topic if the prior opinion expressed by on that topic increase our ability to predict that of B. More formally, if the conditional probability of observing is dependent on for some time and lag , then influences . This lead-lag relationship is measured using the Granger causality test [46]. Source ’s sentiment leads that of on a particular topic if ’s sentiment time series Granger-causes that of on that topic with a -value lower than 0.01.

Figure 1: The network-construction process: A topic-specific sentiment time-series is inferred for each news source based on the content that they publish. Networks are inferred for each topic based on the empirically measured conditional dependence between the sentiment time-series of the news sources.

For each pair of news sources, once lead-lag relationships between agents are identified, they can be used to populate topic-level influence networks. Graphs depicting the relationships between pairs of news sources are then constructed separately for each topic. Where a lead-lag relationship has been determined, both agents are added to the relevant graphs as nodes. An edge is drawn between them, with an arrow leading from the influencer to the influencee, resulting in a directed graph. This procedure is sketched in Figure 1.

Figure 2: An example of the intermedia influence networks. Each node is an agent within the news media and an edge exists going from an influencer to an influencee.

Network sizes vary across topics because a news source is only added to a network if it has at least one edge. This results in sources that are not active in the discussion of a given topic, or which have no impact on their peers, to be excluded. Figure 2 shows an example of such a network.

3.3 Opinion Diversity

One of the questions we wish to address is whether we can deduce how diverse the sentiment surrounding a topic is, given what we are able to capture of the influence that news sources have on one another. We, therefore, desire a measurable statistic that represents how varied news sources are in their sentiment towards a given topic. For this purpose, we define a measure of ‘opinion diversity’ on a topic

as the median of the daily cross-sectional variance of sentiment across news sources:


Low values of

indicate a general consensus in sentiment, whereas higher values reflect greater heterogeneity. We choose the median as a measure of central tendency as it is robust to skewed distributions, which in our domain arise when a topic receives a burst of news activity, such as a breaking news event.

4 Results

The source of the news articles used in this research is the LexisNexis Newsdesk dataset. The dataset consists of 113,000,000 articles from over 34,000 news sources dating from 6 May 2016 to 31 Dec 2016. In addition to the content of the news articles, the dataset contains further metadata, including the source’s country and broad topic labels such as ‘Top Stories’, ‘Sport’, and ‘Fashion’. To only focus on relevant articles, only those with between 100 and 1700 words in length and tagged under ‘Top Stories’ are kept. Furthermore, to eliminate sources that only sporadically publish relevant content, all sources with an average publication rate lower than once per day are ignored. We also remove news aggregators, such as Yahoo! News, and local subsidiaries of large national sources. This leaves 97 news sources useable in this study.

Standard pre-processing is performed on each of the texts including tokenisation, lemmatisation and the removal of stopwords, before performing topic modelling and sentiment extraction on the articles.

Article topics are assigned using an LDA model of topics trained using the Gensim Python package [47]. The number of topics was chosen to maximise the model’s coherence [48] and a subset of the topics are presented in appendix A. Qualitative inspection of the topics learned by the LDA model suggests that the model successfully learns to cluster words based on identifiable news topics. These topics include Britain’s exit from the European Union, which assigns high probability to words including [‘europe’, ‘brexit’, ‘farage’], the scandal involving the state-sponsored doping of Russian athletes [‘olympic’, ‘doping’, ‘russia’] and changes to central banks’ interest rates [‘bank’, ‘rate’, ‘bond’].

4.1 Bellwether Behaviour

Figure 3: An example of how topic-specific sentiment changes over time. The example shown here is the ‘Brexit’ topic. The BBC (orange) is a bellwether on this topic as it leads the consensus (blue) whereas CNN (green) lags the consensus.

Here we present the results and discussion concerning RQ1, which asks whether the sentiment that news sources adopt is influenced by those of their peers.

There has been extensive research into herd behaviour and interdependence within the fields of behavioural economics and social psychology, in which studies have shown that an individual’s bias towards a particular group is a significant contributor towards their social relations [49]. This behaviour has also been proposed as a factor in determining how stories are framed in the media [3].

We distinguish between two indications of potential peer influence. The first ‘one-versus-all’ approach considers whether the dynamics of a news source’s sentiment on a particular topic influences, or is influenced, by the average sentiment expressed by the remainder of their peers and competitors, which we will refer to as the consensus. The second is whether there is a measurable relationship between distinct pairs of agents that cannot be explained away by the consensus.

To test whether an individual news source acts as a leading indicator on that topic it is first removed from the consensus and then tested to see whether the time-series that represents the consensus on a topic is Granger-caused by the news source in question. The inverse is also performed to determine if the sentiment of the source are Granger-caused by the consensus. This test is performed for each topic and each news source that contributes to that topic. We refer to sources that act as leading indicators as ‘bellwethers’ and an example of this is presented in figure 3.

Figure 4 presents an overview of the relative frequencies at which a news source leads versus lags the consensus. Of the 15,800 possible lead-lag relationships, there are 379 (2.4%) cases in which a source Granger-causes the consensus and 256 (1.6%) instances where a news source is Granger-caused by the consensus.

Figure 4: Histogram of the number of topics that each news source lead and lag.

Each of the 97 news sources acts as bellwether for at least one topic with the average news source being credited as a bellwether for 5 different topics. A manual inspection of the set of agents suggests that regional and national-level outlets are more likely to lead on topics within their own geopolitical environments than they are on non-local topics. This is conceivably because they are attuned to issues that are of concern to their readership base. Examples of this are the Miami Herald being a leading source on topics surrounding the ‘Orlando Nightclub Shooting’, or the London Evening Standard being a leading indicator on the topic concerning ‘London’, ‘Londoners’ and ‘Sadiq Kahn’. Conversely, a minority of media outlets are leaders of a proportionately large number of topics. These are found to generally be larger outlets, such as the New York Times and The Telegraph, both of whom act as bellwethers on 13 topics, The Los Angeles Times with 10 topics, and The Washington Post with 9. Considering the instances where a source lags the consensus, the results also indicate that no single news outlet is wholly independent of other sources, but rather all have at least one topic for which they are trend followers.

This first result supports the notion that there is not only some level of interdependence in the topics which the media chooses to report on but also in the manner in which they do so. Next, we consider influence at a pairwise level and whether this phenomenon is mirrored in the influence networks, described in section 3.2. In particular, we test whether the bellwether news sources are more central in intermedia influence networks than those who are not bellwethers. For this, we assign each news source into one of four groups depending on whether it leads the consensus of a topic (i.e., a bellwether), whether it lags the consensus, whether it neither leads nor lags the consensus or whether it both leads and lags the consensus (mutual Granger causality). The PageRank centrality of each news source is calculated for each of the topic-influence networks that it sits in. The distributions of the resulting centrality measures of the groups, shown in figure 5, are then compared by performing a Kruskal-Wallis H-test (KW test) for independent samples111The assumptions for an ANOVA are not satisfied. The KW test controls whether the median centrality of a news source on a topic differs depending on whether that source was earlier identified as leading the topic consensus, lagging it, neither, or both. Specifically, the test results indicate whether the centrality distribution of at least one of the groups stochastically dominates the others.

The KW test’s results reject the null hypothesis that the groups are generated from the same distribution (

). While from this result we can assume that there is at least one group whose distribution stochastically dominates the others, it does not state which one. To determine where the stochastic dominance occurs we furthermore perform the test between pairs of groups, the results of which are shown in Table 1. The results table shows us that when a news source is a bellwether on a topic, it is also more likely to have higher centrality in the network associated with that topic than when it is not, but there is no statistical difference between those who lag the consensus and those who are neither leaders nor laggers.

Figure 5: The distribution of PageRank centrality values grouped by whether a node is a bellwether, whether it lags the discussion, whether it neither leads nor lags the discussion or whether it both leads and lags the discussion.
Lead Lag Both Neither
Lead () 1
Lag () 0.050 1
Both () 0.785 0.049 1
Neither () 3.47e-5*** 0.362 1.20e-11*** 1
Table 1: The results of the Kruskal-Wallis H-test performed for each pair of groups

4.2 The Structural Properties of Intermedia Influence Networks

Having established that there is evidence of intermedia agenda-setting in which news sources are responsive to the content published by one another, we proceed by considering the structural properties of the networks that capture the intermedia influence.

4.2.1 Intermedia Gatekeeping

The ‘gatekeeping’ function of the media is well documented in which, even in the age of social media, the editors act as information filters and barriers to the content that is disseminated to the public [50]. What is less well understood is whether there is intermedia gatekeeping in which a small subset of actors have influence that spans disproportionately far. We employ our inferred networks to address this question by testing whether individual news sources have disproportionately large influence that could enable them to govern the agenda of the discussion on particular issues. The scope here is not to definitively state whether or not certain players dominate the sentiment on a topic, but rather to ask whether the network structure facilitates this behaviour. We reason that if the media does have highly influential gatekeepers who can directly impact others, then we would expect to see a considerable proportion of topics where the associated network has at least one source that is significantly more central than would be expected to find if influence were assigned randomly. We, therefore, run the following binomial test: for each network, , we begin by considering the out-degree of the most central node, . We then ask what the probability would be of observing a node with out-degree under an Erdős-Rényi network generation process that has the same average degree as . If the probability, , is smaller than a threshold (where includes Bonferroni correction), then we can argue that has at least one disproportionately central actor.

After performing this test on each of the 200 topics, we observe only 6 cases where we can reject the null hypothesis that the centrality of the most central node in the network could be generated from an Erdős-Rényi process. From this result, it cannot be concluded that there is centralised agenda-setting power within the media or that, if there were, that the intermedia network structure would facilitate this.

4.2.2 Small-World Properties

The small-world property of many empirical social networks has been shown to have a profound impact on how efficiently information spreads through them [51]. Due to their high clustering and small characteristic path length [52], the nodes of networks that exhibit the small-world property locally sit within a close-knit community, whilst simultaneously still allowing for information to pass efficiently through the network as a whole. In networks that exhibit the small-world phenomenon, innovations (i.e., novel perspectives) by an individual are likely to spread through the network and be picked up by others. In the context of intermedia influence networks, this phenomenon translates to most sources appearing to be independent of one another, while in fact requiring only few steps for a new spin on a story to propagate through the network as a whole.

There have been several proposed measures of small-worldness, each of which relies on a slightly varying interpretation of how a small-world network is defined [53, 54, 55]. One popular method, proposed by Humphries and Davies [53], measures small-worldness using , where and are the clustering coefficient and average path length of the network respectively, and and are the clustering coefficient and average path length of an equivalent random network. Though having been criticised as overestimating small-worldness on large networks [54],

has the benefit of having a simple heuristic for classifying a network as small-world: if

then the network is said to be small-world. The networks we consider are not particularly large (ranging from between 20 to 50 nodes per network), making the value an appropriate choice. We use the definition proposed in [56] that extends the definition of clustering from the undirected to directed graph case:

where is the number of nodes in the network, is an an indicator function of whether there is an edge going from node to node , and and are the in-degree and out-degree of node respectively. The average shortest path between directed nodes is defined as [57]:

where is the shortest directed path length from node to node .

To generate equivalent random networks that maintain equivalent degree sequences, we follow previous studies [58] by randomly reshuffling edges, whilst preserving the in-degree and out-degree for each node.

Figure 6 shows a histogram of the values of the topic networks. 87.5% of the networks have a value of , a strong indication that the media exhibits small-world tendencies in the manner in which they influence one another to adopt a particular angle or spin on a story.

Figure 6: Histogram showing the distribution of small-worldness, . A value of is considered not to be small-world.

4.2.3 Opinion Diversity

Now that we have a comprehensive view of the topology of intermedia influence networks, we proceed to explore the impact that the topology has on the level of opinion diversity observed across the media.

Having a variety of opinions, emotions and viewpoints expressed by differing sources in the media is imperative to giving the public an opportunity to gain exposure to alternative perspectives on a particular issue [59]. The above analysis of the structure of the networks indicates that there are both instances of identifiable community structures throughout the networks (which may isolate news sources from one another), as well as relatively short average distances between nodes (which conversely, may result in new information or perspectives spreading and being adopted through the network efficiently). In this final section, we seek to quantify the net impact that these two competing forces have on the diversity of sentiment expressed in the news media. To test this we propose a series of linear models that attempt to capture these effects and compare them based on the significance of their coefficients, while also controlling for the size and density of the networks.

We compare three linear regression models and examine their coefficients. The dependent variable,

, is the log of the opinion diversity, defined in Eq. (1). Each of the models contains as independent variables the average shortest path length as well as the size of the network (measured as the number of nodes) as a control variable. The first model, also includes the network density, , as an independent variable while the second model, includes the average clustering coefficient, , instead. The third model, considers the combined effect that both and have on .

Some topics covered in the news media may be intrinsically highly emotive, though not necessarily be particularly divisive or material to overall public opinion. Topics such as sport or celebrity gossip fall into this category, compared to, for example, media coverage about election candidates or international affairs. Therefore, the topics are inspected manually by the authors, and all of the topics that are clearly about sports, celebrity gossip or lifestyle/entertainment are ignored. This results in a sample size of 161 networks.

Avg. shortest
path length ()
Network Size ()
Avg. clustering
coefficient ()
Density () -
0.32 0.340 0.37
Table 2: The regression coefficients of each of the three linear models.

The results provided in Table 2 show the regression coefficients of the three models. In all three of the models, sentiment diversity increases as a function of the size of a network and decreases with the average shortest path length. In and , neither the average clustering coefficient nor the density adds explanatory power beyond that of just the network size and path length. In the third model, however, we see compounding effects in which increasing network density reduces diversity while higher clustering increases it. A possible interpretation of this result is that greater interconnectedness, in general, captures the known effects of intermedia agenda-setting in which news sources tend to mimic one another and mutually reinforce each other. However, the formation of local subgroups in the media contrasts this effect and allows different groups to express different sentiment on the same issue and therefore reduce the global consensus.

5 Conclusion

In this paper, we presented a method to operationalise intermedia agenda-setting by identifying networks of relationships in the media that capture how news sources influence one another to adopt particular sentiment on issues.

We used network analysis to explore the propagation of sentiment about a topic in the media. In doing so, we revealed characteristics of online news dissemination and the sentiment adoption process of the media. We found that, consistent with the expectations of intermedia agenda-setting theory, there is identifiable influence among the news media, both where sources lead/lag the herd, as well as on a peer-to-peer level. While those who lead the consensus sentiment vary from topic to topic, we found that elite news sources, such as the New York Times, Washington Post etc., are more likely to be identified as a bellwether than those who are not elite. This observation contrasts the findings of other recent studies that explore the level of influence maintained by different actors in the media. A study by Vargo and Guo [18] on the influence between types of media concluded that, in response to a shifting audience, the large elite media have become attentive to the agendas of smaller online news sources. Our results suggest that, while the elite media sources can be attentive to others, they do still have comparably more power.

Moreover, while some news sources are more central than others for certain topics networks, we found no significant indication of gatekeeping via centralised agenda-setting structure in the intermedia network, be that by the elite sources or otherwise. Instead, there is evidence of small-world characteristics between interacting news sources, and that this phenomenon is pervasive across topics. Previous work on the network structure of news dissemination has also found small-world properties when investigating the propagation of breaking news [19]. While such work demonstrated that breaking news has the ability to propagate quickly, by focusing on the associated sentiment of prolonged topics, the observed small-worldness in our results indicates that changes in the framing of a story are also subject to being rapidly disseminated. Finally, by comparing descriptive networks statistics to the dynamics of how the media discusses topics we found evidence that intermedia influence can be a positive factor in presenting diverse perspectives. In particular, we saw that while greater degrees of interconnectedness (in the form of more densely connected networks) were associated with reduced diversity in the overall sentiment that the media presents on an issue, this effect was diminished for topics in which the media forms clusters of influence.

As the first study of its kind to look at the impact that the structure of an influence network has on how perspectives are adopted in the media, there are several questions that we hope to answer in future work. Notably, we would like to include entity-specific information into our model, primarily whether there are partisan effects or differences between news agencies and publishers, and how the opinion dynamics vary differently within communities versus between them. We also believe that there is more to be understood of the nature of the influence through further studying the impact of the magnitudes and duration of the lead-lag relationships.

Competing interests

The authors declare that they have no competing interests.

Author’s contributions

SS designed the research and conducted the experiments. GL and RES supervised the research. All authors wrote the manuscript.


We thank Nora Ptakauskaite, Simone Righi, and David Tuckett for helpful comments on preliminary versions of this manuscript. GL acknowledges support from an EPSRC Early Career Fellowship (Grant No. EP/N006062/1).


  • [1] McCombs, M., Shaw, D.L.: The Agenda-Setting Function of Mass Media. The Public Opinion Quarterly 36(2), 176–187 (1972). Accessed 2018-04-20
  • [2] McCombs, M., Llamas, J.P., Lopez-Escobar, E., Rey, F.: Candidate Images in Spanish Elections: Second-Level Agenda-Setting Effects. Journalism & Mass Communication Quarterly 74(4), 703–717 (1997). doi:10.1177/107769909707400404. Accessed 2018-04-20
  • [3] Wilczek, B.: Herd Behaviour and Path Dependence in News Markets: Towards an Economic Theory of Scandal Formation. Journal of Interdisciplinary Economics 28(2), 137–167 (2016). doi:10.1177/0260107916643469. Accessed 2018-12-11
  • [4] Guo, L., Vu, H.T., McCombs, M.: An Expanded Perspective on Agenda- Setting Effects. Exploring the third level of agenda setting. Revista de Comunicación 11, 51–68 (2012)
  • [5] Vargo, C.J., Guo, L., Amazeen, M.A.: The agenda-setting power of fake news: A big data analysis of the online media landscape from 2014 to 2016. New Media & Society 20(5), 2028–2049 (2018). doi:10.1177/1461444817712086. Accessed 2018-12-02
  • [6] Shu, K., Wang, S., Liu, H.: Beyond News Contents: The Role of Social Context for Fake News Detection. In: Proceedings of the Twelfth ACM International Conference on Web Search And Data Mining. WSDM ’19, pp. 312–320. ACM, New York, NY, USA (2019). doi:10.1145/3289600.3290994. event-place: Melbourne VIC, Australia. http://doi.acm.org/10.1145/3289600.3290994 Accessed 2019-08-15
  • [7] Sikder, O., Smith, R.E., Vivo, P., Livan, G.: When facts fail: Bias, polarisation and truth in social networks. arXiv preprint arXiv:1808.08524 (2018)
  • [8] Harder, R.A., Sevenans, J., Van Aelst, P.: Intermedia Agenda Setting in the Social Media Age: How Traditional Players Dominate the News Agenda in Election Times. The International Journal of Press/Politics 22(3), 275–293 (2017). doi:10.1177/1940161217704969. Accessed 2019-11-14
  • [9] Nyman, R.B.E.: An Algorithmic Investigation of Conviction Narrative Theory: Applications in Business, Finance and Economics. Doctoral, UCL (University College London) (February 2016). http://discovery.ucl.ac.uk/1473522/ Accessed 2017-07-17
  • [10] Coleman, R., Wu, H.D.: Proposing Emotion as a Dimension of Affective Agenda Setting: Separating Affect into Two Components and Comparing Their Second-Level Effects. Journalism & Mass Communication Quarterly 87(2), 315–327 (2010). doi:10.1177/107769901008700206. Accessed 2020-02-05
  • [11] Sheafer, T.: How to evaluate it: The role of story-evaluative tone in agenda setting and priming. Journal of Communication 57(1), 21–39 (2007)
  • [12] Kim, J.Y., Kiousis, S.: The Role of Affect in Agenda Building for Public Relations: Implications for Public Relations Outcomes. Journalism & Mass Communication Quarterly 89(4), 657–676 (2012). doi:10.1177/1077699012455387. Accessed 2020-02-05
  • [13] Delre, S.A., Jager, W., Janssen, M.A.: Diffusion dynamics in small-world networks with heterogeneous consumers. Computational and Mathematical Organization Theory 13(2), 185–202 (2007). doi:10.1007/s10588-006-9007-2. Accessed 2018-11-06
  • [14] Watts, D.J., Dodds, P.S.: Influentials, Networks, and Public Opinion Formation. Journal of Consumer Research 34(4), 441–458 (2007). doi:10.1086/518527. Accessed 2018-08-17
  • [15]

    Li, W., Aste, T., Caccioli, F., Livan, G.: Reciprocity and impact in academic careers. EPJ Data Science

    8(1), 20 (2019)
  • [16] Anger, I., Kittl, C.: Measuring Influence on Twitter. In: Proceedings of the 11th International Conference on Knowledge Management and Knowledge Technologies. i-KNOW ’11, pp. 31–1314. ACM, New York, NY, USA (2011). doi:10.1145/2024288.2024326. http://doi.acm.org/10.1145/2024288.2024326 Accessed 2018-04-27
  • [17] Livan, G., Caccioli, F., Aste, T.: Excess reciprocity distorts reputation in online social networks. Scientific reports 7(1), 1–11 (2017)
  • [18] Vargo, C.J., Guo, L.: Networks, Big Data, and Intermedia Agenda Setting: An Analysis of Traditional, Partisan, and Emerging Online U.S. News. Journalism & Mass Communication Quarterly 94(4), 1031–1055 (2017). doi:10.1177/1077699016679976. Accessed 2018-04-20
  • [19] Liu, N., An, H., Gao, X., Li, H., Hao, X.: Breaking news dissemination in the media via propagation behavior based on complex network theory. Physica A: Statistical Mechanics and its Applications 453, 44–54 (2016). doi:10.1016/j.physa.2016.02.046. Accessed 2019-08-07
  • [20] McCombs, M.: A look at agenda-setting: Past, present and future. Journalism studies 6(4), 543–557 (2005)
  • [21] Lim, J.: First-level and second-level intermedia agenda-setting among major news websites. Asian Journal of Communication 21(2), 167–185 (2011). doi:10.1080/01292986.2010.539300. Accessed 2020-01-30
  • [22] Harcup, T., O’Neill, D.: What is News? Journalism Studies 18(12), 1470–1488 (2017). doi:10.1080/1461670X.2016.1150193. Accessed 2019-11-21
  • [23] Welbers, K., Atteveldt, W.v., Kleinnijenhuis, J., Ruigrok, N.: A Gatekeeper among Gatekeepers. Journalism Studies 19(3), 315–333 (2018). doi:10.1080/1461670X.2016.1190663. Accessed 2020-02-04
  • [24] Peng, S., Zhou, Y., Cao, L., Yu, S., Niu, J., Jia, W.: Influence analysis in social networks: A survey. Journal of Network and Computer Applications 106, 17–32 (2018). doi:10.1016/j.jnca.2018.01.005. Accessed 2018-02-19
  • [25] Kiss, C., Bichler, M.: Identification of influencers — Measuring influence in customer networks. Decision Support Systems 46(1), 233–253 (2008). doi:10.1016/j.dss.2008.06.007. Accessed 2018-04-27
  • [26] Romero, D.M., Galuba, W., Asur, S., Huberman, B.A.: Influence and Passivity in Social Media. In: Proceedings of the 20th International Conference Companion on World Wide Web. WWW ’11, pp. 113–114. ACM, New York, NY, USA (2011). doi:10.1145/1963192.1963250. http://doi.acm.org/10.1145/1963192.1963250 Accessed 2018-04-27
  • [27] Weng, J., Lim, E.-P., Jiang, J., He, Q.: TwitterRank: Finding Topic-sensitive Influential Twitterers. In: Proceedings of the Third ACM International Conference on Web Search And Data Mining. WSDM ’10, pp. 261–270. ACM, New York, NY, USA (2010). doi:10.1145/1718487.1718520. http://doi.acm.org/10.1145/1718487.1718520 Accessed 2018-04-27
  • [28] Ver Steeg, G., Galstyan, A.: Information-theoretic Measures of Influence Based on Content Dynamics. In: Proceedings of the Sixth ACM International Conference on Web Search And Data Mining. WSDM ’13, pp. 3–12. ACM, New York, NY, USA (2013). doi:10.1145/2433396.2433400. http://doi.acm.org/10.1145/2433396.2433400 Accessed 2018-04-27
  • [29] Stern, S.O., Tuckett, D., Smith, R.E., Nyman, R.: Measuring the Influencers in the News Media’s Narratives. In: 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis And Mining (ASONAM), pp. 698–701 (2018). doi:10.1109/ASONAM.2018.8508540
  • [30] Hatfield, E., Cacioppo, J.T., Rapson, R.L.: Emotional Contagion. Current Directions in Psychological Science 2(3), 96–100 (1993). doi:10.1111/1467-8721.ep10770953. Accessed 2018-05-07
  • [31] Moors, A.: Theories of emotion causation: A review. Cognition and Emotion 23(4), 625–662 (2009). doi:10.1080/02699930802645739. Accessed 2018-05-10
  • [32] Van Kleef, G.A., Van Doorn, E.A., Heerdink, M.W., Koning, L.F.: Emotion is for influence. European Review of Social Psychology 22(1), 114–163 (2011). doi:10.1080/10463283.2011.627192. Accessed 2018-05-09
  • [33] Stieglitz, S., Dang-Xuan, L.: Emotions and Information Diffusion in Social Media—Sentiment of Microblogs and Sharing Behavior. Journal of Management Information Systems 29(4), 217–248 (2013). doi:10.2753/MIS0742-1222290408. Accessed 2018-03-07
  • [34] Ferrara, E., Yang, Z.: Measuring Emotional Contagion in Social Media. PLOS ONE 10(11), 0142390 (2015). doi:10.1371/journal.pone.0142390. Accessed 2018-04-27
  • [35] Xiong, X., Li, Y., Qiao, S., Han, N., Wu, Y., Peng, J., Li, B.: An emotional contagion model for heterogeneous social media with multiple behaviors. Physica A: Statistical Mechanics and its Applications 490, 185–202 (2018). doi:10.1016/j.physa.2017.08.025. Accessed 2018-04-27
  • [36] Coviello, L., Sohn, Y., Kramer, A.D.I., Marlow, C., Franceschetti, M., Christakis, N.A., Fowler, J.H.: Detecting Emotional Contagion in Massive Social Networks. PLOS ONE 9(3), 90315 (2014). doi:10.1371/journal.pone.0090315. Accessed 2018-05-07
  • [37] Hui, P., Gregory, M.: Quantifying Sentiment and Influence in Blogspaces. In: Proceedings of the First Workshop on Social Media Analytics. SOMA ’10, pp. 53–61. ACM, New York, NY, USA (2010). doi:10.1145/1964858.1964866. http://doi.acm.org/10.1145/1964858.1964866 Accessed 2018-04-27
  • [38] Song, X., Chi, Y., Hino, K., Tseng, B.: Identifying Opinion Leaders in the Blogosphere. In: Proceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management. CIKM ’07, pp. 971–974. ACM, New York, NY, USA (2007). doi:10.1145/1321440.1321588. http://doi.acm.org/10.1145/1321440.1321588 Accessed 2018-04-30
  • [39] Zhou, H., Zeng, D., Zhang, C.: Finding leaders from opinion networks. In: 2009 IEEE International Conference on Intelligence and Security Informatics, pp. 266–268 (2009). doi:10.1109/ISI.2009.5137323
  • [40] Eliacik, A.B., Erdogan, N.: Influential user weighted sentiment analysis on topic based microblogging community. Expert Systems with Applications 92, 403–418 (2018). doi:10.1016/j.eswa.2017.10.006. Accessed 2018-03-15
  • [41] Brady, W.J., Wills, J.A., Jost, J.T., Tucker, J.A., Van Bavel, J.J.: Emotion shapes the diffusion of moralized content in social networks. Proceedings of the National Academy of Sciences of the United States of America 114(28), 7313–7318 (2017). doi:10.1073/pnas.1618923114. WOS:000405177100063
  • [42] Hill, A.L., Rand, D.G., Nowak, M.A., Christakis, N.A.: Emotions as infectious diseases in a large social network: the SISa model. Proceedings of the Royal Society of London B: Biological Sciences 277(1701), 3827–3835 (2010). doi:10.1098/rspb.2010.1217. Accessed 2018-05-07
  • [43] Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet Allocation. J. Mach. Learn. Res. 3, 993–1022 (2003). Accessed 2018-05-13
  • [44] Joly, J.: Disentangling media effects: The impact of short-term and long-term news coverage on Belgian emergency assistance. Cooperation and Conflict 51(4), 428–446 (2016). doi:10.1177/0010836716640829. Accessed 2019-10-11
  • [45] Tuckett, D., Smith, R.E., Nyman, R.: Tracking phantastic objects: A computer algorithmic investigation of narrative evolution in unstructured data sources. Social Networks 38, 121–133 (2014). doi:10.1016/j.socnet.2014.03.001. Accessed 2018-04-24
  • [46] Granger, C.W.J.: Investigating Causal Relations by Econometric Models and Cross-spectral Methods. Econometrica 37(3), 424–438 (1969). doi:10.2307/1912791. Accessed 2018-05-13
  • [47] Rahman, M.M., Wang, H.: Hidden Topic Sentiment Model. In: Proceedings of the 25th International Conference on World Wide Web. WWW ’16, pp. 155–165. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland (2016). doi:10.1145/2872427.2883072. https://doi.org/10.1145/2872427.2883072 Accessed 2018-04-24
  • [48]

    Mimno, D., Wallach, H.M., Talley, E., Leenders, M., McCallum, A.: Optimizing Semantic Coherence in Topic Models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. EMNLP ’11, pp. 262–272. Association for Computational Linguistics, Stroudsburg, PA, USA (2011). event-place: Edinburgh, United Kingdom.

    http://dl.acm.org/citation.cfm?id=2145432.2145462 Accessed 2019-11-21
  • [49] Hogg, M.A.: Intergroup Relations. In: DeLamater, J., Ward, A. (eds.) Handbook of Social Psychology. Handbooks of Sociology and Social Research, pp. 533–561. Springer, Dordrecht (2013)
  • [50] Shoemaker, P.J., Vos, T.P.: Media gatekeeping. In: An Integrated Approach to Communication Theory and Research, pp. 89–103. Routledge, ??? (2014)
  • [51] Latora, V., Marchiori, M.: Efficient Behavior of Small-World Networks. Physical Review Letters 87(19), 198701 (2001). doi:10.1103/PhysRevLett.87.198701. Accessed 2018-10-10
  • [52] Watts, D.J., Strogatz, S.H.: Collective dynamics of ‘small-world’ networks. Nature 393(6684), 440–442 (1998). doi:10.1038/30918. Accessed 2018-12-23
  • [53] Humphries, M.D., Gurney, K.: Network ‘Small-World-Ness’: A Quantitative Method for Determining Canonical Network Equivalence. PLoS ONE 3(4) (2008). doi:10.1371/journal.pone.0002051. Accessed 2018-10-10
  • [54] Telesford, Q.K., Joyce, K.E., Hayasaka, S., Burdette, J.H., Laurienti, P.J.: The Ubiquity of Small-World Networks. Brain Connectivity 1(5), 367–375 (2011). doi:10.1089/brain.2011.0038. Accessed 2018-09-28
  • [55] Neal, Z.: Making Big Communities Small: Using Network Science to Understand the Ecological and Behavioral Requirements for Community Social Capital. American Journal of Community Psychology 55(3), 369–380 (2015). doi:10.1007/s10464-015-9720-4. Accessed 2018-12-13
  • [56] Fagiolo, G., Moneta, A., Windrum, P.: A Critical Guide to Empirical Validation of Agent-Based Models in Economics: Methodologies, Procedures, and Open Problems. Computational Economics 30(3), 195–226 (2007). doi:10.1007/s10614-007-9104-4. Accessed 2018-11-08
  • [57] Rubinov, M., Sporns, O.: Complex network measures of brain connectivity: Uses and interpretations. NeuroImage 52(3), 1059–1069 (2010). doi:10.1016/j.neuroimage.2009.10.003. Accessed 2019-08-15
  • [58] Liao, W., Ding, J., Marinazzo, D., Xu, Q., Wang, Z., Yuan, C., Zhang, Z., Lu, G., Chen, H.: Small-world directed networks in the human brain: Multivariate Granger causality analysis of resting-state fMRI. NeuroImage 54(4), 2683–2694 (2011). doi:10.1016/j.neuroimage.2010.11.007. Accessed 2019-07-31
  • [59] Mutz, D.C., Martin, P.S.: Facilitating Communication across Lines of Political Difference: The Role of Mass Media. The American Political Science Review 95(1), 97–114 (2001). Accessed 2018-12-20

Appendix A

Figure 7: The inferred topics with the 20 highest coherence values