Sentiment Dynamics in Social Media News Channels

08/21/2019 ∙ by Nagendra Kumar, et al. ∙ Indian Institute of Technology Hyderabad Columbia University Carnegie Mellon University 0

Social media is currently one of the most important means of news communication. Since people are consuming a large fraction of their daily news through social media, most of the traditional news channels are using social media to catch the attention of users. Each news channel has its own strategies to attract more users. In this paper, we analyze how the news channels use sentiment to garner users' attention in social media. We compare the sentiment of social media news posts of television, radio and print media, to show the differences in the ways these channels cover the news. We also analyze users' reactions and opinion sentiment on news posts with different sentiments. We perform our experiments on a dataset extracted from Facebook Pages of five popular news channels. Our dataset contains 0.15 million news posts and 1.13 billion users reactions. The results of our experiments show that the sentiment of user opinion has a strong correlation with the sentiment of the news post and the type of information source. Our study also illustrates the differences among the social media news channels of different types of news sources.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 9

page 11

page 12

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Television, radio and print media are the three primary types of news sources in the world. Due to the unique nature of each communication medium and the manner in which their audience consume the information, they show a significant difference in their news articles. Despite these differences, all these sources are commonly accused of using exaggerated headlines to garner attention ecker2014effects and for focusing on negative news soroka2015news. Today, social media has emerged as a powerful platform for consumption of news with 62% of U.S. adults reportedly getting their news from social media gottfried2016news. Therefore, traditional news channels have started generating and disseminating news through various social media platforms.

As growing number of people consume, share and discuss news online, it is important to understand whether the lack of regulation inherent in social media is being exploited to spread more aggressive and negative news. We surprisingly found that news is not necessarily negative across all the news channels. Rather, there is considerable variation in the way news is presented by different news channels and is heavily dependent on the medium through which these channels traditionally disseminated news, namely radio, TV or print media. Print and radio media based channels post more positive news while TV based channels post more negative news. The difference is not only because of difference in the type of stories covered but because of how the same news is presented by different types of channels. For instance, we can observe from Table 1 how the Dakota Access Pipeline protest, a major movement in the Northern United States to protect natural resources and spiritual sites, reported by CNN and The Economist are extreme in tone, but opposite in polarity (we measure polarity on the scale of -5 to +5) despite being posted at the same time and referring to the same incident.

Organization News Polarity
CNN We’re at the Standing Rock Sioux Camp in North Dakota. Protesters here are fighting to block the Dakota Access Pipeline and have vowed to stand their ground — despite growing calls for them to leave camp and threats of prosecution from law enforcement. Any questions for CNN’s Sara Sidner? -5
The Economist Whatever the final result of the huge, long-running protests by native Americans against the Dakota Access Pipeline, the demonstrations will surely be remembered as a landmark in relations between organised religion, Christianity in particular, and indigenous people +3
Table 1: Polarity of a news generated by two different types of channels

As opposed to one-to-many communication structure of traditional media, social media enables many-to-many communication by allowing the users to engage with the news articles by liking, sharing and commenting on them. The number of likes, shares, and comments received by a news are good objective measures of engagement and provide insight into what type of news interests users. We use this information to understand to what extent the sentiment policy employed by news media have been successful in catching users’ attention. We show that negative news receive more number of comments and shares compared to positive news, which gets more number of likes compared to negative news. This finding is extremely interesting as it agrees with the popular ‘negativity bias’ theory111A popular theory in social psychology that states that humans are more likely to focus on bad news rozin2001negativity. for actions that require greater involvement such as sharing and commenting but does not agree for the relatively simpler actions such as liking the news article.

Comments allow users to express their opinion regarding a news item. These opinion can be used for opinion mining to gather information on how users perceive the news, predict real-world outcomes, gain useful insight into users’ collective behavior, etc. These mining tasks often involve aggregating the users’ opinions from different news channels, which may potentially bias the result because users’ opinion on a topic depends on many factors cambria2013new; vinodhini2012sentiment, such as the region where the news is published, the time when it is published, the type of information source that published the news, the sentiment with which the news is written, etc. In this paper, we analyze how users’ opinion depend on two of these factors: sentiment of the news article and type of news channel. We obtain interesting insights that can be used to correct the bias arising due to the wavering nature of users’ comments. To the best of our knowledge, very few studies bautin2008international; godbole2007large have been devoted to the sentiment analysis of news articles and none has arrived to study the role of information sources coupled with the sentiment of news on the users’ perception of the news. To gain more insight into the factors affecting the sentiment of news, we also categorize the news based on the topic, time, and their significance.

Some of our major findings show that sentiment generated by social media channels of different types of information sources are different and most of the time, these news channels generate either positive or negative news. Surprisingly, print and radio based channels generate predominantly positive news on their social media pages disagreeing with the popular opinion patterson2011out; stieglitz2013emotions that news sites mostly post negative news to take advantage of the negativity bias. We also found that negative news were shared and commented on more often but positive news were liked more often throwing light on how the negativity bias operates at a different level of user engagement. Additionally, we show that polarity of the comments is strongly related to the sentiment polarity of the news. As news become more negative, their comments also become more negative in tone and vice versa. News from TV based news channels prompt more negative reactions from news compared to print and radio based channels, suggesting that people react not only to the type of news article but also to the source of the news article.

Our key contributions are as follows:

  • We analyze sentiment of posts created by news channels in their social media pages.

  • We investigate users’ reactions to news posts of varying sentiment from different types of news channels.

  • We categorize the news posts into different topics to gain insight into the sentiment of news posts created under these topics.

  • We compare the sentiment of big news headlines with niche news to investigate how big headlines impact sentiments of the news posts.

  • We explicate the relationship between sentiment polarity of news articles and the polarity of textual reactions in conjunction with type of information source.

  • We perform temporal analysis to investigate the posting behavior of news articles having different sentiments over a period of time.

  • We perform our experiments on a large dataset containing 1.13 billion users reactions and 0.15 million news articles.

The rest of the paper is organized as follows. In Section 2, we briefly survey the related work. Section 3 presents our methodologies. Section 4 analyzes the polarity of different types of social media news channels. Section 5 describes the relationship between the popularity and polarity of news articles. In section 6, we analyze the users’ opinions on different types of channels with varying post sentiments. We present temporal analysis in Section 7 and conclude our work in Section 8.

2 Related Work

Online news content analysis is an active research topic in Computer Science. One of the major sub-topic is to predict the popularity of news content ahmed2013peek; bandari2012pulse; lee2010approach; lerman2010using; szabo2010predicting; tatar2014popularity. Lerman et al. lerman2010usingestimated the overall popularity of a content based on early users’ reactions on it. However, Bandari et al. bandari2012pulse predicted the popularity of a content without using early users’ reactions. They considered multiple features of a news post to predict its popularity prior to its posting, such as subjectivity of the news, category of the news, etc. In this paper, we study the relationship between news post sentiment and the news post popularity across different types of news channels. Unlike popularity prediction studies, we analyze the sentiment of news articles, and it’s effect on popularity and users’ opinions.

Several approaches arapakis2014user; coscia2014average; hansen2011good; naveed2011bad; wu2011does have been proposed to analyze the news propagation. Naveed et al. naveed2011bad

showed that negative news is more attractive to users and easily catch their attention. They have used 15 different set of content-based features and predict the likelihood of a tweet being retweeted using logistic regression. Wu et al. 

wu2011does

showed that the lifetime of negative news is very short but positive news stay for a longer time. They predict the decay of social media content using a classification technique namely, support vector machine. Coscia et al. 

coscia2014average studied the content of news and showed that news content should be significantly different compared to already existing content, as unique content spreads faster than average or existing content. However, we show that it is not only the negative news that catch user attention but also positive news can garner a lot of user attention. Popularity gained by both positive and negative news is usually higher than neutral news.

Recently, researchers reis2014magnet at UFMG developed a tool that present news to users based on their interest or polarity. They ranked the news articles based on their popularity and sentiment score. Reis et al. dos2015breaking analyzed the news headlines from a popular global news channel. They showed that sentiment of headline correlates with the popularity of news and negative comments are posted independently of the sentiment score of the headlines. Diakopoulos and Naaman diakopoulos2011topicality analyzed the relationships between news comment topicality, temporality, sentiment, and quality. They showed that comment sentiments (positive and negative) are the important indicators of discourse quality. Our analysis is complementary to existing studies, which shows that polarity of comment is not completely independent of polarity actual post; it is a function of the polarity of the news post.

Further, a work published on the social news on the web222http://snow2013.isti.cnr.it/?cat=4, dealt with the problem of finding the topics of a news channel and users’ interests in social media zubiaga2013newspaper. Authors of the paper performed analysis on The New York Times channel and showed that news stories selected by editors for their newspaper or channel are far from those catching users’ attention on social media. However, in this paper, we perform sentiment analysis on multiple social media news channels and show the top topics of editor interests as well as sentiment associated with each topic. Unlike the findings reported by Zubiaga zubiaga2013newspaper that news channels mainly report hard news with top priority, we have found that news channels uniformly generate hard news (e.g., politics, money, world) as well as soft news (e.g., entertainment, lifestyle, sports, science and technology) through their social media pages. Another study by Nies et al. de2013towards showed the impact of an actual news article on social media. They compared the relevancy of an arbitrary news article with social media news to measure its impact. However, we analyze the impact of social media news content with the varying sentiment on social media users. We show how a news with different sentiments shapes the users’ opinion about the news content.

Compared to existing works, our focus is on analyzing the sentiments of news posts in social media and users’ reactions to it. We examine the correlation between the polarity of comments and the sentiment polarity of posts. Unlike previous studies burke2016once; castillo2014characterizing; cheng2014community; d2013dont; moosa2014comment, our analysis shows that the sentiment of comment is a function of the news post sentiment and the type of channel through which news was traditionally disseminated.

3 Methodology

In this section, we first describe the process of collecting the news from Facebook pages of news channels. We then describe the method employed to measure the sentiment polarity. Next, we present the method to categorize the news posts. In the rest of the paper, we use terms such as ‘post’, ‘news post’, ‘news content’, and ‘news article’ interchangeably.

3.1 News Posts Collection

In order to characterize the news posted on social media, we collected the news from Facebook pages of five major news media channels. Our choice of Facebook over other social networking platforms is informed by research from Pew Research center, which notes that Facebook has the highest reach with 44% of adults in the US getting news on the platform gottfried2016news. Further, we chose news sites with the highest valuation as calculated by Virtue’s Social Page Evaluator333http://www.adamsherk.com/social-media/most-valuable-news-site-facebook-pages/. In order to understand the differences between different media, we choose two each of television and print media based channels and one radio based channel. Dataset thus includes posts from the Facebook pages of CNN and Fox News which are television news channels, The Economist and The New York Times (or NYT) which are daily and weekly newspaper organizations respectively, and NPR which is a public radio network.

We extract the dataset444The dataset will be made available for download from the author’s website. from Facebook pages using the Facebook Graph API facebook:graph. The dataset contains news articles posted by the pages, reactions on the post, link to the original news article and attributes including the number of users who liked the page, organization name, post creation time, reaction time, etc. Users can react to posts created by pages in the form of like, comment and share. Reactions consist of textual comments and rating score in the form of likes and shares. For each news channel, we present the number of posts, comments, likes, shares, and time interval in the collected news dataset as follows:

News Channels Posts Comments Likes Shares Time Interval
CNN 33324 26582081 147310056 52936764 Dec 2016-April 2012
NPR 18266 4585776 56007054 18847580 Dec 2016-Nov 2013
Fox News 26525 83957661 443933576 143762565 Dec 2016-Jan 2014
The Economist 24272 1336956 20206137 6376644 Dec 2016-Dec 2014
NYT 47522 9226029 93891025 25616593 Dec 2016-April 2013
Table 2: Dataset Statistics

From the Table 2, we can infer that Fox News is the most popular news channel as it has the highest reaction per post ratio. Whereas The Economist is the least popular news channel among all the news channels as it has the lowest reaction per post ratio. Here, the reaction is the popularity measure which is the sum of likes, comments, and shares. We also perform pre-processing to remove noisy and unimportant words from the textual posts and comments. It avoids the trivial words, which appear frequently in the posts. We remove stop-words, such as ‘a’, ‘an’, ‘the’, etc., as these words do not contain significant information for our analysis. We also employ stemming and lemmatization uysal2014impact to reduce inflected or derived words to their root forms. Throughout the rest of the paper, we consider a common time frame from December 2014 to December 2016 for our analysis.

3.2 Sentiment Polarity Identification

In social media, users use informal language to present their textual contents which differentiate social media texts different from standard texts. We are listing a few examples of the usage of informal language as follows:

  • Social media texts especially comments usually contain emoticons such as :), :(, :-), |-o, etc.

  • Increasing number of users use acronyms such as LOL, smh, ty, wth, etc.

  • Social media users use slang words very often, and these words became a part of social media lexicon. For example, meh, yep, giggly, nah are few commonly used slang words.

  • Users also use multiple punctuation marks to emphasize the certain words in a text sentence.

In order to tackle all the above-mentioned issues, we have used Valence Aware Dictionary and sEntiment Reasoner (VADER), which is a powerful sentiment analyzer to find the sentiment of social media texts hutto2014vader. VADER is a lexicon and rule-based sentiment reasoner and is the best suited for sentiment analysis of contents originating in social media tamersoy2015characterizing. It creates and utilizes a new gold standard sentiment lexicon with 7500 lexical features that are commonly used to express the sentiment in a social media text. It uses the rule based method consisting of five rules that embody grammatical and syntactical conventions for expressing the sentiment intensity555https://github.com/cjhutto/vaderSentiment. VADER has been compared with 11 sentiment analysis tools/techniques including SentiWordNet baccianella2010sentiwordnet, SenticNet cambria2012senticnet, LIWC pennebaker2001linguistic. It is shown that VADER outperforms all of them. Further, VADER provides a sentiment score in the range of -1 to +1, with -1 being extremely negative, +1 being extremely positive and, 0 being neutral. For the sake of interpretability, we convert these polarity scores to an integer between -5 to +5. The polarity scores inferred were as expected and a sample of the same can be observed in Table 3.

Score Sample Post
+5 It’s just an amazing thing to watch good old-fashioned regular human beings and a whole lot of love change the world seismically
+4 Follow the Queen’s Diamond Jubilee celebrations with the latest photos, videos, facts and trivia. Tell us which part of the festivities you’re most impressed with
+3 When you do something extraordinary, it’s shown that you can inspire other people." #CNNHeroes
+2 The world’s first permanent ice hotel has opened in Sweden, thanks to new solar-powered cooling technology
+1 Farmers in the Australian desert are growing 15,000 tons of tomatoes using seawater — and thousands of mirrors
0 In a tweet, President-elect Donald J. Trump says his businesses won’t do any new deals while he’s in office
-1 Between 2007 and 2014, 30% of African elephants disappeared
-2 Being exposed to the daily hassles of traffic can lead to higher chronic stress and higher blood pressure,” according to a recent study conducted in Texas
-3 Are we on the verge of a second Cold War?
-4 Terror attacks have ripped apart small towns and big cities across the Middle East and Africa throughout 2016, and this weekend was no different
-5 A young newlywed couple died a horrible death at the hands of the bride’s family
Table 3: Sentiment polarity of sample posts

3.3 News Posts Categorization

To get insight into posting behaviour of news channels across the categories, it is useful to categorize the news posts into multiple categories such as sport, entertainment, politics, science and technology (sci&tech), etc. Unlike online news sites, news posted on social media channels is not categorized. In order to categorize these news posts, we use unsupervised method LDA blei2003latent

. LDA is a probabilistic topic modeling algorithm which represents each document (in this case, a news post) as a mixture of various topics with definite probabilities (

). A topic is comprised of words or terms. The terms that often occur together, are placed under the same topic with high probabilities (). Document-topic distribution () and term-topic () are computed using Gibbs sampling as follows:

(1)
(2)

where , and represent the number of topics, documents, and smoothing constant respectively. is the number of times a term appears in document that has been assigned to topic . , and represent the number of terms, topics, and smoothing constant respectively. is the number of occurrences of a word that has been assigned to topic . Gibbs sampling method integrate these two assignments and update the topic assignment until convergence.

In order to make the topic modeling richer, we augment the post message with its URL information obtained using the Graph API. The augmented text from external documents using URL was later processed to remove invalid characters and corrected for mistakes in spelling. To determine the ideal number of topics , we perform 5-fold cross validation on perplexity at different values of . We then compute the rate of perplexity change (RPC) zhao2015heuristic on a 10% random sample. Perplexity is a statistical measure and often used to measure the performance of topic models Griffiths5228. Perplexity reflects the capacity of a model to generalize to test set or unseen posts. The point where the rate of perplexity no longer falls significantly with an increase in the number of topics is used as the ideal number of topics. In our experiment, we find the optimum value of is 10 from where perplexity does not change significantly.

As studied by Chang et al. chang2009reading that perplexity and human judgment are not well correlated, we evaluate our topics manually using precision agrawal2009diversifying. We ask five research scholars having knowledge of topic modeling to judge the relevancy of topical words generated by the topic model. We ask research scholars to label each topical word as relevant or non-relevant to assigned topic by the topic model. Topics are labeled by researchers independently without influencing each other. Topics for which researchers did not agree on were discussed until a consensus was reached. We then compute precision as a fraction of generated topical words that are relevant to the assigned topic. We find that the topic model performs reasonably well with 80.3% precision. One of the reasons for this is that number of topics selected for LDA categorization is the best suited for Facebook news posts dataset. Moreover, posts created by news channels in their social media pages are well framed unlike user-generated contents such as comments, tweets, etc.

Further, we provide the label for each topic based on the most relevant terms that uniquely define the topic. Since each topic contains thousands of terms, we extract top relevant words based on term-topic distribution. Relevance () of a term to topic is computed as follows:

(3)

where is the term-topic probability, and is the empirical probability of the word in the corpus. is a weighting term and we choose 0.6 as an optimal value for as shown by Sievert et al. sievert2014ldavis. We assign each post or document to these labeled topics based on document topic probability (). If document shows the highest probability for topic , is assigned to topic .

Figure 1: Distribution of news posts across categories

Figure 1 shows the distribution of posts for each channel across categories. We observe that post distribution of TV based channels CNN, Fox News is almost similar where politics, crime categories contain a higher percentage of posts. In case of print media based channels like The New York Times and The Economist, lifestyle, money, politics, and crime news seem to be more common. On the other hand, NPR which is a Radio based channel, posts lifestyle news most often followed by entertainment (entertain). We show how channels post the news in these categories in Section 4.1.

4 Analysis of News Posts Polarity

We begin our investigation by analyzing the distribution of polarity of post messages grouped as positive, negative and neutral for the social media news channels as discussed in Section 1.

Figure 2: Polarity of news posts generated by pages

A quick glance at Figure 2 reveals that the dominant sentiment in posts by all the news channels is always either positive or negative but not neutral. Moreover, posts with neutral sentiment are least common in Facebook pages of all media sites except NPR. These inferences support the claim that all the channels tend to generate more positive or negative messages on their Facebook pages to attract the users’ attention.

Another interesting aspect to be noted is the similarity in the distribution of sentiment polarity of posts between media channels that function through the same medium of communication. Posts by television based news channels, such as Fox News and CNN, are predominantly negative where Fox News generates the highest percentage (40%) of negative news across all the channels. On the other hand posts by radio and print media based channels such as NPR, The Economist and The New York Times are mostly positive. Radio based news channel, NPR generates the highest percentage (43%) of positive news and the least percentage of negative news (28%) across all the channels. Print based media channels, The Economist and The New York Times generate a large proportion of positive news and a less proportion of neutral news. Despite the similar pattern of news generation by these two channels, The Economist generates relatively a larger percentage of both the positive and negative news compared to the neutral news. One of the reasons for this is that The Economist reports growth (i.e. positive news) and decline (i.e. negative news) in business, commerce, and trade substantially. To investigate how different types of channels report the same news, we have also performed the sentiment experiment on the same news events covered by these channels. However, we did not notice a significant difference in the sentiment compared to Figure 2. We have observed the sentiment pattern similar to Figure 2 for all the channels.

The news reported by news sources has evolved differently because of the manner in which users consume the information in each medium bennett2016news. Our analysis suggests that these differences remain despite disseminating information on a common platform. News media are often criticized for their focus on negative news rather than providing a balanced picture of the world hansen2011good; budak2016fair; leetaru2011culturomics. This phenomenon has been attributed to journalistic cynicism and inherent preference for negative news among users. However, we observe through our analysis, that print and radio based social media channels post more positive news than negative news. This finding raises important questions: Is this change in the type of content posted by print and radio based channels precipitated by user’s preference for positive news on social networking platforms? Does this mean that the popular negativity bias theories rozin2001negativity; baumeister2001bad, which state that humans have a predilection for negativity, not hold true in the case of news consumption in social media? We attempt to answer these questions in Section 5.

4.1 News Posts Polarity across Categories

In this section, we analyze the polarity of news posts across categories to investigate how news channels generate the news across categories. We compare the polarity of news generated in multiple categories such sports, politics, health, entertainment, etc. We observe that channels from the similar type of sources show the similar pattern. Due to space constraints, we present the results of only one news channel of each type of information sources such as print, television, and radio as follows:

Figure 3: Fox News
Figure 4: The Economist
Figure 5: NPR

It can be observed from Figures 4 and 4 that news belonging to the crime, world and health categories are predominantly negative, for both print and television based channels. One of the reasons for this is that most of the times news related to crime is woeful and unpleasant. The news related to health and world easily catch the attention of the channels if any terrible accident takes place across the world. However, in case of NPR (Figure 5), which is a radio based channel, all types of news except crime news, are predominantly positive in tone. This suggests the possibility that the trend in dominant sentiment observed in the Section 4 could also be a result of the same type of news being covered differently by different news channels depending on the primary medium. Thus, by analyzing the sentiments across categories for different organizations, we can conclude that both the differences in the type of news that is often covered, and the difference in the tone with which the same news is covered are responsible for the difference in dominant sentiment observed in the previous section.

Moreover, except for NPR, across all categories, the proportion of news that is neutral in tone is the smallest. It indicates that trend of a higher fraction of positive or negative news, which we observed in the Section 4, is not limited to a few categories but is one of the tactic that is adapted for the generation of all types of news. NPR, however, stands out with negative news being the least common in majority of the categories. This observation is also consistent with the predominantly positive nature of news generated by NPR that we observed in the previous section. Moreover, the similarity in the sentiment distribution for news channels using the same medium further asserts the influence of the medium on the tone with which news is disseminated by channels.

4.2 Big headlines versus niche news

In this section, we compare the polarity of big headlines and niche news. We analyze the differences in the sentiment of news posts reported by different types of channels for big headlines as well as niche news. We report the findings in the following figures:

Figure 6: Big headlines
Figure 7: Niche news

As can be seen in Figures 7 and 7, the sentiment polarity of big headlines are different from the polarity of niche news. All the channels generate a higher percentage of positive headline news compared to negative and neutral news (refer to Figure 7). Among all the channels, NPR generates the highest percentage (43%) of the positive headline news. One of the reasons for generating a higher percentage of positive news is that these big headlines are very popular and exist for a longer time. If news channels continuously generate a higher fraction of negative news for these types of events, users may lose their interests and it would lead to less engagement on these news channels. Negative news goes away very fast wu2011does and if there is a big headline that is usually persistent for some time, channels create more number of positive news to maintain the sustainability. As studied by researchers in psychology johnston1997psychological that negative news causes worries to users, channels generate more positive news about the headline to retain the users’ interests over time. On the other hand, we do not observe a significant difference in the polarity of niche news as compared to Figure 2. Positive or negative news are more popular than the neutral news. TV based channels report more negative news compared to the radio and print media based channels whereas print and radio based channels report more positive news.

4.3 Polarity of Same News Events across Channels

In this section, we examine how different types of news channels report the same news. We analyze sentiments of ten different real-world news events that are posted by social media news channels. In Table 4, positive indicates the percentage of positive news created for the event and negative indicates the percentage of negative news created for the event. Due to brevity, we did not mention the neutral news, which can be determined by subtracting the sum of positive and negative news from hundred.


News Channel /
News
CNN
Fox News
NPR
NYT
The Economist

Presidential Election
Positive: 42.3%
Negative: 22.5%
Positive: 61.5%
Negative: 18.2%
Positive: 55.3%
Negative: 8.1%
Positive: 48.7%
Negative: 18.7%
Positive: 41.9%
Negative: 23.8%


Same-sex Marriage
Positive: 45.6%
Negative: 32.8%
Positive: 52.3%
Negative: 40.7%
Positive: 45.2%
Negative: 40.4%
Positive: 60.5%
Negative: 19.8%
Positive: 65.3%
Negative: 18.6%

Obamacare
Positive: 50.3%
Negative: 32.5%
Positive: 42.1%
Negative: 30.6%
Positive: 45.2%
Negative: 21.7%
Positive: 41.3%
Negative: 35.6%
Positive: 65.2%
Negative: 16.8%
Football
Positive: 35.7%
Negative: 40.5%
Positive: 38.9%
Negative: 42.4%
Positive: 45.2%
Negative: 18.7%
Positive: 45.3%
Negative: 35.6%
Positive: 52.3%
Negative: 31.6%
Dakota Access Pipeline
Positive: 37.4%
Negative:42.3%
Positive: 38.5%
Negative:40.9%
Positive: 60.4%
Negative: 25.7%
Positive: 39.4%
Negative: 38.5%
Positive: 50.6%
Negative: 28.3%
US Ambassador
Positive: 32.3%
Negative:45.7%
Positive: 37.7%
Negative:48.6%
Positive: 35.3%
Negative: 25.6%
Positive: 48.5%
Negative: 41.3%
Positive: 41.4%
Negative: 35.4%
Hollywood
Positive: 32.4%
Negative:42.8%
Positive: 35.2%
Negative:43.5%
Positive: 43.6%
Negative: 27.5%
Positive: 45.2%
Negative: 30.3%
Positive: 53.6%
Negative: 25.7%
Ebola
Positive: 39.4%
Negative:42.3%
Positive: 34.7%
Negative:39.8%
Positive: 38.6%
Negative: 46.5%
Positive: 28.3%
Negative: 45.7%
Positive: 25.4%
Negative: 56.8%
MH370 Flight
Positive: 15.3%
Negative:50.2%
Positive: 10.3%
Negative:60.4%
Positive: 6.8%
Negative: 75.6%
Positive: 8.4%
Negative: 60.3%
Positive: 5.3%
Negative: 66.8%
Zika Virus
Positive: 19.4%
Negative: 41.6%
Positive: 34.2%
Negative: 47.5%
Positive: 28.8%
Negative: 50.3%
Positive: 10.4%
Negative: 48.8%
Positive: 31.5%
Negative: 40.3%
Table 4: Sentiment polarity of news events across channels

As can be seen in Table 4, different news channels report the same news events differently. Despite being generating the same news with different percentage of sentiment polarities, all the channels generate a large fraction of positive news for big headlines such as Presidential election, Same-sex Marriage, and Obamacare (refer Section 4.2). However, if a headline is very negative in nature such as MH370 Flight Disappearance, all the news channel generate a large fraction (more than 50%) of negative news due to nature of the news event. Also, all the channels generate a major fraction of negative news, if a news is related to flu epidemics such as Ebola and Zika Virus, an attack, and natural disaster.

Although all the big-headlines are reported with a similar pattern of sentiment, regular events and the events that are not part of big-headlines are reported differently by the channels. If a news is related to regular events or minor headlines (i.e., Football, US Ambassador, Hollywood, Dakota Access Pipeline), channels usually generate a different sentiment pattern of positive, negative and neutral news. In this case, positive or negative news are more popular than the neutral news. TV based channels report more negative news compared to the radio and print media based channels. On the other hand, print and radio based channels report more positive news. This observation is in line with our analysis in Section 4 and due to the majority of these regular events, we observe the similar pattern of sentiment in Figure 2.

Apart from the results that are shown in Table 4, we also observe that news related to major breakthrough in Science and Technology (i.e., news from NASA, MIT), highly reputed awards (i.e., Nobel Prize, Oscars) and persons (i.e., Pope, Dalai Lama) are reported significantly positive (more than 55%) across all the channels. One of the reasons is that these are very esteemed organizations, awards or persons. Due to the highly positive nature, all the channels generate a major fraction of the positive news related to these organizations, awards, and persons.

5 Popularity versus Polarity

We analyze the popularity of a news post as a function of its polarity. Affinity metrics such as comments, likes, and shares received on a post are good indicators of its popularity. However, each of these actions involves a different level of interaction and are assigned different weights in the Facebook newsfeed algorithm kim2017like with share receiving the highest weight and like the least. Hence, we do not aggregate these counts but analyze them separately. In order to account for the large difference in popularity of different news sites under consideration, we scale these counts of affinity metrics in the range of 0 to 1 and use the normalized values to determine the popularity of news posts.

Figure 8: Likes
Figure 9: Comments
Figure 10: Shares

We observe in Figure 8-10 that posts, which are either positive or negative, are more popular than the neutral ones in most of the cases. This suggests that news posts that are either positive or negative in tone tend to be more popular in social media. This finding is inline with observations made by Naveed et al. naveed2011bad stating that people are more attracted towards positive or negative news compared to neutral ones. Exceptions to this are Fox News only for comments and The Economist channel. One of the reasons for this is that The Economist reported a major fraction of the news related to money and lifestyle (refer to Figure 1). The Economist also reported the highest number of money or business related news compared to other news channels. We have observed that these news were mostly reported facts and figures (usually neutral in sentiment), which leads to a higher number of reactions for neutral news.

Further, in relation to preference between positive and negative content, we observe that more likes are received for the positive posts whereas more comments and shares are received for the negative posts. It shows that the results agree with “Negativity bias” for actions that involve a greater level of engagement such as commenting and sharing but disagrees with when it comes to simpler actions such as liking the post. Trussler and Soroka trussler2014negative performed an eye tracking experiment to understand consumer demand for negative news frames and found a similar result. They found that participants “said”, they preferred good news but in reality often chose negative news stories over positive ones. While it is apparent that negative news receives a greater level of engagement, it is important to understand whether users are engaging positively or negatively with content in order to design any plan to receive appropriate users’ opinions. We answer this question in the upcoming section.

6 User Opinion Analysis

In addition to indicating the popularity of the post, user opinion (or comment) can provide a great deal of information about the tone of the audience, which can conclude whether the post is being perceived positively or negatively. To understand how users respond to posts of different sentiment polarities, we determine average sentiment polarity of comments received for each of the posts. We show the relationship between average sentiment polarity of comments and sentiment polarity of posts as follows:

Figure 11: CNN
Figure 12: Fox News
Figure 13: The Economist
Figure 14: The New York Times

We observe from all the five channels that as posts become more and more positive, comments also become increasingly positive. As can also be seen in Table 16 that there is a strong correlation between the sentiment polarity of comments and the sentiment polarity of posts. Comments of all the three types of channels have high sentiment correlation with the posts, and among all the channels TV based channels show the highest correlation. To validate our findings, we perform p-test dahiru2008p, which shows correlations between posts and comments sentiments are significant at p < .05. We can thus infer that the posts written with varying levels of sentiment polarity prompt different reactions from users. A high correlation between post sentiment and comment sentiment suggests that measures of sentiment polarity of posts can be used to correct for biases that occur when aggregating comments from various channels for tasks such as opinion mining, opinion summarization, real-world outcome prediction, etc.

Figure 15: NPR
News Channel Correlation CNN 0.97 Fox News 0.98 The Economist 0.95 NYT 0.97 NPR 0.93
Figure 16: Correlation between post sentiment and comment sentiment

However, the polarity for which a post starts attracting negative comments varies based on the medium of the channel. While Facebook pages of TV based channels, on an average, attract negative comments for negative posts and vice versa, comments for print media based channels do not become negative until posts become strongly negative in tone (i.e. sentiment score less than -3). It is interesting to note that the average comment sentiment polarity of NPR, which posts the highest proportion of positive content amongst all the channels, remains positive irrespective of the polarity of post. It must be recalled from Section 4 that posts by TV based news channels were predominantly negative whereas those by print media and radio based channels were predominantly positive with radio based channel having the highest percentage of positive posts. This suggests that sentiment expressed in the comments is not only strongly influenced by the polarity of that particular post but also by users’ opinion about the channel posting the news. The user opinion about the news channel is in turn shaped by the whether the majority of the post messages have a positive or negative tilt. That is, Facebook pages of TV news channels which mostly post negative content attract more negative comments whereas channels that are positive in tone like print media and radio attract fewer negative comments.

7 Temporal Analysis

In this Section, we analyze the polarity of news posts temporally. We investigate how the polarity varies over the years. We also investigate whether posts of certain polarity drastically increase or decrease in particular months or days of the week. A common time frame from December 2014 to December 2016 is considered for the analysis. We present the results as follows:

Figure 17: The Economist
Figure 18: NPR

Figures 20-18 show the polarity of the news articles over the period of years. We observe that the behavior of television, print and radio based channels remain same as that revealed in Section 4, i.e. negative sentiment dominates in the television based channels, positive sentiment dominates in the radio and print media based ones majority of the time.

We observe that, over the time, Facebook pages of print based media channels (Fig. 18) show a gradual decrease in the percentage of positive posts while neutral posts increase and negative ones remain almost constant. As can also be seen in Figure 18 and 18, there is a peak of positive sentiment during the months of April to July 2015. One of the reasons for this is that two big headlines about Obamacare and same-sex marriage were in the trending news during that time. A few example news of these headlines are as follows: (1). The Supreme Court has ruled on Obamacare subsidies; (2). US supreme court declares same-sex marriage legal. These big news headlines lead to a sentiment peak in Figure 18 and 18, which is also in line with our analysis in Section 4.2 that report that news channels slightly generate more positive news for big headlines compared to negative news.

Figure 19: Fox News
Figure 20: CNN

Both the Television based channels, CNN and FOX exhibit a slight increase in the positive news over the negative news in the year 2016. Such instances of graphs of same-medium news sites (Fig. 20 and 20) showing similar trends suggest that they behave similarly and respond similarly to external events. One of the primary reasons for slightly dip in negative sentiment and a rise in positive sentiment is a big headline, namely US presidential election. These TV based channels are somewhat biased towards a party and generate positive news about that party. Another big headline news was Nobel peace prize award to Colombian president. These headlines are one of the reasons for slightly dip in negative news and rise in positive sentiment.

We also analyze the news posts over the months and weeks but we do not notice considerable variations in their polarities. On inspecting sentiment distribution over the months, we do not observe any consistent and significant change for certain months of the year. Even the weekly analysis does not reveal any significant change in the distribution except for a slight increase in positive posts on weekends. One of the reasons for this is that news channels have posted slightly more entertainment, lifestyle, and sports news during the weekend. This posting behavior of news channels leads to a slight increase in positive posts on weekends. Moreover, it is noted that the dominant sentiment in both the monthly and weekly analysis is also similar to the behavior of the channels observed in previous Section 4. Thus, by analyzing the sentiment temporally, we can conclude that, on average, the specific characteristics observed in Section 4 are exhibited consistently across the weeks, months and years.

8 Conclusion

In this paper, we conducted an extensive analysis on social media news channels from three types of news information sources to analyze the sentiment of the news generated by these channels and its effect on users’ reactions. We characterized the news in different categories to uncover the distribution of the news posts and their sentiment polarity across categories. Our analysis revealed that sentiment of the news posted by different types of social media channels is dependent on the medium through which these channels traditionally disseminated the news. We also investigated popularity of news with different polarity to get insight into the type of news that attract lots of users’ attention. Interestingly, we found that news with positive or negative sentiment receive lots of users’ attention. We also found that users’ opinion depend on the sentiment of news articles and the type of information sources. Finally, we performed temporal analysis to understand how news posting behaviour of social media channels evolve over a period of time. Future work can look at actual online news articles from different types of online news channels to investigate the differences in the news articles and to compare these articles with social media news posts.

References