Combating Fake News with Interpretable News Feed Algorithm

Nowadays, artificial intelligence algorithms are used for targeted and personalized content distribution in the large scale as part of the intense competition for attention in the digital media environment. Unfortunately, targeted information dissemination may result in intellectual isolation and discrimination. Further, as demonstrated in recent political events in the US and EU, malicious bots and social media users can create and propagate targeted `fake news' content in different forms for political gains. From the other direction, fake news detection algorithms attempt to combat such problems by identifying misinformation and fraudulent user profiles. This paper reviews common news feed algorithms as well as methods for fake news detection, and we discuss how news feed algorithms could be misused to promote falsified content, affect news diversity, or impact credibility. We review how news feed algorithms and recommender engines can enable confirmation bias to isolate users to certain news sources and affecting the perception of reality. As a potential solution for increasing user awareness of how content is selected or sorted, we argue for the use of interpretable and explainable news feed algorithms. We discuss how improved user awareness and system transparency could mitigate unwanted outcomes of echo chambers and bubble filters in social media.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

11/29/2018

Combating Fake News with Interpretable News Feed Algorithms

Nowadays, artificial intelligence algorithms are used for targeted and p...
04/04/2019

Open Issues in Combating Fake News: Interpretability as an Opportunity

Combating fake news needs a variety of defense methods. Although rumor d...
08/28/2020

Linked Credibility Reviews for Explainable Misinformation Detection

In recent years, misinformation on the Web has become increasingly rampa...
03/05/2019

Trust and Trustworthiness in Social Recommender Systems

The prevalence of misinformation on online social media has tangible emp...
06/24/2018

Balanced News Using Constrained Bandit-based Personalization

We present a prototype for a news search engine that presents balanced v...
08/28/2020

TRUSTD: Combat Fake Content using Blockchain and Collective Signature Technologies

The growing trend of sharing news/contents, through social media platfor...
06/11/2018

How Curiosity can be modeled for a Clickbait Detector

The impact of continually evolving digital technologies and the prolifer...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

Introduction

The popularity of social media has resulted in large, continuously updating collections of user data. Alongside the human users’ activity in sharing information to socialize and express their opinions in a virtual social environment, machine learning algorithms actively process social media content for different purposes in very large scale. Early examples include recommendation and personalization algorithms used in online e-commerce websites to make relevant product suggestions. Similar approaches found their way into social media to assist users with finding preferred content and viewing more relevant advertisements. With the growing amount of user data in social media, giant tech companies started transferring user preference data between different sectors and even third-party companies.

Meanwhile, the public as well as lawmakers have realized the importance of privacy and the influence of social media algorithms in serious matters. Lawmakers such as the EU General Data Protection Regulation (GDPR) started legislation to protect user data transfer without consent. The implications of personalized data tracking for the dissemination and consumption of news has caught the attention of many, especially given evidence of the influence of malicious social media accounts on the spread of fake news to bias users during the 2016 US election [Bessi and Ferrara2016]. Recent reports show that social media outperforms television as the primary news source [Allcott and Gentzkow2017], and the targeted distribution of erroneous or misleading “fake news” may have resulted in large-scale manipulation and isolation of users’ news feeds as part of the intense competition for attention in the digital media space [Kalogeropoulos and Nielsen2018].

Although online information platforms are replacing the conventional news sources, personalized news feed algorithms are not immune to bias [Bozdag2013] and can even cause intellectual isolation [Pariser2011] over time. Despite common beliefs that automated processes can reduce the chance of human judgment errors and biases, artificial intelligence systems are susceptible to accumulating bias from multiple sources. Examples of sources of bias for machine learning systems include human bias in the creation of training data, imbalances in dataset distribution, or biases in the engineering and training processes. Biased algorithms may cause unintentional discrimination that may result in a loss of opportunities (e.g., in employment, education, or housing opportunities), economic loss (e.g., in loan and credit application approval and differential prices for goods) and social stigmatization (e.g., loss of liberty and stereotype reinforcement) in large scale. It is remarkable that despite all these limitations, lack of end-user awareness results in the blind trust of the user to AI systems. For instance, a study [Rader and Gray2015] on users beliefs and judgments about Facebook’s news feed algorithm shows a significant lack of awareness in users on how the system works and whether if the news feed algorithm is biased or not.

Transparency could be a solution in such circumstances by explaining why a certain recommendation or decision is made, but many advanced machine learning algorithms lack transparency and interpretability. Lack of transparency in machine learning algorithms has been studied before the creation of deep neural networks (e.g., 

[Swartout1983]), and research shows a trade-off between performance and transparency of the machine learning systems [Eddy2006]

. With the recent advancement of deep learning algorithms, researchers are further emphasizing the necessity of interpretable machine learning systems 

[Gunning2017]. Machine learning explanations could help end users understand how systems works and why certain algorithmic results are generated. Recommendation algorithms were among the first machine learning algorithms to provide explanations to help users build trust. Examples of explainable recommendation algorithms include movie and other product recommendations in e-commerce and entertainment platforms [Berkovsky, Taib, and Conway2017]. While news feed algorithms are basically recommendation systems for news content, their function is similar to decision-making algorithms since the user has no chance (or very little) to choose between multiple choice of news. Lack of explanation on how the content is selected for the user may result in unaware users who think they have access to all available information rather than only a small subset of news. Explanation of the composition of news feeds and personalized search engines can help users understand how their news content is selected and if any bias is included in the results.

Figure 1:

A summary of fake content detection methods at different stages of the news life in social media. While natural language processing and social media data mining methods are popular in analyzing news at the consumption stage, there is a limited amount of research on accountable news recommendation algorithms to manage the propagation of fake content. The inner arrow shows how user’s social data is used to update personalized news feed algorithms.

In this paper, we first review how fake news and other falsified contents are being managed in online/social media. We then categorize common methods for detecting the creation and circulation of falsified content in social media. We also discuss how news feed algorithms may promote falsified content, impact news diversity, and hinder credibility. We hypothesize that news feed algorithms are likely to align with user confirmation bias to isolate users to specific news source and perception of reality. After all, these algorithms are designed to encourage users to click and increase interest and profit from advertisements. Finally, we also discuss interpretable news feed algorithms as a potential solution to increase user awareness and news diversity in social media.

Analyzing Fake Content in Social Media

Fake content is being generated intentionally for the purpose of various political and financial benefits. Although dishonest/biased coverage of political news and false commercial advertisement are among traditional human-made fake content, nowadays machine learning algorithms can also generate fake content (e.g., falsified videos111https://github.com/deepfakes/faceswap and images, and text [Blom and Hansen2015]) with targeted user groups and at a very large scale in social media. Trolls and malicious users also actively spread fake content to disrupt online communities. Shu et al. shu2017fake provides a comprehensive review of data mining methods, recognition metrics, and training data sets used in fake news detection research. However, their work acknowledges a missing piece in current research on detecting and managing fake news content in social media. We point to a research gap in studying effects of news recommendation algorithms on spread of fake content, crediting unreliable sources, and polarizing users feed (see Figure 1). In this section, we review human review and data mining approaches to recognize fake content and open a discussion on the impact of news recommenders and distribution algorithms.

Human Review

Human review is the first category and manual process of analyzing content to determine its accuracy. Fact checking is a knowledge-based approach to judging the veracity of news piece with external references [Vlachos and Riedel2014]. Experts review the truthfulness of the news by evidence and determine whether claims are accurate or false (partially or entirely). Fact checking is not limited to the correctness of textual content—images and videos may be evaluated or used as evidence as well. As an example of human fact checking, Politifact222http://www.politifact.com/

is a non-profit fact checking organization that gathers experts to review daily news by investigating related reports and interviews in order to classify the relative accuracy of the news using a six-point rating scale. However, expert review fact checking methods are time-consuming, expensive, and not scalable for stopping the spread of fake content in social media.

To further increase the scalability of the expert review method, visual analytics tools can help experts in detecting social streams and track their evolution over time. For instance, LeadLine [Dou et al.2012] is an interactive visual analytics system designed to help identify meaningful events in the news and enable experts to visually review temporal changes of these events. Other tools also visualize summaries of social streams on topical and geographical aspects [Wu et al.2018]. Crowdsourcing techniques have also been employed to reduce fact checking costs by collecting the wisdom of the crowd. To solicit help from active users, some social media platforms are also including options for users to report suspicious posts and accounts to be reviewed later as a way of leveraging of the power of crowd.

Data Mining

Falsified information and fake news may also be detected by analyzing their content using various types of data mining and machine learning techniques. One approach, for instance, is to use linguistic features to analyze writing styles to detect possible false content [Afroz, Brennan, and Greenstadt2012]. For example, recognizing deception-oriented [Rubin and Lukoianova2015] and hyper-partisan content [Potthast et al.2017] can be used as a basis for detecting intentionally falsified information. Researchers use existing falsified content on social media (e.g., from twitter 333\(https://about.twitter.com/en_{u}s/values/elections-integrity.html\)) and fact checking organizations (e.g., PolitiFact [Wang2017]) as training data for such methods. Another approach is to use clickbait detection algorithms [Potthast et al.2016] to analyze inconsistency between headlines and content of the news for possible fake news detection. Image and video verification methods including deep learning algorithms (e.g., [Julliand, Nozick, and Talbot2015, Afchar et al.2018]) have also been explored to recognize falsified visual content.

In addition to analyzing the news by their content, other social media information such as source credibility [Castillo, Mendoza, and Poblete2013], users’ stance [Jin et al.2016], and news temporal spreading pattern  [Kwon et al.2013] have been used to assess the veracity of the news. Such social features can be applied to user groups to evaluate the credibility of specific news pieces by considering the stance of a group of users for the news topics [Tacchini et al.2017]. Similarly, rumor detection methods aim to detect and track posts discussing a specific topic [Zubiaga et al.2018]. Although most mentioned data mining methods do not perform direct fake news detection, these methods leverage social and textual feature to identify suspicious news pieces for human review (crowdsourced or expert review). To increase fake news detection accuracy, training models for analyzing multi-source and multi-modal data is suggested as another approach. For example, Shu et al. shu2017exploiting explored the correlation between news publisher bias, user stance, and user engagement together in their Tri-Relationship fake news detection framework. In the following work, Shu et al. shu2018fakenewsnet proposed a training dataset to include news content and social context along with dynamic information of news and user accounts.

Despite the research on detecting the falsified content by its textual and social features, machine learning study on the importance of news recommendation algorithms in distribution of false content is very limited. The importance of news feed algorithms is their critical role in providing news content and their vulnerability for being misused and causing discrimination. For the rest of this paper, we discuss the accountability of news feed algorithms for propagating fake content by creating echo chambers and filter bubbles in social media and news search engines. We suggest interpretable news feed algorithm will solve these problems by increasing user awareness, holding users accountable for their content, and exposing users to opposite political views.

Limitations of News Feed Algorithms

Although recommendation algorithms were primarily designed as adaptive systems for recommending, sorting and filtering search result entertainment and e-commerce platforms, they found their way into the news recommendation and search engines to provide news based on users interest. Nowadays, personalized search engines and news recommendations on social media and provide content to users based on users’ profiles, interests, social media friends, and other past click behavior. However, research shows the personalization feature may create adverse situations such as filter bubble and echo chamber effect by promoting contents which are in favor of the user’s existing mindset and eliminating conflicting viewpoints and their sources. Further, lack of transparency in these algorithms may cause discrimination (e.g., by providing biased recommendations) and even open the door to targeted distribution of malicious paid contents to bypass regulations. In this section, we review the downsides of news feed and personalized news search algorithms that may propagate and accredit of fake news.

Echo Chambers in Social Media

Echo chamber in social media describes a situation where homogeneous views are reinforced by communication inside closed groups. Research shows the personalization feature can create the “echo chamber” effect similar to echo chambers in social media [Quattrociocchi, Scala, and Sunstein2016] by promoting contents which are in favor of the user’s existing mindset. Echo chambers have been previously studied in relation to creating polarized opinions and shaping a false sense of credibility for users who frequent news sources through social media [Zajonc2001, Paul and Matthews2016]. This false sense of credibility holds users in a vulnerable position of accepting biased and fake news content. In similar circumstances, although news recommendation algorithms are meant to provide content related to users interest, these machine learning algorithms can isolate a user’s news feed with a certain perception of reality and trigger user confirmation bias to over-trust partisan sources.

Investigating such phenomena, a recent study by Geschke et al. geschke2018triple presented a simulation of different information filtering scenarios that may contribute to social fragmentation of users into distinct echo chambers. Their observation of agent-based modeling found that social and technological filters can boost social polarization and lessen the interconnections of social media echo chambers. This evidence clarifies the potentially negative impact of news recommendation algorithms in creating more distinct echo chambers which calls for further investigation of solutions. Others study methods to overcome echo chambers in social media. For instance, Lex et al. lex2018mitigating presented a content-based news recommendation that can increase exposure of the opposite view to users in order to mitigate the echo chamber effect in social media. They computed and compared the diversity of the news on Twitter by their hashtags in different modes of a news recommender to balance users’ news feeds and prevent the creation of echo chambers. In another work, Hou et al. hou2018balancing demonstrated methods to balance popularity bias in network-based recommendation systems and to significantly improve the system’s diversity and accuracy.

Filter Bubbles in Search Engines

Filter bubble [Pariser2011] is another term to describe negative effects of personalized search engines and news feeds. Filter bubbles represent a state of intellectual isolation where users are only exposed to a certain perspective of information brought to them by personalized search engines and news feed algorithms. The lack of news diversity and exposure to conflicting viewpoints in the long term creates a filter bubble for individuals and social media groups and increase the chance of accepting misleading information and accrediting unreliable sources.

In their study of targeted news in Facebook, Bakshy et al. bakshy2015exposure discuss inconclusive conclusions about the role of news feeds in creating biased social media environments. However, other work indicates negative effects of personalization and explores ways to mitigate these effect. For example, Nguyen et al. [Nguyen et al.2014]

measure content diversity at individual user level in a longitudinal study on collaborative filtering-based recommender system. They contribute a new metric to measure content diversity and their study on a movie recommendation system indicates that recommender algorithms expose users a narrower set of items over time. In another work, Haim et al. haim2018burst conducted an exploratory analysis of personalized news search engine effect on news diversity. They studied both explicit (i.e., user-defined filters) and implicit personalization (i.e., by giving algorithms the chance to observe a given agent during a week-long period), and their results showed a general bias toward over-presenting certain news outlets and under-presenting other sources.

Another approach to encounter unwanted negative effects of these algorithms is to define new measures and standards for sensitive data and products recommendation systems. For better measuring the quality of recommender systems, Ekstrand et al. valcarce2018robustness and Valcarce et al. ekstrand2018all proposed new evaluation measures to account for other considerations like users’ popularity bias or content diversity. The importance of new measures for news recommendation algorithms comes from considering the diversity of content across different user groups.

Biased Algorithms and Paid Content

Algorithmic bias is another issue with news and content recommendation systems. Example of biased news recommendation system is to eliminate content from a certain geographic area and not by a thorough analysis of the content itself. The importance of algorithmic bias is more sensible since the bias could enter the algorithms from different sources including bias from the training data, data cleaning and engineering, and bias in training.

Multiple works study bias and discrimination in algorithms via different techniques such as auditing [Sandvig et al.2014] to investigate algorithms that are still working well. Researchers also propose using diversity metrics [Valcarce et al.2018, Ekstrand et al.2018] and bias quantification methods [Kulshrestha et al.2017] as other ways to study bias and discrimination in recommendation algorithms. For example, Kulshrestha et al. kulshrestha2017quantifying proposed a framework to quantify bias in ranked search results in political-related queries in Twitter. Their framework can distinguish bias from news content and ranking algorithm, and they found evidence of significant effects of both input content and search algorithms in producing bias.

Further, paid advertisements can enter feeds for targeted purposes. Paid content that is regularly in the form of commercial advertisements may also create opportunities for malicious accounts to spread false content to manipulate users for political gains. In a sense, paid content enters users’ news feeds from a back door along with related social posts and news to attract user attention and increase impact. Recent examples of distributing paid content are seen in the influence of Cambridge Analytica, a political consulting firm, for political advertisements in social media. They attracted heavy attention and criticism for their use of psychographic-profiling methods on Facebook user profiles in order to perform unique voter-targeting models during the United States presidential campaign and the Brexit campaign [Persily2017].

Along with the aforementioned weaknesses of news recommendation algorithms, research has shown many users are not aware of the existence of news feed algorithms or do not know how such algorithms work. Results from Rader and Gray’s rader2015understanding study on how individuals make sense of the Facebook’s news feed algorithm shows user understanding of “how the news feed works” is limited to their repeated experience with the system. We argue that increasing user awareness by transparent news search engines and news feed algorithms not only will improve the users understanding of these algorithms, but also increases the diversity of news in social media. Transparency and accountability in such large-scale employed algorithms is essential to detect possible algorithmic bias and user data misuse to distribute fake content and cause discrimination.

Figure 2: Different approaches to detect and manage falsified content and fake news in social media. While most data mining methods aim to identify false information that is already spread in social media, interpretable and transparent news feed algorithms can manage the propagation of falsified content before it spreads.

The Benefits of Interpretability

In the previous sections, we discussed current methods to detect and manage fake content in social media. We also discussed how news feed algorithms and personalized search engines may contribute to spreading fake content by not showing diverse news. In this section, we propose interpretable and accountable news feed algorithms as a potential solution by increasing news diversity and improving user control over readers’ content.

Interpretable machine learning algorithms generate explanations and describe the reasoning behind machine learning decisions and predictions. Machine learning explanation enables the user to understand how the data is processed and supports user awareness to identify possible bias and systems malfunctions. Interpretable machine learning research has many applications in algorithmic decision making such as fair and ethical computing and accountability in AI [Doshi-Velez and Kim2017]. For example, to measure users’ perception of justice in intelligent decision making, Binns et al. [Binns et al.2018] studied explanations in daily-life systems such as determining car insurance rates and loan application approvals. Their results highlight the importance of machine learning explanations in users’ comprehension and trust in algorithmic decision-making systems. Recommendation algorithms have been among the earliest algorithms to include explanations to improve users trust and experience. Explanation in recommender systems have been shown to increase recommendation uptake by improving system’s transparency and user’s trust. In regard to recommendation systems in real-world applications, Berkovsky et al. Berkovsky:2017:RUT:3025171.3025209 studied user trust with various recommendation interfaces and content selection strategies. They investigated user reliance on a movie recommender system in various conditions of recommendation interfaces.

However, machine learning explanations in product recommender systems are limited to providing product information, highlighting users need by their search history, and other domain-specific features. In contrast, news recommendation algorithms in social media often process more complicated and sensitive data. Also, news recommendations generate a user’s reading list and do not give options for the user to choose among content. In these circumstances, lack of explanation on how the content is selected for the user may bias the user toward what the algorithm is providing in the news feed and not the entire reality.

Interpretable news feed algorithms can help users understand how the information selection process is done and allow users to review a system’s objective. Explaining the user preference model and other parameters (e.g., the priority of paid content) that contribute to the news selection will help users to have a more accurate mental model of these algorithms and build appropriate trust to the system. In a study on the effects of news feed explanations on user awareness, Rader et al. rader2018explanations showed complementary explanations for the news feed algorithm helped users to determine if the news feed algorithm was biased or if they had control over the content. In their study, they measured users’ awareness, correctness, and accountability to evaluate algorithmic transparency. Although they used a general explanation for how Facebook’s news feed works, their results showed an increase in user awareness with explanations.

We argue that further detailed explanations of a news recommendation algorithm (e.g., in form of visualization of model parameters) enables the user to detect biased models and suspect odd outputs. Such an explanation would tell the user what kind of

soft filters are applied on their news feed to select the user content. Related to this, Burbach et al. burbach2018user studied user preferences of recommendation filters through an online survey. Their survey shows users have different patterns of recommendation method preference for different product categories. A significant finding was that in more sensitive categories (e.g., news recommendation) users preferred content-based recommendations and rejected social-based methods that require personal data to process. In other work, Rader and Gray’s rader2015understanding study showed that the mismatch between the objective of those who design recommendation algorithm and the object of the user may lead to reduced user interests in social media platforms. Therefore, empowering the user to adjust and prioritize the influence of recommendation could also have positive outcomes for long-term acceptance of social media. To achieve accountable news feed algorithms, it may be sufficient to allow users to eliminate unrelated and unintended content by reviewing and adjusting the algorithm’s parameters, but designing usable explanation interfaces to allow such functionality may be challenging.

Table 2

shows a list of potential solutions to manage fake news content in social media at different stages of news life. Detecting fake content with human review can be done through expert review and crowdsourcing techniques at the early stages. Also, computer vision techniques help in detecting forged images and videos known as DeepFakes 

[Afchar et al.2018]. New methods like provenance analysis have also been utilized for content validation via generating provenance graph of images as the same content is shared and modified over time [Bharati et al.2018]. The next stage of fake news life in social media is the distribution of content via search engines and news feed algorithms. Others also examined the use of spacial news feed in virtual reality environment as opposed to linear feeds to reduce the creation of filter bubbles [Linder et al.2018]

. Multiple sources of evidence show personalized news feed algorithms and targeted advertisement of political content can have drastic effects on news diversity and the creation of echo chambers in social media. The final stage of analyzing fake news is to process social media users’ stance, analyze news propagation patterns, and estimate news source credibility to find possible false content in social media. Although research on social media data mining shows that combinational methods help to reduce the cost of computation and increase fake news detection accuracy 

[Shu, Wang, and Liu2017], using social media data means waiting until the fake content is already exposed to the users.

Looking at the current research in fake news and current advancements in interpretable machine learning algorithms, we suggest that adding a layer of explanation to news recommendation algorithms would serve as a major leap toward accountable news streaming algorithms and eliminate propagation of fake content by increasing user awareness and transparency in social media news feed and news search engines.

Conclusion

We reviewed fake news in social media over three main stages and discussed current methods and tools for analyzing and detecting potentially false content. We then focused on limitations and risks (e.g., misuse of user data) of using personalized content selection algorithms in social media. We argue that the large-scale targeted propagation of content for political gain using personalized news feed may cause polarized social media and spreading of fake content. After reviewing current research on fake news detection and personalized news feed algorithms, we suggest interpretable news feed algorithms as a potential solution to increase user awareness and hold users accountable for their own content rather than algorithms with undisclosed/opaque objectives.

References

  • [Afchar et al.2018] Afchar, D.; Nozick, V.; Yamagishi, J.; and Echizen, I. 2018. Mesonet: a compact facial video forgery detection network. arXiv preprint arXiv:1809.00888.
  • [Afroz, Brennan, and Greenstadt2012] Afroz, S.; Brennan, M.; and Greenstadt, R. 2012. Detecting hoaxes, frauds, and deception in writing style online. In Security and Privacy (SP), 2012 IEEE Symposium on, 461–475. IEEE.
  • [Allcott and Gentzkow2017] Allcott, H., and Gentzkow, M. 2017. Social media and fake news in the 2016 election. Journal of Economic Perspectives 31(2):211–36.
  • [Bakshy, Messing, and Adamic2015] Bakshy, E.; Messing, S.; and Adamic, L. A. 2015. Exposure to ideologically diverse news and opinion on facebook. Science 348(6239):1130–1132.
  • [Berkovsky, Taib, and Conway2017] Berkovsky, S.; Taib, R.; and Conway, D. 2017. How to recommend?: User trust factors in movie recommender systems. In Proceedings of the 22Nd International Conference on Intelligent User Interfaces, IUI ’17, 287–300. New York, NY, USA: ACM.
  • [Bessi and Ferrara2016] Bessi, A., and Ferrara, E. 2016. Social bots distort the 2016 us presidential election online discussion.
  • [Bharati et al.2018] Bharati, A.; Moreira, D.; Brogan, J.; Hale, P.; Bowyer, K. W.; Flynn, P. J.; Rocha, A.; and Scheirer, W. J. 2018. Beyond pixels: Image provenance analysis leveraging metadata. arXiv preprint arXiv:1807.03376.
  • [Binns et al.2018] Binns, R.; Van Kleek, M.; Veale, M.; Lyngs, U.; Zhao, J.; and Shadbolt, N. 2018. ’it’s reducing a human being to a percentage’: Perceptions of justice in algorithmic decisions. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 377. ACM.
  • [Blom and Hansen2015] Blom, J. N., and Hansen, K. R. 2015. Click bait: Forward-reference as lure in online news headlines. Journal of Pragmatics 76:87–100.
  • [Bozdag2013] Bozdag, E. 2013. Bias in algorithmic filtering and personalization. Ethics and information technology 15(3):209–227.
  • [Burbach et al.2018] Burbach, L.; Nakayama, J.; Plettenberg, N.; Ziefle, M.; and Valdez, A. C. 2018. User preferences in recommendation algorithms: the influence of user diversity, trust, and product category on privacy perceptions in recommender algorithms. In Proceedings of the 12th ACM Conference on Recommender Systems, 306–310. ACM.
  • [Castillo, Mendoza, and Poblete2013] Castillo, C.; Mendoza, M.; and Poblete, B. 2013. Predicting information credibility in time-sensitive social media. Internet Research 23(5):560–588.
  • [Doshi-Velez and Kim2017] Doshi-Velez, F., and Kim, B. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
  • [Dou et al.2012] Dou, W.; Wang, X.; Skau, D.; Ribarsky, W.; and Zhou, M. X. 2012. Leadline: Interactive visual analysis of text data through event identification and exploration. In Visual Analytics Science and Technology (VAST), 2012 IEEE Conference on, 93–102. IEEE.
  • [Eddy2006] Eddy, D. M. 2006. Accuracy versus transparency in pharmacoeconomic modelling. Pharmacoeconomics 24(9):837–844.
  • [Ekstrand et al.2018] Ekstrand, M. D.; Tian, M.; Azpiazu, I. M.; Ekstrand, J. D.; Anuyah, O.; McNeill, D.; and Pera, M. S. 2018. All the cool kids, how do they fit in?: Popularity and demographic biases in recommender evaluation and effectiveness. In Conference on Fairness, Accountability and Transparency, 172–186.
  • [Geschke, Lorenz, and Holtz2018] Geschke, D.; Lorenz, J.; and Holtz, P. 2018. The triple-filter bubble: Using agent-based modelling to test a meta-theoretical framework for the emergence of filter bubbles and echo chambers. British Journal of Social Psychology.
  • [Gunning2017] Gunning, D. 2017. Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web.
  • [Haim, Graefe, and Brosius2018] Haim, M.; Graefe, A.; and Brosius, H.-B. 2018. Burst of the filter bubble? effects of personalization on the diversity of google news. Digital Journalism 6(3):330–343.
  • [Hou, Pan, and Liu2018] Hou, L.; Pan, X.; and Liu, K. 2018. Balancing the popularity bias of object similarities for personalised recommendation. The European Physical Journal B 91(3):47.
  • [Jin et al.2016] Jin, Z.; Cao, J.; Zhang, Y.; and Luo, J. 2016. News verification by exploiting conflicting social viewpoints in microblogs. In AAAI, 2972–2978.
  • [Julliand, Nozick, and Talbot2015] Julliand, T.; Nozick, V.; and Talbot, H. 2015. Image noise and digital image forensics. In International Workshop on Digital Watermarking, 3–17. Springer.
  • [Kalogeropoulos and Nielsen2018] Kalogeropoulos, A., and Nielsen, R. K. 2018. Social inequalities in news consumption. In FACTSHEET, NEWS MEDIA DIGITAL MEDIA, 461–475. Reuters Institute for the Study of Journalism.
  • [Kulshrestha et al.2017] Kulshrestha, J.; Eslami, M.; Messias, J.; Zafar, M. B.; Ghosh, S.; Gummadi, K. P.; and Karahalios, K. 2017. Quantifying search bias: Investigating sources of bias for political searches in social media. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, 417–432. ACM.
  • [Kwon et al.2013] Kwon, S.; Cha, M.; Jung, K.; Chen, W.; and Wang, Y. 2013. Prominent features of rumor propagation in online social media. In 2013 IEEE 13th International Conference on Data Mining, 1103–1108. IEEE.
  • [Lex, Wagner, and Kowald2018] Lex, E.; Wagner, M.; and Kowald, D. 2018. Mitigating confirmation bias on twitter by recommending opposing views. arXiv preprint arXiv:1809.03901.
  • [Linder et al.2018] Linder, R.; Stacy, A. M.; Lupfer, N.; Kerne, A.; and Ragan, E. D. 2018. Pop the feed filter bubble: Making Reddit social media a VR cityscape. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 619–620. IEEE.
  • [Nguyen et al.2014] Nguyen, T. T.; Hui, P.-M.; Harper, F. M.; Terveen, L.; and Konstan, J. A. 2014. Exploring the filter bubble: the effect of using recommender systems on content diversity. In Proceedings of the 23rd international conference on World wide web, 677–686. ACM.
  • [Pariser2011] Pariser, E. 2011. The filter bubble: What the Internet is hiding from you. Penguin UK.
  • [Paul and Matthews2016] Paul, C., and Matthews, M. 2016. The russian “firehose of falsehood” propaganda model. RAND Corporation.
  • [Persily2017] Persily, N. 2017. The 2016 us election: Can democracy survive the internet? Journal of democracy 28(2):63–76.
  • [Potthast et al.2016] Potthast, M.; Köpsel, S.; Stein, B.; and Hagen, M. 2016. Clickbait detection. In European Conference on Information Retrieval, 810–817. Springer.
  • [Potthast et al.2017] Potthast, M.; Kiesel, J.; Reinartz, K.; Bevendorff, J.; and Stein, B. 2017. A stylometric inquiry into hyperpartisan and fake news. arXiv preprint arXiv:1702.05638.
  • [Quattrociocchi, Scala, and Sunstein2016] Quattrociocchi, W.; Scala, A.; and Sunstein, C. R. 2016. Echo chambers on facebook.
  • [Rader and Gray2015] Rader, E., and Gray, R. 2015. Understanding user beliefs about algorithmic curation in the facebook news feed. In Proceedings of the 33rd annual ACM conference on human factors in computing systems, 173–182. ACM.
  • [Rader, Cotter, and Cho2018] Rader, E.; Cotter, K.; and Cho, J. 2018. Explanations as mechanisms for supporting algorithmic transparency. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 103. ACM.
  • [Rubin and Lukoianova2015] Rubin, V. L., and Lukoianova, T. 2015. Truth and deception at the rhetorical structure level. Journal of the Association for Information Science and Technology 66(5):905–917.
  • [Sandvig et al.2014] Sandvig, C.; Hamilton, K.; Karahalios, K.; and Langbort, C. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry 1–23.
  • [Shu et al.2017] Shu, K.; Sliva, A.; Wang, S.; Tang, J.; and Liu, H. 2017. Fake news detection on social media: A data mining perspective. ACM SIGKDD Explorations Newsletter 19(1):22–36.
  • [Shu et al.2018] Shu, K.; Mahudeswaran, D.; Wang, S.; Lee, D.; and Liu, H. 2018. Fakenewsnet: A data repository with news content, social context and dynamic information for studying fake news on social media. arXiv preprint arXiv:1809.01286.
  • [Shu, Wang, and Liu2017] Shu, K.; Wang, S.; and Liu, H. 2017. Exploiting tri-relationship for fake news detection. arXiv preprint arXiv:1712.07709.
  • [Swartout1983] Swartout, W. R. 1983. Xplain: A system for creating and explaining expert consulting programs. Technical report, UNIVERSITY OF SOUTHERN CALIFORNIA MARINA DEL REY INFORMATION SCIENCES INST.
  • [Tacchini et al.2017] Tacchini, E.; Ballarin, G.; Della Vedova, M. L.; Moret, S.; and de Alfaro, L. 2017. Some like it hoax: Automated fake news detection in social networks. arXiv preprint arXiv:1704.07506.
  • [Valcarce et al.2018] Valcarce, D.; Bellogín, A.; Parapar, J.; and Castells, P. 2018. On the robustness and discriminative power of information retrieval metrics for top-n recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems, 260–268. ACM.
  • [Vlachos and Riedel2014] Vlachos, A., and Riedel, S. 2014. Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, 18–22.
  • [Wang2017] Wang, W. Y. 2017. ” liar, liar pants on fire”: A new benchmark dataset for fake news detection. arXiv preprint arXiv:1705.00648.
  • [Wu et al.2018] Wu, Y.; Chen, Z.; Sun, G.; Xie, X.; Cao, N.; Liu, S.; and Cui, W. 2018. Streamexplorer: a multi-stage system for visually exploring events in social streams. IEEE transactions on visualization and computer graphics 24(10):2758–2772.
  • [Zajonc2001] Zajonc, R. B. 2001. Mere exposure: A gateway to the subliminal. Current directions in psychological science 10(6):224–228.
  • [Zubiaga et al.2018] Zubiaga, A.; Aker, A.; Bontcheva, K.; Liakata, M.; and Procter, R. 2018. Detection and resolution of rumours in social media: A survey. ACM Computing Surveys (CSUR) 51(2):32.