In the ‘offline’ world, long before the Web became publicly available, users could rely on traditional forms of information verification, such as the presence of traditional media intermediaries such as experts, by considering their reputation, or trust them based on first-hand experiences. Nowadays, in the Social Web scenario, almost everyone can spread contents on social media in the form of User-Generated Content (UGG), almost without any traditional form of trusted control . This ‘disintermediation’ process  has lead, on the one hand, to the democratization of the information diffusion, but, on the other hand, to the spread of possible fake news and misinformation, which risk to severely affect social aspects of living.
We live, in fact, in a so-called ‘post-truth’ era, in which objective facts are less influential in shaping public opinion than appeals to emotion and personal belief. This is partially due to the fact that, in social media, user-created networks can become real echo chambers , in which one point of view dominates all the others, the verification of the statements has usually no effect, and this allows the repetition of unverified statements without refutation. This is connected to the cognitive phenomenon of confirmation bias that, in psychology, is defined as the tendency to search for, interpret, favor, and recall information in a way that affirms one’s prior beliefs or hypotheses . Furthermore, the echo chamber phenomenon is emphasized by the filtering algorithms that are the basis of social media in proposing information of interest: by suggesting personalized (information) items that consider different elements of the user profile, such as location, past click-behavior and search history, users become separated from information that disagrees with their viewpoints, effectively isolating them in their own cultural or ideological bubbles, the so-called filter bubbles . The echo chamber and filter bubble phenomena are currently being studied by several researchers, in different fields [3, 9, 38].
In this scenario, it becomes essential to try to find and develop automatic solutions (i.e., algorithms, user interfaces, search engines, mobile apps) that assist the user to get out of their echo chambers and become aware of the level of credibility of the information they come into contact with. In particular, the contexts in which the problem of credibility assessment is receiving the most interest are those of opinion spam detection, fake news detection and evaluation of health-related information that spreads online.
Opinion spam refers to malicious activities attempting to mislead either human readers, or automated opinion mining and sentiment analysis systems. In most cases, it concernsfake reviews, whose spread has negative effects on both businesses and potential customers . Fake news are particularly diffused in microblogging platforms, where millions of users act as real-time news diffusers, since the so-called trending topics can be considered, in all respects, “headline news or persistent news” . In this context, detecting fake news is particularly relevant to prevent public opinion from being manipulated [5, 8]. Concerning health-related information, numerous are the communities of individuals who share contents about personal health problems or try to carry out self-diagnosis through Question-Answering (QA) systems or professional healthcare services allowing people to interact online . In this case, incurring in inaccurate, unverified, unsuitable information can lead to potentially very harmful consequences for individuals with a low health literacy .
In all three of these contexts, multiple solutions have been studied or are being studied to solve context-specific problems related to the credibility of the information diffused. However, the approaches proposed so far in the literature share common characteristics that allow to classify them from a more general point of view, as illustrated in this article, which is organized as follows. Initially, the concept of credibility will be presented, addressed both from the point of view of the social/communication sciences and within the Social Web scenario (Section0.2); then, a classification of the main state-of-the-art approaches for the automatic or semi-automatic evaluation of the credibility of online information will be provided (Section 0.3); finally, the open issues that remain to be faced for the development of effective solutions to the considered problem will be highlighted, as well as some further research directions (Section 0.4).
0.2 From ‘Offline’ to ‘Online’ Credibility Assessment
The study of the credibility of information has a long history and concerns different contexts and research fields, such as social sciences (including in particular communication and psychology), and computer science, also at interdisciplinary level. Over the years, the interest has gradually moved from traditional communication environments, characterized by interpersonal and persuasive communication, to mass communication and interactive-mediated communication, with particular reference to online communication .
Therefore, in this section, the main aspects related to the evolution of the study of information credibility are briefly discussed, focusing, in particular, on two different scenarios: the ‘offline’ scenario, where information is disseminated through means of ‘traditional’ media that do not require the use of digital technologies, and the ‘online’ scenario, where information is diffused through the (Social) Web by means of Web 2.0 technologies.
0.2.1 Offline Information Credibility
In the research field of communication, the notion of credibility has been investigated since ancient times. In fact, among the first works that have come down to us that discuss this concept, there are the Phaedrus by Plato, and the Aristotle’s Rhetoric, both dating back to the 4th Century BC, where the focus is, in particular, on the study of the characteristics of the speaker that make her/him believe credible. Over the years, depending on the context, credibility has been associated with other many related concepts or combinations of them, such as believability, trustworthiness, perceived reliability, expertise, accuracy, etc. An in-depth analysis of the nuances of meaning related to these concepts is provided in .
In the modern era, in the fields of psychology and communication, the research undertaken in the 1950s by Hovland et al.  constitutes the first systematic work about credibility and mass media, focusing specifically on the study of the impact that the credibility of the information source, i.e., source credibility, has on the assessment of the credibility of mass communication messages. The authors investigated, in particular, how people evaluated these messages coming from “high credibility” and “low credibility” sources.
Later on, Fogg and Tseng in  stated that credibility, as a perceived quality of the information receiver, it is composed of multiple dimensions that have to be considered. In this sense, the process of assessing information credibility involves different characteristics, which can be connected to: the source of information, the information that is diffused, i.e., the message, considering both its structure and its content, and the media used to diffuse information .
0.2.2 Online Information Credibility
Online, and in the Social Web in particular, information credibility assessment deals with the analysis of both UGC and their authors’ characteristics , and the intrinsic community-based nature of social media platforms . Specifically, this means to take into account credibility features connected to: the users forming a virtual community (i.e., the information sources), their User-Generated Content (i.e., the information the users diffuse), and the social relationships connecting the users in the virtual community (i.e., the main characteristic of an online social networking system), also considering other kinds of entities possibly connected to users with other kinds of relationships .
The problem of evaluating the above-mentioned aspects, in a scenario characterized by a large amount of information generated directly by users such as the Social Web, becomes particularly complex for information receivers. While in previous media contexts people usually dealt with traditional trusted intermediaries on which they could rely on, e.g., reputed newspapers, popular enterprises, no profit organizations, etc., social media “have in many ways shifted the burden of information evaluation from professional gatekeepers to individual information consumer” . This constitutes a tricky problem because, even if credibility is a characteristic perceived by individuals, credibility assessments can be very complex for human beings, especially in the online environment, because the effort it takes to verify the credibility of Web-based information would be disproportionate and often ineffective . In fact, humans have limited cognitive capacities to effectively evaluate the information they receive, especially in situations where the complexity of the features involved and of their interactions increases .
For the above-mentioned reasons, there is nowadays the need of developing interfaces, tools or systems that are designed to help users in automatically or semi-automatically assess information credibility. In the next section, an high-level classification of the approaches that have been proposed so far to tackle this issue in the Social Web is provided.
0.3 Approaches to Credibility Assessment
A first classification that can be made with respect to recent approaches that assess the credibility of online information concerns the implementation of data-driven or model-driven approaches. In the first case, starting from some available data, a bottom-up model is learned to identify credible information with respect to non-credible one. In the second case, some domain knowledge is available, which is used to build a top-down model to tackle the considered issue. Another classification dimension that can be considered, concerns the fact of studying the propagation of (false) online information or the attempt to produce a classification (or a ranking) of information based on its credibility level.
As illustrated in Section 0.1, the approaches that fall into the above-mentioned categories are used in different research contexts to the evaluation of the credibility of online information, such as opinion spam detection, fake news detection, and credibility assessment of online health-related information . Although each of these contexts has its own peculiarities, the common aspects that remain valid for each category of approaches will be explained in the following.
0.3.1 Information Propagation
In general, propagation-based approaches are mainly concerned with studying the influence that spammers or social bots have on the dissemination of (false) information or how low-credibility information spreads over the social network structure . The identification of genuine (or, conversely, non-credible) information deals, in this case, mostly with the identification of these malicious users, or with the detection of suspicious behaviors in the information diffusion.
These approaches usually rely on the analysis of the graph-based representation of the social network structure, and often employ unsupervised learning algorithms to detect cliques of malicious users (e.g., via Social Network Analysis techniques and metrics) and/or to identify the so-calledbursts, i.e., sudden increases of the use of particular sets of keywords in a very short period of time . Propagation-based approaches can also rely on some pre-computed credibility values (usually learned from a classifier) and study their spread over the social network structure.
The study of malicious users’ behaviors and false information propagation has in many cases a slightly different purpose (even if it is closely related) with respect to the assessment of information credibility (meant as the valuation of information items), which is more related to classification-based approaches, which are illustrated below.
0.3.2 Information Classification
In classification-based approaches
, fall both data-driven methods that are based on the use of (mostly supervised or semi-supervised) learning algorithms, and model-driven approaches which are based on some priordomain knowledge
; both of them aim at classifying in a binary way information items (i.e., credible VS non-credible) or at producing a ranking based on an estimated credibility probability or a credibility score computed for each item. The model-driven category includes both the use of the Multi-Criteria Decision Making (MCDM) paradigm, and the use of Knowledge Bases and Semantic Web technologies .
Approaches based on Credibility Features
When considering machine learning techniques and the MCDM paradigm to assess information credibility, different characteristics, namely credibility features, are taken into account. As illustrated in Section 0.2.2, these features are generally related to the users in the virtual community, the information items that are generated and diffused, and the virtual relationships among users and other entities in the community. For this reason, it is possible to provide the following classification:
Behavioral features: they are related to the users generating and diffusing information. They can be extracted both from public Web data, e.g., user ID, usual time of posting, frequency of posting, presence of a public image profile, etc., and private/internal Web data, e.g., IP and MAC addresses, time taking to post an information item, physical location of the user, etc. .
Content-based features: they are related to the textual content of the information item. They can be both lexical features such as word -grams, part-of-speech, and other lexical attributes, and stylistic features, e.g., capturing content similarity, semantic inconsistency, use of punctuation marks, etc. 
Social (Graph-based) features: they capture complex relationships among users, the information they diffuse, and other possible entities (e.g., products and stores) in the social network .
In the case of using data-driven approaches that employ well-known (supervised) machine learning techniques (e.g., SVM, Random Forests, etc.), often a binary classification of information items with respect to their credibility is obtained by training a model over the considered set of features and one or more suitable labelled dataset(s). These approaches are hence based on a feature extraction and selection phase, and are dependent on the availability of (unbiased) labelled data, which is not always the case, as illustrated in the literature. Furthermore, some of the machine learning techniques proving to be effective in the considered research field, are often inscrutable to observers (they are characterized by the so-called ‘black-box’ behavior), making it difficult to evaluate the importance of distinct and/or interacting features in obtaining the classification results. A possible solution would be the study of approaches based on the so-called
eXplainable Artificial Intelligence(XAI), referring to methods and techniques in the application of AI technology such that the results of the solution can be understood by human experts .
Model-driven Approaches. Approaches for assessing credibility that allow the human being to be involved in the decision process and to (possibly) understand the result obtained, are those modeling the considered problem as a Multi-Criteria Decision Making problem, characterized by the presence of a set of alternatives, i.e., the information items to be evaluated in terms of credibility, and a set of criteria, i.e., the considered set of credibility features. In an MCDM problem, each alternative ‘satisfies’ each criterion to a certain extent, producing this way a performance score, i.e., the credibility score, one for each criterion associated with the alternative. For each alternative, the aggregation of these multiple credibility scores produces an overall performance score, i.e., an overall credibility score.
In such kind of modeling of the problem, there is usually a prior knowledge of the features to be considered, and of the importance that each feature has in terms of credibility (e.g., this information can be taken from prior literature specifically addressing the problem from a social and economic point of view ). Furthermore, some satisfaction functions are defined to transform the values of the features into suitable credibility scores. Finally, suitable aggregation functions to obtain the overall credibility scores can be selected or defined. This way, having an overall credibility score associated with each information item, it is possible to provide both:
A binary classification of information items into genuine/fake (by selecting a suitable classification threshold, as illustrated in ).
A ranking of the information items based on their overall credibility score.
Different an numerous are the families of aggregation functions to be considered for tackling the problem, depending on the preferences of the decision maker [46, 48]. Ordered Weighted Averaging (OWA) operators, in particular, have been extensively studied in the literature . In fact, they give the possibility to guide the aggregation by linguistic quantifiers (e.g., all, some, many, …), which allow the decision maker to obtain the best alternative(s) based on the satisfaction of a ‘certain amount’ of the criteria by the alternative(s). Recently, an MCDM approach has been proposed that allows to model interacting features, by employing the Choquet integral ; in this work, and in general in MCDM approaches based on aggregation operators, it can be complex to define the model when the number of features increases .
Approaches based on Knowledge Bases
Another way to use prior domain knowledge to assess the credibility of online information is to refer to the use of Knowledge Bases . In this context, knowledge is not referred to what are the credibility features associated with information, and their importance. In this case, the information that is known to be credible is available, and it is expressed in terms of facts. This type of approach is used in particular for automated fact checking , in situations where: manual information credibility assessment is not feasible, e.g., the number of human experts is too limited with respect to the amount of information to be verified; crowdsourcing-based information credibility assessment does not guarantee the credibility of assessors.
Technically speaking, a fact is a set of (Subject, Predicate, Object) (SPO) triples extracted from texts that well-represent a given information. E.g., “Giacomo Puccini is a composer” can be expressed as (GiacomoPuccini, Profession, Composer). Facts must be processed and cleaned up (i.e., redundant, outdated, conflicting, unreliable or incomplete data are removed) to build a Knowledge Base, i.e., a set of SPO triples. A graph structure, known as the knowledge graph, can be used to represent the SPO triples in a Knowledge Base, where the entities (i.e., subjects or objects in SPO triples) are represented as nodes, and relationships (i.e., predicates in SPO triples) are represented as edges. Knowledge Bases are suitable candidates for providing ground truth to information credibility assessment studies, i.e., it is possible to reasonably assume the existing triples in a Knowledge Base represent true facts.
Making these premises, to evaluate the credibility of to-be-verified information items, in turn represented as SPO triples, it is sufficient to compare them with the SPO triples contained in the Knowledge Base(s), obtaining, in this way, a classification of the information items. Open issues connected to this kind of approach concern how to consider missing facts in the Knowledge Base(s), and, connected to this problem, how to constantly update the Knowledge Base(s) with up-to-date credible information.
0.4 Open Issues and Further Research
As illustrated in this paper, numerous approaches belonging to different categories have been proposed up to now to assess information credibility in the Social Web, with respect to distinct research contexts, i.e., opinion spam detection, fake news detection, health-related information credibility assessment. Every category of approaches presents both advantages and drawbacks, which are summarized in the following.
Propagation-based approaches allow to effectively identify misinformation spammers and spam bots, leading to the possible identification of false information, but they are affected by the problem of managing and analyzing complex structures such as graphs, and, in some cases, by the need of having pre-computed credibility values to capture and investigate how (genuine or fake) information spreads over the network.
Classification-based approaches are affected by the possible inscrutability of results and data dependency in the case of supervised ‘black-box’ approaches, and by the difficulty of managing the complexity of the model when the number of criteria increases in the case of the MCDM scenario; despite this, both turned out to be particularly effective in well defined tasks .
In approaches based on the use of Knowledge Bases, some issues emerge about the treatment of missing information, conflict resolution and triples update in the knowledge graph(s), especially in the Social Web scenario where new information is produced with a very high frequency and volume. Despite this, they can be particularly useful and of particular support to automated fact checking.
From a technical point of view, further research in the information credibility assessment field must deal with the above-mentioned issues, by developing, for example, hybrid approaches able to simultaneously exploit domain knowledge, model-driven aspects, and supervised learning when unbiased training data are available, even of a small number. Moreover, a problem of particular relevance in recent years is that of early identification of false information  or of hate speech  (whose detection is a problem that is related to information credibility assessment), to act immediately to avoid the diffusion of contents that are harmful in several respects (e.g., racist, sexist, homophobic, scientifically inaccurate contents) before they can cause further damage to the community.
Another problem that should be addressed in the near future is how to consider, in the credibility assessment process, the amount of confidential information that is released within UGC, an aspect that has not been sufficiently investigated in the literature until now [6, 27]. On the one hand, it is necessary to develop solutions to protect this type of information, on the other hand, how to use it (once properly sanitized) to have additional evidence about the reliability of the source of information and therefore of the credibility of her/his content.
From the point of view of the research contexts in which the problem has been addressed in recent years, it must be emphasized that, while the study of techniques for opinion spam detection and fake news detection has seen the scientific community particularly prolific [19, 42], the study of the problem of evaluating the credibility of health-related information can still be considered at a preliminary level. In fact, at the time of writing, despite the great interest that this area has, there are still few solutions that started to evaluate the credibility of the health-related UGC spreading on social media, e.g., . Most approaches have focused on assessing the credibility of the means of communication that disseminates the information, e.g., the credibility of websites or blogs that publish medical information [2, 49]. Other works have done meta-analytical studies, e.g., . It is therefore necessary to develop automatic or semi-automatic approaches that help people not to incur the possible negative consequences that the social “word-of-mouth” can have, especially in such a delicate context as that of health.
Quality of wikipedia articles: analyzing features and building a ground truth for supervised classification.
Proceedings of the 11th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR, pp. 338–346. External Links: Cited by: 2nd item.
-  (2006) The effect of source credibility on consumers’ perceptions of the quality of health information on the internet. Medical informatics and the Internet in medicine 31 (1), pp. 45–52. Cited by: §0.1, §0.4.
-  (2019) Bubble trouble: strategies against filter bubbles in online social networks. In International Conference on Human-Computer Interaction, pp. 441–456. Cited by: §0.1.
-  (2012) A multi-dimensional and event-based model for trust computation in the Social Web. In Proc. of SocInfo, Cited by: §0.2.2.
-  (2015) Automatic deception detection: methods for finding fake news. Proceedings of the Association for Information Science and Technology 52 (1), pp. 1–4. Cited by: §0.1.
-  (2009) Trading anonymity for influence in open communities voting schemata. In 2009 International Conference on Social Informatics (SocInfo’09), pp. 63–67. Cited by: §0.4.
-  (2017) Automated hate speech detection and the problem of offensive language. In Eleventh international aaai conference on web and social media, Cited by: §0.4.
-  (2019) Fake news detection in microblogging through quantifier-guided aggregation. In International Conference on Modeling Decisions for Artificial Intelligence, pp. 64–76. Cited by: §0.1.
-  (2015) Echo chambers in the age of misinformation. arXiv preprint arXiv:1509.00189. Cited by: §0.1.
-  (2008) Credibility of health information and digital media: new perspectives and implications for youth. In Digital Media, Youth, and Credibility, pp. 123–154. Cited by: §0.1.
-  (2013) Exploiting burstiness in reviews for review spammer detection.. ICWSM 13, pp. 175–184. Cited by: §0.3.1.
-  (2013) Privacy in social collaboration. In Handbook of Human Computation, pp. 857–878. Cited by: §0.1.
-  (2008) Digital Media and Youth: Unparalleled Opportunity and Unprecedented Responsibility. In Digital Media, Youth, and Credibility, The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning, pp. 5–28. External Links: Cited by: §0.2.2.
-  (1999) The elements of computer credibility. In Proc. of SIGCHI, Cited by: §0.2.1.
-  (1995) Ontologies and knowledge bases towards a terminological clarification. Towards very large knowledge bases: knowledge building & knowledge sharing 25 (32), pp. 307–317. Cited by: §0.3.2.
-  (2010) A decade of application of the choquet and sugeno integrals in multi-criteria decision aid. Annals of Operations Research 175 (1), pp. 247–286. Cited by: §0.3.2.
-  (2016) Multiple criteria decision analysis. Springer. Cited by: §0.3.2.
-  (2014) Developing online health communities through digital media. International Journal of Information Management 34 (2), pp. 311–314. Cited by: §0.1.
-  (2015) Detection of review spam: a survey. Expert Systems with Applications 42 (7), pp. 3634–3642. Cited by: §0.4.
-  (2018) From machine learning to explainable AI. In Proc. of DISA, Cited by: §0.3.2.
-  (1953) Communication and persuasion. New Haven: Yale University Press. Cited by: §0.2.1.
-  (2008) Echo chamber: rush limbaugh and the conservative media establishment. Oxford University Press. Cited by: §0.1.
-  (2008) Opinion spam and analysis. In Proceedings of the 2008 International Conference on Web Search and Data Mining, pp. 219–230. Cited by: §0.1.
-  (2010) What is twitter, a social network or a news media?. In Proceedings of the 19th international conference on World wide web, pp. 591–600. Cited by: §0.1.
-  (2000) The limited capacity model of mediated message processing. Journal of communication 50 (1), pp. 46–70. Cited by: §0.2.2.
-  (2018) Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks. In Thirty-Second AAAI Conference on Artificial Intelligence, Cited by: §0.4.
-  (2019) Data confidentiality and information credibility in online ecosystems. In Proceedings of the 11th International Conference on Management of Digital EcoSystems (MEDES’19), Limassol, Cyprus, pp. 12–14. Cited by: §0.4.
-  (2016) Fake it till you make it: reputation, competition, and yelp review fraud. Management Science 62 (12), pp. 3412–3427. Cited by: §0.3.2.
-  (2017) User generated content and credibility evaluation of online health information: a meta analytic study. Telematics and Informatics 34 (5), pp. 472–486. Cited by: §0.4.
-  (2007) Making sense of credibility on the web: models for evaluating online information and recommendations for future research. JASIST 58 (13), pp. 2078–2091. Cited by: §0.2.2, §0.2.
-  M. Moens, J. Li, and T. Chua (Eds.) (2014) Mining user generated content. Social Media and Social Computing, Chapman and Hall/CRC. Cited by: §0.2.2.
-  (2013) Spotting opinion spammers using behavioral footprints. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 632–640. Cited by: 1st item.
-  (2014) People on drugs: credibility of user statements in health communities. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 65–74. Cited by: §0.4.
-  (2011) The filter bubble: what the internet is hiding from you. Penguin UK. Cited by: §0.1.
-  (2019) A Multi-Criteria Decision Making approach based on the Choquet integral for assessing the credibility of User-Generated Content. Information Sciences 503, pp. 574–588. Cited by: §0.3.2.
-  (1993) The psychology of judgment and decision making.. Mcgraw-Hill Book Company. Cited by: §0.1.
-  (2010) The social media bible: tactics, tools, and strategies for business success. 2nd edition, Wiley Publishing. Cited by: §0.2.2.
-  (2019) On the inevitability of online echo chambers. arXiv preprint arXiv:1905.03919. Cited by: §0.1.
-  (2005) Text categorization. In Encyclopedia of Database Technologies and Applications, pp. 683–687. Cited by: 1st item.
-  (2014) Credibility. In An integrated approach to communication theory and research, pp. 449–470. Cited by: §0.2.1, §0.2.1.
-  (2018) The spread of low-credibility content by social bots. Nature communications 9 (1), pp. 4787. Cited by: §0.3.1.
-  (2019) Combating fake news: a survey on identification and mitigation techniques. ACM Transactions on Intelligent Systems and Technology (TIST) 10 (3), pp. 1–42. Cited by: §0.4.
-  (2016) Fact checking in heterogeneous information networks. In Proc. of WWW, Cited by: §0.3.2.
-  (2017-09) Fake news detection on social media: a data mining perspective. SIGKDD Explor. Newsl. 19 (1), pp. 22–36. External Links: Cited by: 3rd item.
-  (2018) Automated fact checking: task formulations, methods and future directions. arXiv preprint arXiv:1806.07687. Cited by: §0.3.2.
-  (2016) A multi-criteria decision making approach for the assessment of information credibility in social media. In Proc. of WILF, Cited by: §0.3.2.
-  (2017) Credibility in Social Media: Opinions, News, and Health Information - A Survey. WIREs Data Mining and Knowledge Discovery 7 (5). Cited by: §0.3.2, §0.3, §0.4.
-  (2017) Quantifier guided aggregation for the veracity assessment of online reviews. International Journal of Intelligent Systems 32 (5), pp. 481–501. Cited by: §0.3.2.
-  (2008) Health information, credibility, homophily, and influence via the internet: web sites versus discussion groups. Health communication 23 (4), pp. 358–368. Cited by: §0.4.
-  (2012) The ordered weighted averaging operators: theory and applications. Springer Science. Cited by: §0.3.2.