Aligning Daily Activities with Personality: Towards A Recommender System for Improving Wellbeing

09/09/2019 ∙ by Mohammed Khwaja, et al. ∙ Telefonica Imperial College London 0

Recommender Systems have not been explored to a great extent for improving health and subjective wellbeing. Recent advances in mobile technologies and user modelling present the opportunity for delivering such systems, however the key issue is understanding the drivers of subjective wellbeing at an individual level. In this paper we propose a novel approach for deriving personalized activity recommendations to improve subjective wellbeing by maximizing the congruence between activities and personality traits. To evaluate the model, we leveraged a rich dataset collected in a smartphone study, which contains three weeks of daily activity probes, the Big-Five personality questionnaire and subjective wellbeing surveys. We show that the model correctly infers a range of activities that are 'good' or 'bad' (i.e. that are positively or negatively related to subjective wellbeing) for a given user and that the derived recommendations greatly match outcomes in the real-world.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction and related work

The pursuit of happiness is the ultimate goal for many people - described by Aristotle as the meaning and purpose of life. Interestingly, the scientific pursuit of happiness and life satisfaction (together referred to as ”subjective wellbeing”, or SWB) has intensified in the last quarter of the 20th century. Since then, the number of articles in the domain of SWB have grown exponentially (Myers, 2000). Now we have a wealth of knowledge (Diener et al., 2018) on how to measure SWB and a deeper understanding on how it correlates with cognitive, behavioral, environmental, biological and genetical factors. On the other hand, interdisciplinary research has successfully demonstrated the potential of mobile computing to quantify and monitor human behaviors accurately and at a scale larger than ever before (Matic et al., 2012; Miluzzo et al., 2007; Atallah and Yang, 2009; Srivastava et al., 2012), gain insight into mental wellbeing (Servia-Rodríguez et al., 2017; Lane et al., 2011; Lathia et al., 2013) and automatically predict user characteristics such as personality traits (Khwaja et al., 2019; Khwaja and Matic, 2019). The advances of mobile technologies coupled with knowledge from SWB research can open the door to proactive recommendations to help people make decisions or adapt their daily activities to maximize happiness and life satisfaction. While the field of recommender systems (RSs) has provided numerous tools to support user decision making by identifying personalized and relevant content, services or products (Ricci et al., 2015), RSs that provide personalized suggestions to boost SWB have not attracted a considerable research interest yet.

One of the key issues of utilizing advances in RSs for providing SWB recommendations is the existence of appropriate datasets. Contrary to the availability of rich datasets on movies, music, books, products, etc. matched with user profiles, it is not trivial to collect equally large datasets that match precise information about users’ activities and ground-truth information about their SWB. This represents a common ’cold-start’ problem – a situation of having sparse historical data or not having enough information about new users (Hu and Pu, 2011; Tkalcic and Chen, 2015). In order to tackle the cold-start problem in developing RSs for SWB, we use personality traits in this study as a proxy to user profiles, relevant for matching daily activities to subjective wellbeing at an individual level. Our approach is inspired by the congruence between personality and daily activities identified in psychology (Diener et al., 1999) as well as by recent trends of leveraging personality traits for addressing the cold-start problem in RSs (Hu and Pu, 2011; Tkalcic et al., 2009; Elahi et al., 2013). Personality has been used successfully to recommend movies (Karumur et al., 2016; Fernández-Tobías et al., 2016; Wu et al., 2013), music (Hu and Pu, 2011; Fernández-Tobías et al., 2016; Ferwerda and Schedl, 2016), books (Fernández-Tobías et al., 2016), and also leisure activities and events (Nurbakova et al., 2017). However the aim of these studies was to optimize item or event selection rates rather than to optimize users’ SWB. Behavioral Economics suggests that people have biases in understanding the link between behaviours and their SWB (Kahneman and Krueger, 2006). Importantly, literature also suggests that it is possible to improve an individual’s SWB111

Despite the fact that genetical predisposition explains 50% of SWB variance

(Lyubomirsky et al., 2005; Diener et al., 1999). Our work is novel in providing a technological foundation for an RS to support people in improving SWB, with the following distinct contributions:

  • A machine learning algorithm that predicts users’ SWB based on the congruence between their reported Big-Five personality traits and distribution of their activities (Section 

    3).

  • Evaluation of the predictive power of the algorithm in simulating distribution of activities that would result in low or high SWB for a specific user - thus constituting a personalized whitelist and blacklist for increasing SWB (Section 4).

2. In-the-wild Data Collection

To build the dataset required for the SWB RS, we used a smartphone app that captures patterns of daily activities that people were engaged in over a period of 2-3 weeks. The onboarding survey (delivered through the smartphone app) prompted participants to answer the Big-Five personality questionnaire (Goldberg et al., 2006) and a SWB question. We used the question ’Overall, how satisfied are you with your life nowadays’ as the ground-truth information for SWB, referred to as ”life satisfaction” (Dolan et al., 2011). During the following 2-3 weeks, we applied the Ecological Momentary Assessment method (Shiffman et al., 2008)

by prompting participants at five random times distributed over a day to report the activity they were engaged in at that moment.

The data collection was conducted in two trials. The first trial (first dataset) ran between February and August 2018 with 151 participants completing the study. Participants in this sample were involved through a recruitment agency from five different countries (UK, Spain, Colombia, Peru and Chile). The second trial (second dataset) was conducted between January and March 2019 with 256 participants completing the study from a major UK university. Participants in this sample were recruited using an email sent to students and staff at the university. For their effort to complete the study, participants in both samples were rewarded with a monetary contribution. We used the first dataset to build the model described in Section 3 and the second to validate the RS described in Section 4. We considered this to be a rigorous way of testing the generality of our model than, for instance, performing cross-validation or using dataset splitting methodologies. This is attributed to the fact that the two datasets included different demographics (in terms of level of education, socio-economic status, age and nationality) and were collected during different periods of the year to account for potential seasonal effects.

3. Modelling Congruence and its relationship with wellbeing

Psychologists have emphasized that the congruence between internal and external factors is a predictor of SWB (Diener et al., 1999). In practice, internal factors are usually static or slow-changing (inherent to an individual’s innate characteristics), whereas external factors are mostly dynamic (exhibited by an individual’s behaviour and environmental circumstances). Inspired by this, we defined the congruence user model that quantifies the alignment between the internal and external factors (i.e. internal and external user models). Figure 1 summarizes the development process of the congruence model and it takes into account the difference between the internal and external user models to predict SWB. As an example, the external model may indicate that a user behaves as an introverted person (based on her activities) although she is an extroverted person (based on the Big-Five questionnaire). The congruence model quantifies the gap between the two. Subsequently, we explore if the machine learning model that considers this misalignment between who she is (internal model) and how she behaves (external model) predicts a lower SWB. The performance of this model is evaluated using the first dataset, and then the generated recommendations at an individual level are validated with the second dataset.

Figure 1. Classification of high/low SWB based on the congruence user model.

3.1. Congruence User Model

We used the Big-Five personality traits (Goldberg et al., 2006) to represent the user’s internal model, as they are one of the most representative personal characteristics that describe and predict human behaviour (Corr and Matthews, 2009). As personality is obtained from a self-reported questionnaire, we refer to this as the reported personality. Formally, for user

, the reported personality vector is given by:

(1)

where, represent the 5 reported personality (extraversion, agreeableness, conscientiousness, neuroticism and openness) scores respectively, for the user.

We built a user’s external user model from his/her patterns of daily activities, e.g., eating, working, watching TV, shopping, listening to music, using social media, exercising, and so on. Due to the variety of momentary activity items, we grouped the reported activity items into activity categories as defined by Goldberg (Goldberg, 2009). Goldberg conducted a 10 year long study with 800 individuals, and clustered 400 activity items into 33 categories. For simplicity, we refer to the activity categories as activities throughout the paper. To model the alignment between personality and activities, we also used Goldberg’s study as the state-of-the-art dataset to provide correlations between activities and personality traits.

First, we captured the distribution of a user’s activities, which corresponds to the normalized frequencies of all the activities that the user reports. Mathematically, this is represented by a vector as: . Here, is the frequency of activity for user . Moreover, the sum of all components of for any user equals 1. Our external model builds a secondary personality that is exhibited through the user’s activity distribution, to directly compare against the internal model. We computed a dynamic construct called exhibited personality, that contains five dimensions similar to and is modelled based on the user’s activity distribution . For a user , the exhibited personality vector is given by:

(2)

where, represent the 5 mapped personality scores, as a function of . To obtain , we first acquired a weight vector that defines the positive/negative accumulated effect (or weight) of each activity on the traits. By using the correlation matrix between activities and personality defined in Goldberg (Goldberg, 2009), represented as , we define the weight of exhibited activities on the personality as: . Here, is the matrix product and is obtained by using as a column matrix. Using this, we derived the weighted median personality that represents the change over or below the median personality exhibited by one’s activity patterns. Thus, is obtained as the vector sum of the median personality and the weighted median personality :

(3)

indicating that , where is the Hadamard product. Finally, the congruence user model of a person is the difference between and , normalised by . This difference, or delta, is given by:

(4)

where are the delta components of extraversion, agreeableness, conscientiousness, neuroticism and openness respectively. As each component of decreases, more congruent is the user’s behaviour with respect to his/her personality; and as per our hypothesis, more is the SWB.

Model Type (Features) Accuracy Kappa AUC
Personality () 57% 0.11 0.55
Activity () 52% 0.04 0.52
Personality-Activity () 61% 0.18 0.58
Congruence () 70% 0.38 0.71
Table 1. Classification of SWB

3.2. Experimentation and Results

We built a machine learning model that predicts the SWB score for a user , by using the individual delta scores along the five personality dimensions, . For preforming this analysis, we use the first dataset described in Section 2. We cluster momentary activity items into 15 activity categories from the 33 defined in (Goldberg, 2009), that are most relevant to items reported in the dataset. The values of each component of

vary from 10-50 and SWB is rated on a scale from 1-10. We treat the prediction of SWB as a binary classification problem by dividing the continuous variable into high (1) and low (0), using the median value as the threshold. We use the leave-one-sample-out method to evaluate the the model accuracy. We tested different machine learning algorithms, namely: random forest, naïve bayes and support vector machine (SVM), and observed that the latter performs the best. For brevity, we report results with SVM only. To assess the added value of the congruence user model, we compared it to classifiers that used only personality traits

, only activity distributions and the combination of the two , shown in Table 1. We observed that the classifier relying on the congruence user model and the computed delta features outperforms the other classifiers in predicting SWB.

This illustrates the strength of incorporating the congruence theory into the SWB classification model, in comparison to using a typical ”black-box” model that relies on the same inputs – personality and activity distributions. By relying on this model, an RS to improve a user’s SWB would aim to suggest activity distributions that improves the personality-activities alignment, i.e. reduce the gap quantified through delta features. Certainly, there is more than one distribution of activities that corresponds to low values of delta features for a given user, and the importance of minimizing all of the five delta values (i.e. alignment with all the five personality traits) is not equally important. In our experiments, we observed that the relative ratio among different activities and also among delta features matters more than the absolute values of activity frequency or the delta values. This also corroborates with an intuitive assumption that there is no one unique lifestyle beneficial for an individual - i.e. understanding the range of activity distributions is beneficial to improving a user’s SWB.

4. Activity Range Recommendation

In this section we describe a methodology to recommend a range of relevant activities that are ’good’ or ’bad for a user, and subsequently test the results using the second dataset to assess if the model scales to a different population. The flow diagram of the RS, along with the validation procedure is summarized in Figure 2. Firstly, the activity range recommender compares different exhibited personalities for all simulated combinations of activity distributions with a user’s reported personality that will result in high or low SWB, using the SWB classifier described in Section 3.2. Through this process we obtain the range of activity distributions that form the whitelist (good) and the blacklist (bad) for the user’s SWB. The whitelist recommendation is validated by comparing against the actual activity distribution for users in the second dataset that have high SWB. The same is done for the blacklist recomendation with users that have low SWB.

Figure 2. Model for the activity range recommendation, along with the validation procedure

4.1. Methodology

Though the sum of all components of equals 1 for a user , i.e., , some of these components vary significantly across the sample, while others show a lower variance. The activities that do not vary across the sample also do not predominantly impact the calculated values of exhibited personality . It is also intuitively clear that there are certain activities (usually those that we have less control of) that occupy a similar proportion of time spent by most people (such as working, studying, sleeping, eating, etc.). Hence, to narrow down the list of recommended activities (such as using social media, watching TV, reading, exercising, etc.) that may be more actionable (i.e. more under a user’s control) for providing recommendations, we consider only those activities that have high variance in the sample. Without loss of generality, it is assumed that there are activities that have high variance from activities (with ) so that . Here, is a relatively low value (0.2) that covers the joined variance of the activities. For the selected activities, we obtained all potential combinations in increments of 0.1 such that this condition is met. For each of these combinations, we calculate and compute the delta, for each user by comparing against . We predict the SWB for all possibilities using the SWB classifier described in Section 3, and marked the cases where this is high. Using these, we determined the range (sorted from lowest to highest) of distributions for each of the activities that are expected to give high SWB. We performed the same procedure for the cases of low SWB, indicating the range of activity distributions that result in low SWB.

4.2. Validation and Results

We use the second dataset described in Section 2 to validate the hypothetical outputs of our RS. As with the first dataset, we use the same categories to cluster the activities. We observed that clusters have significant variance () across the sample and have the most effect on , and in this dataset, the value of . Using the SWB report of users in the second dataset, we divide each user into either high or low class - if they are either over or under the median value respectively (as done previously with the first dataset). For each of the users in the high SWB class and low SWB class, we obtained the range of activity distributions that would provide high and low SWB respectively, according to our model. We evaluated the extent to which the activity proportions of the user fall within the range of all selected activities (all 8 activities) or majority (at least 5) of them. These results are reported in Table 2. It is important to note that testing an activity RS proactively would be a difficult endveour as behaviour change is a complex task, and evaluating the effectiveness of interventions is particularly difficult for health and wellbeing (Campbell et al., 2007). Contextual factors, external events and sense of autonomy play a large role in determining the extent to which a recommendation will be followed and performed by users (Deci and Ryan, 1987). Hence, we evaluated the effectiveness of the RS for both an ideal case (user performs all activities as per the recommendation) and a realistic case (user performs the majority of the activities).

When considering all the activities that are important for high & low SWB, 51% of users in the high SWB class fall in the range, while for low SWB class the number increases to 74%. This indicates that our method is able to infer the range of activity distributions that are beneficial for user’s SWB fairly well, and furthermore it is more successful in inferring the ranges of activities that have negative consequences for SWB. When including the majority of the activities (5), these numbers improve significantly - rising to 71% for users in the high SWB class and 92% in the low SWB class. This resonates well with real-life scenarios as users may not be able to maintain the exact proportion for all activities all the time.

Activities Considered Sample Under Consideration Predicted - Actual Accuracy
All (8) High SWB 51%
Low SWB 74%
Majority () High SWB 71%
Low SWB 92%
Table 2. Validation of Optimal Activity Range Recommendation System

5. Conclusion

In this paper, we presented a novel approach to developing an RS that supports users in balancing daily activities to improve their subjective wellbeing (SWB). Developing an SWB RS is a challenging task, as it faces the cold-start problem due to the lack of large-scale datasets that map user characteristics, daily activities and ground-truth information about SWB. We addressed this limitation by collecting a dataset with the above information and developed a user model based on psychological literature suggesting that the congruence between internal and external factors impacts SWB (Diener et al., 1999). Using this model, we built a binary classifier that predicts SWB based on the alignment between an individual’s behavior (in terms of distribution of activities) and his/her personality. Our model outperformed the three benchmarking classifiers that relied on the same input parameters, by 9-18%. Subsequently, we simulated the range of activity distributions that would result in high or low SWB for users in another dataset, and compared personalized recommendations (i.e. white- and black- listed activity distributions) to the ground-truth of users’ activity distributions. Our model inferred the range of activities that are ’good’ or ’bad’ for a given user with accuracy up to 92%, demonstrating that the model successfully captures the link between SWB and the alignment between real-life activity patterns with personality. We believe that this work will encourage more research, both in RSs devoted to SWB, and in psychology, aiming to deepen understanding of the congruence between individual’s daily activities and personality.

In addition to its development, testing the impact of an RS of this nature is a challenge, as the inference and provision of perfectly aligned activity distributions to one’s SWB does not guarantee that a user will follow the recommendations. For this reason, we have created a tool that suggests daily activities to promote SWB. We plan to test this tool in the near-future and explore the change in SWB achieved through the suggestions provided by our system.

Acknowledgements

This work has been supported from funding awarded by the European Union’s Horizon 2020 research and innovation programme, under the Marie Sklodowska-Curie grant agreement no. 722561.

References

  • (1)
  • Atallah and Yang (2009) Louis Atallah and Guang-Zhong Yang. 2009. The use of pervasive sensing for behaviour profiling - a survey. Pervasive and mobile computing 5, 5 (2009), 447–464.
  • Campbell et al. (2007) Neil C Campbell, Elizabeth Murray, Janet Darbyshire, Jon Emery, Andrew Farmer, Frances Griffiths, Bruce Guthrie, Helen Lester, Phil Wilson, and Ann Louise Kinmonth. 2007. Designing and evaluating complex interventions to improve health care. Bmj 334, 7591 (2007), 455–459.
  • Corr and Matthews (2009) Philip J Corr and Gerald Matthews. 2009. The Cambridge handbook of personality psychology. Cambridge University Press Cambridge, UK:.
  • Deci and Ryan (1987) Edward L Deci and Richard M Ryan. 1987. The support of autonomy and the control of behavior. Journal of personality and social psychology 53, 6 (1987), 1024.
  • Diener et al. (2018) Ed Diener, Shigehiro Oishi, and Louis Tay. 2018. Advances in subjective well-being research. Nature Human Behaviour (2018), 1.
  • Diener et al. (1999) Ed Diener, Eunkook M Suh, Richard E Lucas, and Heidi L Smith. 1999. Subjective well-being: Three decades of progress. Psychological bulletin 125, 2 (1999), 276.
  • Dolan et al. (2011) Paul Dolan, Richard Layard, and Robert Metcalfe. 2011. Measuring subjective well-being for public policy. (2011).
  • Elahi et al. (2013) Mehdi Elahi, Matthias Braunhofer, Francesco Ricci, and Marko Tkalcic. 2013.

    Personality-based active learning for collaborative filtering recommender systems. In

    Congress of the Italian Association for Artificial Intelligence

    . Springer, 360–371.
  • Fernández-Tobías et al. (2016) Ignacio Fernández-Tobías, Matthias Braunhofer, Mehdi Elahi, Francesco Ricci, and Iván Cantador. 2016. Alleviating the new user problem in collaborative filtering by exploiting personality information. User Modeling and User-Adapted Interaction 26, 2-3 (2016), 221–255.
  • Ferwerda and Schedl (2016) Bruce Ferwerda and Markus Schedl. 2016. Personality-based user modeling for music recommender systems. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 254–257.
  • Goldberg (2009) Lewis R Goldberg. 2009.

    Personality, Demographics, and Self-Reported Behavioral Acts: The Development of Avocational Interest Scales from Estimates of the Amount of Time Spent in Interest-Related Activities.

    Then a miracle occurs: Focusing on behavior in social psychological theory and research (2009), 205.
  • Goldberg et al. (2006) Lewis R Goldberg, John A Johnson, Herbert W Eber, Robert Hogan, Michael C Ashton, C Robert Cloninger, and Harrison G Gough. 2006. The international personality item pool and the future of public-domain personality measures. Journal of Research in personality 40, 1 (2006), 84–96.
  • Hu and Pu (2011) Rong Hu and Pearl Pu. 2011. Enhancing collaborative filtering systems with personality information. In Proceedings of the fifth ACM conference on Recommender systems. ACM, 197–204.
  • Kahneman and Krueger (2006) Daniel Kahneman and Alan B Krueger. 2006. Developments in the measurement of subjective well-being. Journal of Economic perspectives 20, 1 (2006), 3–24.
  • Karumur et al. (2016) Raghav Pavan Karumur, Tien T Nguyen, and Joseph A Konstan. 2016. Exploring the value of personality in predicting rating behaviors: a study of category preferences on movielens. In Proceedings of the 10th ACM conference on recommender systems. ACM, 139–142.
  • Khwaja and Matic (2019) Mohammed Khwaja and Aleksandar Matic. 2019. Personality is Revealed During Weekends: Towards Data Minimisation for Smartphone Based Personality Classification. In Proceedings of the 17th IFIP Conference on Human-Computer Interaction. Springer.
  • Khwaja et al. (2019) Mohammed Khwaja, Sumer S. Vaid, Sara Zannone, Gabriella M. Harari, A. Aldo Faisal, and Aleksandar Matic. 2019. Modeling Personality vs. Modeling Personalidad: In-the-wild Mobile Data Analysis in Five Countries Suggests Cultural Impact on Personality Models. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 88.
  • Lane et al. (2011) Nicholas D Lane, Mashfiqui Mohammod, Mu Lin, Xiaochao Yang, Hong Lu, Shahid Ali, Afsaneh Doryab, Ethan Berke, Tanzeem Choudhury, and Andrew Campbell. 2011. Bewell: A smartphone application to monitor, model and promote wellbeing. In 5th international ICST conference on pervasive computing technologies for healthcare. 23–26.
  • Lathia et al. (2013) Neal Lathia, Veljko Pejovic, Kiran K Rachuri, Cecilia Mascolo, Mirco Musolesi, and Peter J Rentfrow. 2013. Smartphones for large-scale behavior change interventions. IEEE Pervasive Computing 12, 3 (2013), 66–73.
  • Lyubomirsky et al. (2005) Sonja Lyubomirsky, Kennon M Sheldon, and David Schkade. 2005. Pursuing happiness: The architecture of sustainable change. Review of general psychology 9, 2 (2005), 111–131.
  • Matic et al. (2012) Aleksandar Matic, Venet Osmani, Alban Maxhuni, and Oscar Mayora. 2012. Multi-modal mobile sensing of social interactions. In 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops. IEEE, 105–114.
  • Miluzzo et al. (2007) Emiliano Miluzzo, Nicholas D Lane, Shane B Eisenman, and Andrew T Campbell. 2007. CenceMe–injecting sensing presence into social networking applications. In European Conference on Smart Sensing and Context. Springer, 1–28.
  • Myers (2000) David G Myers. 2000. Hope and happiness.
  • Nurbakova et al. (2017) Diana Nurbakova, Léa Laporte, Sylvie Calabretto, and Jérôme Gensel. 2017. Users psychological profiles for leisure activity recommendation: user study. In Proceedings of the International Workshop on Recommender Systems for Citizens. ACM, 3.
  • Ricci et al. (2015) Francesco Ricci, Lior Rokach, and Bracha Shapira. 2015. Recommender systems: introduction and challenges. In Recommender systems handbook. Springer, 1–34.
  • Servia-Rodríguez et al. (2017) Sandra Servia-Rodríguez, Kiran K Rachuri, Cecilia Mascolo, Peter J Rentfrow, Neal Lathia, and Gillian M Sandstrom. 2017.

    Mobile sensing at the service of mental well-being: a large-scale longitudinal study. In

    Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 103–112.
  • Shiffman et al. (2008) Saul Shiffman, Arthur A Stone, and Michael R Hufford. 2008. Ecological momentary assessment. Annu. Rev. Clin. Psychol. 4 (2008), 1–32.
  • Srivastava et al. (2012) Mani Srivastava, Tarek Abdelzaher, and Boleslaw Szymanski. 2012. Human-centric sensing. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 370, 1958 (2012), 176–197.
  • Tkalcic and Chen (2015) Marko Tkalcic and Li Chen. 2015. Personality and recommender systems. In Recommender systems handbook. Springer, 715–739.
  • Tkalcic et al. (2009) Marko Tkalcic, Matevz Kunaver, Jurij Tasic, and Andrej Košir. 2009. Personality based user similarity measure for a collaborative recommender system. In Proceedings of the 5th Workshop on Emotion in Human-Computer Interaction-Real world challenges. 30–37.
  • Wu et al. (2013) Wen Wu, Li Chen, and Liang He. 2013. Using personality to adjust diversity in recommender systems. In Proceedings of the 24th ACM conference on hypertext and social media. ACM, 225–229.