The Potential of the Confluence of Theoretical and Algorithmic Modeling in Music Recommendation

11/17/2019 ∙ by Christine Bauer, et al. ∙ Johannes Kepler University Linz 0

The task of a music recommender system is to predict what music item a particular user would like to listen to next. This position paper discusses the main challenges of the music preference prediction task: the lack of information on the many contextual factors influencing a user's music preferences in existing open datasets, the lack of clarity of what the right choice of music is and whether a right choice exists at all; the multitude of criteria (beyond accuracy) that have to be met for a "good" music item recommendation; and the need for explanations on relationships to identify (and potentially counteract) unwanted biases in recommendation approaches. The paper substantiates the position that the confluence of theoretical modeling (which seeks to explain behaviors) and algorithmic modeling (which seeks to predict behaviors) seems to be an effective avenue to take in computational modeling for music recommender systems.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

Before the era of the Internet, access to music content (e.g., music recordings) was restricted to local availability of their physical representations (e.g., vinyl). Thereby, the selection and aggregation of content had traditionally been exposed to human control (Oechslein and Hess, 2014). For instance, a small group of Artist&Repertoire managers working for the major music labels scouted new artists and developed them commercially.

Nowadays,—owing to the development of the Social Web that allows for easy distribution of user-generated content—the intermediary level of experts (e.g., the Artist&Repertoire managers at music labels) that traditionally “prefiltered” content before it reached potential consumers is bypassed. This results in the situation that users currently face: music content is abundantly available online and the amount of overall available content increases tremendously on a daily basis.

amples of differences associated with music items, users, and their consumption behaviour include the following(Schedl et al., 2015):

  • very low consumption time in the dimension of minutes, whereas a book or a travel are consumed during days or weeks;

  • consumption in sequences (e.g., playlists);

  • music often consumed passively (e.g., while jogging, travelling, working);

  • consumption is highly driven by situational context;

  • users are likely to appreciate the re-recommendation of the same item while a user is less likely to read the same news article over and over again; and

  • music evokes strong emotions.

The specialties of the music domain

However, the opportunity to access a large amount of content frequently leads to information overload (Bawden and Robinson, 2009) or choice overload (Häubl and Trifts, 2000), because people do not find the content that they are interested in or do not know what to choose. Assisting users in searching, sorting, and filtering the massive amount of online content (Montaner et al., 2003), recommender systems have become important tools in people’s everyday life and do not only facilitate the interaction with music content (Celma, 2010), but also support versatile activities such as shopping (Oestreicher-Singer and Sundararajan, 2012), consuming news (Oechslein and Hess, 2014), or finding persons for any kind of social matching (Mayer et al., 2015).

Recommender systems are computer systems that provide suggestions for items that are deemed interesting to a particular target user, assisting that particular user in various decision-making processes (e.g., relating to what music to listen to) (Ricci et al., 2015). The general term used to denote to what the system recommends to users is “item” (Ricci et al., 2015); in case of music recommender systems (MRS) it is the music item (e.g., musical work, artist, genre).

There are universally valid principles for designing recommender systems, such as that a recommender system typically consist of three key components (i.e., user, item, and matching mechanism) (Bauer et al., 2017). Still, a recommender system needs to be put into context because there are product- and sector-specific characteristics that a recommender system needs to consider (be customized to) to provide useful and effective recommendations for the specific type of item  (Ricci et al., 2015; Schedl et al., 2014). Sidebar 1 presents the specialties of the music domain compared to other domains deploying recommender systems.

2. Rationale

An ideal MRS proposes “the right music, to the right user, at the right moment” 

(Laplante, 2014). However, this is a complex task because various factors influence a user’s music preferences in a given situation (Bauer and Schedl, 2018b). Many studies have investigated the relationships between music preferences and various person-related characteristics (e.g., demographics (Cheng et al., 2017), personality traits (Schäfer and Mehlhorn, 2017), social influences (Bonneville-Roussy and Rust, 2018). Besides person-related characteristics, also situation-related factors (e.g., temporal aspects (Dias and Fonseca, 2013), or weather (Braunhofer et al., 2011)) influence a user’s music preferences. The task of an MRS is to predict what a particular user would like to listen to next. Basically, there are two computational modeling approaches to build upon for this music preference prediction task:

  • Theoretical modeling seeks to explain users’ listening behavior. For advancing MRS, the first step would be to observe a user’s listening behavior and perform analyses to explain where a user’s listening behavior results from (e.g., from person-related characteristics or situational factors, and from which of these in particular). Then, building on these findings (e.g., knowing that Finnish listeners are more likely to prefer heavy metal than Italian listeners (Schedl, 2017)), future user models may be created for predictions.

  • Algorithmic modeling seeks to predict users’ listening behavior. Algorithmic modeling may rely on approaches that are capable of identifying listening patterns within a user’s listening history or across users without necessarily delivering descriptions that help explaining

    the relationships of the identified patterns. For instance, approaches such as deep neural networks frequently leave us with “black boxes” 

    (Koh and Liang, 2017) because the resulting models are complex and frequently they do not produce an intelligible description of the results produced in each case. Still, the resulting models may be apt to deliver remarkably accurate predictions. In other words, algorithmic modeling may recommend music to the user what he or she will indeed like in the very moment without understanding whether it was indeed the “right” choice—and if—why it was “right”.

3. Challenges

One challenge for music preference prediction is that it is (almost) impossible to say what is the right choice for a particular user in the particular moment; it is typically a set of items that is right or okay.

Another challenge of algorithmic modeling is that—currently—we can only model based on data that we have available. For MRS, several open datasets exist, such as the Million Song Dataset (Bertin-Mahieux et al., 2011), the LFM-1b dataset (Schedl, 2016), or the recently released Music Streaming Sessions Dataset (Brost et al., 2019). However, there are many factors influencing a user’s music preferences for which we do not have (sufficient) data available (yet) to exploit for algorithmic modeling. Theoretical modeling—thus, the “explaining approach”—may help here to advance MRS. It is also a viable basis to provide an informed route what kind of data should be collected so that algorithmic modeling may come into play here to use its powerful mechanisms to exploit the additional data to make even better predictions.

A further challenge relates to evaluation of MRS: What does it mean if an MRS recommends a music item to a user and the user indeed listens to the item? Potentially, it is the user’s most favorite song and so the user enjoyed listening to it. Maybe, though, the user listens to the item because the algorithm provided it as the next one to listen to in the playlist, but the user was distracted at the very moment because of receiving a phone call (or was not present in the room for some minutes). In such cases, the recommendation was maybe not a “bad” one because the user did not hear it anyways, but was it a good prediction then?

With respect to biases as inherent in recommendation systems (e.g., the popularity bias phenomenon (Celma and Cano, 2008) suggesting that over time the most popular music items tend to get more and more attention, while music items in the long tail get less and less attention (Levy and Bosteels, 2010)), the ability to understand and explain models seems to be a crucial prerequisite to uncover such bias and develop and take effective measures to counteract unwanted bias.

4. Previous and Ongoing Research, and Interests

A major part of my previous and ongoing research is aimed at integrating contextual information into (user) modeling. Basically, my work on context modeling takes a conceptual viewpoint (e.g., (Bauer and Novotny, 2017; Bauer, 2014)). It points towards the various potentially relevant contextual factors that we tend to “forget” in modeling (for various reasons such as, for instance, the non-availability of useful datasets including such contextual information).

With the main objective at improving MRS, some part of my research on MRS is geared towards identifying relationships between various aspects (such as age (Schedl and Bauer, 2017), user connections (Bauer and Schedl, 2019, 2018a), user country (Bauer and Schedl, 2018b), real-world events (Schedl et al., Conference Proceedings), mainstreaminess (Schedl and Bauer, 2017)) and music preferences or listening behavior. Findings are then used to improve MRS performance (for instance in (Schedl and Bauer, 2017; Bauer and Schedl, 2018b; Schedl and Bauer, 2017)).

To a considerable extent, ideas on the (contextual) components that could improve MRS are based on literature from various disciplines such as cognitive science (e.g., (Stevens, 2012)), social psychology (e.g., (Bonneville-Roussy et al., 2013)), and computer science (Laplante, 2014). In addition, ideas emanate from my own experience of many years in the music domain—which is a significant knowledge source that is not available to every researcher.

5. Prospects

Overall, recommender systems research has predominantly focused on improving the prediction accuracy of algorithms based on existing datasets (reflecting users’ historic item ratings or consumption behavior) (Beel et al., 2013). However, to date, comprehensive contextual information about users and the specific situational settings in which those consume the items is rarely available in existing datasets (Adomavicius and Tuzhilin, 2015)—and is especially true for music-related datasets.

The confluence of theoretical and algorithmic modeling seems to be an effective avenue to take in computational modeling for MRS.


  • G. Adomavicius and A. Tuzhilin (2015) Context-aware recommender systems. Book Section In Recommender Systems Handbook, F. Ricci, L. Rokach, and B. Shapira (Eds.), pp. 191–226. External Links: Document Cited by: §5.
  • C. Bauer, M. Kholodylo, and C. Strauss (2017) Music recommender systems: challenges and opportunities for non-superstar artists. Conference Proceedings In 30th Bled eConference, pp. 21–32. External Links: ISBN 978-961-286-043-1, Document Cited by: §1.
  • C. Bauer and A. Novotny (2017) A consolidated view of context for intelligent systems. Journal of Ambient Intelligence and Smart Environments 9 (4), pp. 377–393. External Links: ISSN 18761372, 18761364, Document Cited by: §4.
  • C. Bauer and M. Schedl (2018a) Investigating cross-country relationship between users’ social ties and music mainstreaminess. Conference Proceedings In 19th International Society for Music Information Retrieval Conference, ISMIR‘18, pp. 678–686. Cited by: §4.
  • C. Bauer and M. Schedl (2018b) On the importance of considering country-specific aspects on the online-market: an example of music recommendation considering country-specific mainstream. Conference Proceedings In 51st Hawaii International Conference on System Sciences, HICSS‘18, pp. 3647–3656. External Links: ISBN 978-0-9981331-1-9, Link Cited by: §2, §4.
  • C. Bauer and M. Schedl (2019) A cross-country investigation of user connection patterns in online social networks. Conference Proceedings In 52nd Hawaii International Conference on System Sciences, HICSS‘19, pp. 2166–2175. External Links: ISBN 978-0-9981331-2-6, Link Cited by: §4.
  • C. Bauer (2014) A framework for conceptualizing context for intelligent systems (ccfis). Journal of Ambient Intelligence and Smart Environments 6 (4), pp. 403–417. External Links: ISSN 1876-1364, Document Cited by: §4.
  • D. Bawden and L. Robinson (2009) The dark side of information: overload, anxiety and other paradoxes and pathologies. Journal of Information Science 35 (2), pp. 180–191. External Links: ISSN 0165-5515, 1741-6485, Document Cited by: §1.
  • J. Beel, M. Genzmehr, S. Langer, A. Nürnberger, and B. Gipp (2013) A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation. In International Workshop on Reproducibility and Replication in Recommender Systems Evaluation, RepSys‘13, pp. 7–14. External Links: ISBN 978-1-4503-2465-6, Document Cited by: §5.
  • T. Bertin-Mahieux, D. P.W. Ellis, B. Whitman, and P. Lamere (2011) The million song dataset. In 12th International Conference on Music Information Retrieval, ISMIR‘11. Cited by: §3.
  • A. Bonneville-Roussy, P. J. Rentfrow, M. K. Xu, and J. Potter (2013) Music through the ages: trends in musical engagement and preferences from adolescence through middle adulthood. Journal of Personality and Social Psychology 105 (4), pp. 703–717. External Links: Document Cited by: §4.
  • A. Bonneville-Roussy and J. Rust (2018) Age trends in musical preferences in adulthood: 2. sources of social influences as determinants of preferences. Musicae Scientiae 22 (2), pp. 175–195. External Links: Document Cited by: §2.
  • M. Braunhofer, M. Kaminskas, and F. Ricci (2011) Recommending music for places of interest in a mobile travel guide. In 5th ACM Conference on Recommender Systems, RecSys‘11, pp. 253–256. Cited by: §2.
  • B. Brost, R. Mehrotra, and T. Jehan (2019) The music streaming sessions dataset. In The Web Conference 2019, Cited by: §3.
  • Ò. Celma and P. Cano (2008) From hits to niches?: or how popular artists can bias music recommendation and discovery. In 2Nd KDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition, NETFLIX‘08, New York, NY, pp. 5:1–5:8. External Links: ISBN 978-1-60558-265-8, Document Cited by: §3.
  • Ò. Celma (2010) Music recommendation and discovery: the long tail, long fail, and long play in the digital music space. Book, Springer, Berlin, Heidelberg, Germany. Cited by: §1.
  • Z. Cheng, J. Shen, L. Nie, T. Chua, and M. Kankanhalli (2017) Exploring user-specific information in music retrieval. In 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR‘17, New York, NY, pp. 655–664. External Links: ISBN 978-1-4503-5022-8, Document Cited by: §2.
  • R. Dias and M. J. Fonseca (2013) Improving music recommendation in session-based collaborative filtering by using temporal context. In

    IEEE 25th International Conference on Tools with Artificial Intelligence

    ICTAI‘13, pp. 783–788. External Links: Document, ISSN 1082-3409 Cited by: §2.
  • G. Häubl and V. Trifts (2000) Consumer decision making in online shopping environments: the effects of interactive decision aids. Marketing Science 19 (1), pp. 4–21. Cited by: §1.
  • P. W. Koh and P. Liang (2017) Understanding black-box predictions via influence functions. In

    34th International Conference on Machine Learning

    ICML‘17, pp. 1885–1894. Cited by: 2nd item.
  • A. Laplante (2014) Improving music recommender systems: what can we learn from research on music tags?. Conference Proceedings In 15th International Society for Music Information Retrieval Conference, ISMIR‘14, pp. 451–456. Cited by: §2, §4.
  • M. Levy and K. Bosteels (2010) Music recommendation and the long tail. In 1st Workshop On Music Recommendation And Discovery, WOMRAD‘10. Cited by: §3.
  • J. M. Mayer, Q. Jones, and S. R. Hiltz (2015) Identifying opportunities for valuable encounters: toward context-aware social matching systems. ACM Transactions on Information Systems 34 (1), pp. 1:1–1:32. External Links: ISSN 1046-8188, Document Cited by: §1.
  • M. Montaner, B. López, and J. L. de la Rosa (2003) A taxonomy of recommender agents on the internet. Artificial Intelligence Review 19 (4), pp. 285–330. External Links: ISSN 0269-2821, Document Cited by: §1.
  • O. Oechslein and T. Hess (2014) The value of a recommendation: the role of social ties in social recommender systems. Conference Proceedings In 47th Hawaii International Conference on System Science, HICSS‘14, pp. 1864–1873. External Links: Document Cited by: §1, §1.
  • G. Oestreicher-Singer and A. Sundararajan (2012) Recommendation networks and the long tail of electronic commerce. MIS Quarterly 36 (1), pp. 65–83. External Links: ISSN 0276-7783 Cited by: §1.
  • F. Ricci, L. Rokach, and B. Shapira (2015) Recommender systems handbook. Edited Book, 2nd edition, Springer, New York, NY. External Links: Document Cited by: §1, §1.
  • T. Schäfer and C. Mehlhorn (2017) Can personality traits predict musical style preferences? a meta-analysis. Personality and Individual Differences 116, pp. 265–273. Cited by: §2.
  • M. Schedl and C. Bauer (2017) Introducing Global and Regional Mainstreaminess for Improving Personalized Music Recommendation. In 15th International Conference on Advances in Mobile Computing & Multimedia, MoMM‘17, New York, NY, pp. 74–81. External Links: ISBN 978-1-4503-5300-7, Document Cited by: §4.
  • M. Schedl and C. Bauer (2017) Online music listening culture of kids and adolescents: listening analysis and music recommendation tailored to the young. Conference Proceedings In 1st International Workshop on Children and Recommender Systems, KidRec‘17, New York, NY. Cited by: §4.
  • M. Schedl, E. Gómez, and J. Urbano (2014) Music information retrieval: recent developments and applications. Foundations and Trends in Information Retrieval 8 (2-3), pp. 127–261. Cited by: §1.
  • M. Schedl, P. Knees, B. McFee, D. Bogdanov, and M. Kaminskas (2015) Music recommender systems. Book Section In Recommender Systems Handbook, F. Ricci, L. Rokach, B. Shapira, and P. B. Kantor (Eds.), pp. 453–492. External Links: Document Cited by: §1.
  • M. Schedl, E. Wiechert, and C. Bauer (Conference Proceedings) The effects of real-world events on music listening behavior: an intervention time series analysis. Conference Proceedings In WWW ’18 Companion: The 2018 Web Conference Companion, WWW‘18, pp. 75–76. External Links: ISBN 978-1-4503-5640-4, Document Cited by: §4.
  • M. Schedl (2016) The LFM-1b Dataset for Music Retrieval and Recommendation. In ACM on International Conference on Multimedia Retrieval, ICMR‘16, New York, NY, pp. 103–110. External Links: ISBN 978-1-4503-4359-6, Document Cited by: §3.
  • M. Schedl (2017) Investigating country-specific music preferences and music recommendation algorithms with the LFM-1b dataset. International Journal of Multimedia Information Retrieval 6 (1), pp. 71–84. External Links: ISSN 2192-662X, Document Cited by: 1st item.
  • C. J. Stevens (2012) Music perception and cognition: a review of recent cross-cultural research. Topics in Cognitive Science 4 (4), pp. 653–667. External Links: Document Cited by: §4.