Watching others play video games via streaming services is an increasingly popular practice. During eSports tournaments, thousands of spectators gather in a virtual space to cheer for their favourite teams and players, watch games, and share emotions with fellow fans.
Massive chats, which involve thousands of users, are fast and “loud”, with a rapid flow of text messages making it difficult to engage in a meaningful conversation (Hamilton et al., 2014) leading researchers to treat some of these message cascades as anti-social (Seering et al., 2017). On the other hand, Ford et al.
underline the meaningful and valuable nature of this “crowd speak“ for establishing massive chat coherence. During key moments of the game, spectators fill the chat with cheering, memes, and emotes resembling a “roaring” sports arena(Hamilton et al., 2014).
In contrast to personal game streams, massive eSports tournament broadcasts reduce the streamer’s figure to a mere technical role and limit spectators’ opportunity to appeal directly to the players.
This limitation dictates a one-way link between what occurs between the screen and the chat suggesting the need for a detailed analysis of the game-event-driven nature of chats in tournament settings, which is less apparent in previous research on different game streaming environments (Hamilton et al., 2014).
In this work, we perform this analysis by applying a combination of topic modelling and time series cross-correlation to chat and game event logs of the largest Dota 2 tournament, The International 7.
Our contribution is twofold. First, we show thematic contents of live streaming chats. We build a topic model of live streaming chat content and show the association between topics’ messages and in-game events.
Second, we introduce the sports bar/club metaphor and propose the consideration of this dual nature of massive streaming chats allowing users to participate in both “roaring” and “talking” by providing a means to manually and automatically switch between these two modes. For example, as at a sports bar, one can sit at a table and have a meaningful conversation with a group of friends while the surrounding crowd is roaring. At any moment, one can join the crowd in cheering and switch back to the conversation with friends.
2. Background: Dota 2, tournaments, and streaming
Dota 2 is a free-to-play multiplayer online battle arena video game in which two teams of five players participate. In addition to being a game, Dota 2 is an eSports discipline featuring tournaments and cups, which attract millions of people to watch.
The gameplay of Dota 2 consists of two phases. First, is the “picks” phase, in which players choose characters to play. Second, is the game itself in which players operate their characters. The most common two objectives in the game are to kill enemy characters and destroy buildings.
Dota 2 tournaments resemble real-world sports competitions. After qualifications, the best teams battle each other through in-game sessions. Usually, teams compete for prizes provided by the event organisers and sponsors. Dota 2 tournaments are streamed via different platforms, including Twitch.tv. Spectators on Twitch can communicate with each other and with a streamer using the built-in chat interface.
During the tournament, the game stream is accompanied by commentators who describe what happens in the game, share their feelings, analyse the situations, and predict the outcome of the game.
A professional production team supports the event by operating cameras in the game (so-called cameraman), providing the venue, and maintaining the broadcast. However, unlike traditional streams, there is no streamer to respond to the chat and communicate with the audience. The analyst desk is a crew of professional eSports commentators and analytic specialists. Unlike commentators, analysts appear on a broadcast during breaks between games to discuss previous games or offer predictions about future games. Sometimes, the analyst desk includes an invited person, which was the case during TI7 where professional analysts and popular players participated.
3. Related Work
Sports spectatorship is an inherently social activity as sharing enriches spectators’ experiences (Rubenking and Lewis, 2016). Live sports co-spectatorship in a public place, e.g. a pub or bar, is a “ritual practice“ performed together with “other like-minded fans.“ (Weed, 2007) (cited in (Guschwan, 2016)). This practice can involve an interaction between viewers and the overall feeling of a crowd, and may even result in the development of a new community. Thus, watching sports tournaments in bars can provide an illusion of a stadium through the presence of the strangers, community reactions to game events, and additional interactions such as betting, making acquaintances, or socialising(Eastman and Land, 1997). This experience can be explained via an intra-audience effect (Cummins and Gong, 2017) where the presence of co-spectators and their vocal response to in-game events supply informational cues to an individual. These cues influence the formation of normative behaviour (Vraga et al., 2014; Cialdini and Goldstein, 2004).
ESports, in turn, are computer-mediated. Thus, there is no physical sports arena for fans to visit, as is even the case of an open tournament, which is organised at a physical location, with a monitor remaining as s a barrier (Hamari and Sjöblom, 2017). Fans gather online in forums, social networks, and streaming platforms to participate in an event and share experiences (Kaytoue et al., 2012).
Hamilton et al. in (Hamilton et al., 2014) analysed streaming chats and found that a “waterfall of text, “ which is typical for massive chats, can disrupt a meaningful conversation within the audience or between the audience and a streamer. Seering et al. in their analysis (Seering et al., 2017) concluded that the practice of copying and pasting messages is anti-social and usually senseless as viewers create “long, often nonsensical messages with many emotes, “ thus flooding the chat. Emotes are often employed in online chats to convey emotions which otherwise could not be expressed briefly and accurately in the text form. Emotes require only a few keystrokes or mouse clicks to write allowing users to send messages rapidly (Zhou et al., 2017).
Both (Hamilton et al., 2014) and (Ford et al., 2017) acknowledged that the speed with which new chat messages appear is a problem. People engaged in communication have difficulty reading and responding to incoming messages, which is characteristic of large and massive stream chats (Hamilton et al., 2014).
These communication “breakdowns“ are also associated with in-game events. During exciting moments, spectators can share their emotions and remind themselves they are part of one group (Hamilton et al., 2014). Recktenwald in (Recktenwald, 2017) refers to these sudden bursts of text as “pivoting“, in which “participants create a local meaning for the game event.“
Ford et al. in (Ford et al., 2017) explore the ways spectators build “crowd speak,“ a “distinctive form of communication“ which supports massive chats with thousands of users. Ford et al. emphasise that these practices make the massive chat coherent. Thus, it is “not experienced as a breakdown, overload, or another difficulty“ but as a meaningful medium.
4. Data and Methods
In this paper, we focus on the analysis of chat messages that are connected to broadcast- and game-related events. Using the Chatty 111chatty.github.io/ application, we parsed the stream channel dota2ti, which was used during The International 7.We gathered more than three million messages from approximately 180 000 viewers. On average for this class of tournament, there can be 22 000 messages submitted by 6 000 unique users per game. We enriched the message-related information with in-game event data, e.g., a destruction of a building or death of a character, which we acquired by parsing the Dotabuff.com website and employing the Open Dota 2 API.
We utilised the probabilistic topic modelling algorithm, Latent Dirichlet Allocation (LDA)(Blei, 2012)
, to analyse the contents of the chat. LDA takes a collection of text documents as input and derives a pre-set number of topics – sets of words, which often co-occur in the same documents. LDA associates each text document to all the topics with different probabilities. We classified a topic as “most popular“ if it had the highest probability (out of all topics) in the given text document. Thus, a chat is characterised by a numeric sequence in which each element represents the dominating topic ID in the given time window.
During preliminary analysis, we found that chat messages in our corpora are often one-word, which is not suitable for a proper LDA analysis. We split chat messages into slots, each one covering a seven-second time window, and joined messages within these windows into larger text documents. The window size was preferred as it allowed us to compile text documents long enough for analysis (mean = 7.99 messages per second, SD = 7, max = 115) and short enough to produce distinctive topics.
We fitted models with a number of topics ranging between 30 and 100. We chose the 100-topic model because the produced topics were quite heterogeneous in terms of thematic content and interpretable enough to be labelled.
We applied cross-correlation analysis to establish the relation between in-game events and prevailing topics in the chat (Brockwell and Davis, 2013). For each topic, we compared its appearance as the most popular topic and the number of in-game events (typically, it was 0 or 1) in the given time frame. We tested resulting time-series with the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test to ensure they are stationary (Kwiatkowski et al., 1992)
, and as a result, for each of 100 topics, we obtained a vector of correlation coefficients between two series for each time lag (Figure 1-5).
Cross-correlation tests are applied if two variables correlate with a lag in some range. Thus, for each topic, we obtained a time pattern of its relation to the in-game events. This time pattern represents the overall presence of the given topic in the chat before (negative lag), during (zero lag), and after (positive lag) the event. Considering in our case that chat message content is unlikely to influence the game, we treat the possible connection as a one-way link, i.e. if an in-game event is followed or preceded by a burst of particular topic messages.
After computing cross-correlation profiles, we find clusters of topics similar in their relation to game event occurrences. We clusterized vectors of cross-correlation coefficients for different topics using Spearman’s correlation between them as a measure of similarity, i.e. two topics with a high correlation of cross-correlation profiles are close in our clustering. To select the number of clusters, we used both visual-guided inspection of the dendrogram and silhouette metrics for cluster analysis, resulting in five clusters. We next analysed topic content and interpreted each topic according to the most common comprising words. We labelled clusters based on topic interpretation.
5. Analysis and results
Previous research suggested that communication in massive Twitch.tv chats is event-driven in nature. To explore the connection between game events and the topic structure of messages, we applied cross-correlation analysis to time series of game events and message counts per topic, resulting in clusters based on different temporal patterns of game event-chat reaction connections. The resulting clusters were labelled as: emotional response, stream content, professional scene, boredom and copypasta, and analyst desk. Based on the clustering results, we calculated the average percentage of messages belonging to each cluster for each game.
Association between these clusters and the topic model captures the differences in a context for similar topics and particular tokens (words, emojis), which is an especially fortunate feature for an emote-based contextually rich communication of streaming chats. For example, topics in the Analyst Desk cluster contain a discussion of themes and information brought to viewers during breaks between games by analyst desk anchors and guests. The stream Content cluster contains topics devoted to the technical details and the game setup of the ongoing stream. The Professional Scene cluster contains topics connected to the professional players, teams, and brands within and outside the context of the current game.
5.1. Emotional response
The Emotional Response cluster consists of topics containing responses to in-game and related events. For example, spectators might comment on the game balance, a character death, building destruction, or cameraman missing an interesting in-game event. Messages on these topics are short and often contain only one word, emote, or abbreviation, e.g. “gg”, “ez”, “pogchamp”, or “kreygasm”. These messages are not always cheering as they sometimes depict mockery toward a player or team due to poor performance, .e.g., “322 LUL.“
Most time patterns for topics in this cluster show these “reactions” are tightly connected with in-game events as topics emerge in the chat soon before the event and disappear shortly after (see Figure 1). The only topic which does not fit this pattern is related to complaints of cameramen failing to move the in-game camera toward an in-game event: “TTours DAMN TTours IT TTours FEELS TTours GOOD TTours TO TTours MISS TTours ALL TTours THE TTours ACTION TTours.“ This type of topic is present before, during, and after an in-game event. We suggest that spectators were aware of the in-game situation and knew when something interesting happened, even if it did not appear on the screen. Thus, they began to complain to cameramen before the actual event and continued afterwards.
5.2. Stream content
Topics in the Stream cluster are related to the current streaming content. Spectators reacted to what or who they saw on screen, including players (e.g. “Time to watch my boy Artour win some Dotes FeelsGoodMan“) and teams (e.g., “Hey Virtus Pro, Vlad Putin here. Your president. Doing great! Just a reminder: There are 5 spots open in a Siberian Prison waiting for you if you don’t win. No pressure! See you guys soon!!“). Spectators also commented on new map terrains, which they did not appreciate222https://compete.kotaku.com/the-international-7-reverts-to-default-dota-2-map-due-t-1797504357: “BlessRNG pls use default terrain BlessRNG,“ “TERRAIN WutFace,“ etc. Eventually, in response to these messages, the terrain was changed to the original.
Messages associated with topics in this cluster also contain references to Dota 2 memes and terms. For instance, the message “1k 2k 3k 4k 5k 6k 7k 8k 9k 10k only the chosen one can hold his true MMR“333 Matchmaking Rating, Dota 2’s players skill level estimate.
Matchmaking Rating, Dota 2’s players skill level estimate.uncovers the mechanism of ranking in Dota 2, which is based on gaining or losing MMR-score depending on a game outcome. Another notable theme in this cluster is the first human-vs-AI game in which one of the most famous professional Dota 2 players, Dendi, lost to the AI-player: “dendi lost against bot hahahahaha“, “OPENAI MMR BOOST“, “MrDestructoid TIME MrDestructoid TO MrDestructoid WATCH MrDestructoid MY MrDestructoid BOY MrDestructoid OPENAI MrDestructoid ENSLAVE MrDestructoid THE MrDestructoid HUMAN MrDestructoid RACE MrDestructoid.“
Regarding time patterns, topics in this cluster present in the chat before and after, but not during in-game events (see Fig. LABEL:fig:sub2). Thus, we can state that, in general, spectators tend to comment stream content during the whole stream except for small time windows when in-game events grab their attention causing other topics to prevail.
5.3. Professional Scene
The Professional cluster contains topics related to Dota 2’s professional scene. Spectators discussed topics such as famous professional teams and players (“liquid gh miracle control“), a confrontation between Chinese and Western teams during the tournament (“budstar liquid go lets china lfy“), or the all-starts match where two teams are comprised of players from different organizations (“worst ti allstars“). Also, it includes topics about past incidents with specific players, “rip monitor biblethump secret sobayed,“ referring to a player from the team, Secret, who broke his monitor in anger444https://www.youtube.com/watch?v=lLjCvWxaONk.
This cluster also included notions of some particular teams. A notable example is Team Liquid, which gained the status of the “last stand” between English-speaking fans. In the last two days of the tournament, Team Liquid fought against three Chinese teams and won the tournament. This situation evoked a burst of team mentions and gave rise to topics on the China-West confrontation.
Before in-game events, topics within this cluster rarely appeared. However, after the event, they appeared more often (see Figure 3). We suggest that after interesting in-game events, spectators started cheering for teams and favourite players or just posted team- and player-related memes in the chat.
5.4. Boredom and Copypasta
The cluster for boredom- and copypasta-related topics contained responses from spectators who conveyed their boredom in various ways. For instance, messages such as “90 min game ResidentSleeper“ or “SLEEP TEST IF YOU TOUCH THE BED , GO TO SLEEP ResidentSleeper“ could be submitted for expressing ones’ tiredness of the long game without interesting recent events.
Spectators sometimes tried to launch a flashmob on a chat for the sake of self-entertainment. For example, they posted a message “
╠═══╣Lets build a ladder
╠═══╣“ to provoke others to copy and paste the message creating a visual “ladder” on the screen, which is considered classic Twitch copypasta 555https://www.twitchquotes.com/copypastas/1452. Keywords for the topics in this cluster are not connected to Dota 2 or personalities (e.g., players) in the stream.
The time pattern for topics in this cluster follows a rise before an interesting in-game event, and start declining half a minute before anything occurs (see Figure 4). We assume that spectators anticipate an interesting and exciting in-game event, so they stop posting boredom-related copypasta. During and after the event, the level of these topics in the chat remains low.
5.5. Analyst desk
Topics in the analyst cluster include names of people participating in the desk (“ccnc blitz machine“) or players eliminated from the tournament (“kkona keepo ba mason[player] [b]ulba[player] dc[team]“). It is not surprising that the time pattern for this cluster does not feature changes associated with in-game events, and is negatively correlated (see Figure 5). However, this cluster holds a significant share of messages in the chat suggesting that spectators still discuss related topics during the game, e.g., during the “picks” stage when players choose the characters for the game.
We demonstrate the overall thematic structure of massive eSport events live streaming chats and show how different topics are evoked and discouraged by in-game events. While hinted by previous works and often considered as common knowledge, to our understanding, this phenomenon has never been supported by statistical evidence until now.
The event-driven and sentiment sharing nature of massive tournaments’ chats suggests a rethinking of its design goals. As Hamilton et al. noted in (Hamilton et al., 2014), sudden bursts of emotes and copypasta disrupt meaningful conversations between spectators, and we demonstrated how these bursts are connected to in-game events. Hamilton et al. suggest dividing spectators into groups by isolating them from each other. Thus, any burst of emotions would be relatively small if they even emerge. However, Ford et al. in (Ford et al., 2017) showed that emotional responses expressed in the chat is a significant part of spectators’ experience.
We suggest that the experience of spectators is mediated by the same intra-audience effect (Cummins and Gong, 2017) that emerges during live spectating and is consistent with a sports arena metaphor (Hamilton et al., 2014; Ford et al., 2017). However, we observe that a metaphor of a sports bar is more accurate since players do not receive any feedback from chat participants during the game. While shouts and chants do not reach the addressee, spectators still find these responses important (Ford et al., 2017). The sports bar is a physical space implying spectators are distributed throughout. Thus, local micro-communities can emerge and sustain meaningful conversations while others roar.
On the other hand, stream chats lack physical dimensions. The idea to introduce physical dimensions to online chats has been explored before, e.g. in (Viégas and Donath, 1999; Donath and Viégas, 2002) and in (Miller et al., 2017), which suggested introducing “conversational circles“ based on the concept of “neighbourhoods.“ However, like (Hamilton et al., 2014), this solution suggests reducing the number of visible chat participants to preserve meaningful communication and does not acknowledge the importance of massive chats’ practices.
Instead of trying to ensure or enforce structured discussion by calling chat participants to order via improved moderation algorithms (Seering et al., 2017) and artificially dividing them into groups(Hamilton et al., 2014), we suggest providing affordances to support both goals of sentiment sharing and intra-audience effects, and an opportunity to have more structured conversations. We suggest introducing a new design element to the chat of sentiment indicators, which summarise current sentiments in a chat in the form of simple graphical elements or counters. Thus, copypasta and chants could be filtered, and the chat preserved in this new form. These indicators might also be used to convey the shared sentiment to players creating the currently missing feedback link from the audience.
“Conversation circles,“ in turn, being introduced as an additional chat channel to the main route, can support structured conversation. These might be pre-populated by a social discovery mechanism suggesting previous game partners and friends while watching the stream at the same moment, as it would naturally happen when walking into a sports bar and seeing familiar faces.
- Blei (2012) David M. Blei. 2012. Probabilistic topic models. Commun. ACM 55, 4 (2012), 77–84.
- Brockwell and Davis (2013) Peter J. Brockwell and Richard A. Davis. 2013. Time series: theory and methods. Springer Science & Business Media. https://www.google.com/books?hl=ru&lr=&id=DJ_lBwAAQBAJ&oi=fnd&pg=PR7&dq=brockwell+2013+time&ots=AcqDfG0Jh-&sig=jYfYsuKVU599_M-i4qpW8MayeB0
- Cialdini and Goldstein (2004) Robert B. Cialdini and Noah J. Goldstein. 2004. Social influence: Compliance and conformity. Annu. Rev. Psychol. 55 (2004), 591–621. http://www.annualreviews.org/doi/abs/10.1146/annurev.psych.55.090902.142015
- Cummins and Gong (2017) R. Glenn Cummins and Zijian Gong. 2017. Mediated intra-audience effects in the appreciation of broadcast sports. Communication & Sport 5, 1 (2017), 27–48. http://journals.sagepub.com/doi/abs/10.1177/2167479515593418
- Donath and Viégas (2002) Judith Donath and Fernanda B. Viégas. 2002. The chat circles series: explorations in designing abstract graphical communication interfaces. In Proceedings of the 4th conference on Designing interactive systems: processes, practices, methods, and techniques. ACM, 359–369. http://dl.acm.org/citation.cfm?id=778764
- Eastman and Land (1997) Susan Tyler Eastman and Arthur M. Land. 1997. The best of both worlds: Sports fans find good seats at the bar. Journal of Sport & Social Issues 21, 2 (1997), 156–178.
- Ford et al. (2017) Colin Ford, Dan Gardner, Leah Elaine Horgan, Calvin Liu, Bonnie Nardi, Jordan Rickman, and others. 2017. Chat Speed OP PogChamp: Practices of Coherence in Massive Twitch Chat. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 858–871. http://dl.acm.org/citation.cfm?id=3052765
- Guschwan (2016) Matthew Guschwan. 2016. Broadcast media: live and in-person. Soccer & Society 17, 3 (2016), 332–350. http://www.tandfonline.com/doi/abs/10.1080/14660970.2015.1082760
- Hamari and Sjöblom (2017) Juho Hamari and Max Sjöblom. 2017. What is eSports and why do people watch it? (2017).
- Hamilton et al. (2014) William A. Hamilton, Oliver Garretson, and Andruid Kerne. 2014. Streaming on twitch: fostering participatory communities of play within live mixed media. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 1315–1324.
- Kaytoue et al. (2012) Mehdi Kaytoue, Arlei Silva, Loïc Cerf, Wagner Meira Jr, and Chedy Raïssi. 2012. Watch me playing, i am a professional: a first study on video game live streaming. In Proceedings of the 21st International Conference on World Wide Web. ACM, 1181–1188.
Kwiatkowski et al. (1992)
Peter CB Phillips, Peter Schmidt, and
Yongcheol Shin. 1992.
Testing the null hypothesis of stationarity against the alternative of a unit root: How sure are we that economic time series have a unit root?Journal of econometrics 54, 1-3 (1992), 159–178. http://www.sciencedirect.com/science/article/pii/030440769290104Y
- Miller et al. (2017) Matthew K. Miller, John C. Tang, Gina Venolia, Gerard Wilkinson, and Kori Inkpen. 2017. Conversational Chat Circles: Being All Here Without Having to Hear It All. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 2394–2404. http://dl.acm.org/citation.cfm?id=3025621
- Recktenwald (2017) Daniel Recktenwald. 2017. Toward a transcription and analysis of live streaming on Twitch. Journal of Pragmatics (2017).
- Rubenking and Lewis (2016) Bridget Rubenking and Nicky Lewis. 2016. The Sweet Spot: An Examination of Second-Screen Sports Viewing. International Journal of Sport Communication 9, 4 (2016), 424–439. http://journals.humankinetics.com/doi/abs/10.1123/IJSC.2016-0080
- Seering et al. (2017) Joseph Seering, Robert Kraut, and Laura Dabbish. 2017. Shaping Pro and Anti-Social Behavior on Twitch Through Moderation and Example-Setting. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 111–125.
- Viégas and Donath (1999) Fernanda B. Viégas and Judith S. Donath. 1999. Chat circles. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 9–16. http://dl.acm.org/citation.cfm?id=302981
- Vraga et al. (2014) Emily K. Vraga, Courtney N. Johnson, D. Jasun Carr, Leticia Bode, and Mitchell T. Bard. 2014. Filmed in front of a live studio audience: Laughter and aggression in political entertainment programming. Journal of Broadcasting & Electronic Media 58, 1 (2014), 131–150. http://www.tandfonline.com/doi/abs/10.1080/08838151.2013.875020
- Weed (2007) Mike Weed. 2007. The pub as a virtual football fandom venue: An alternative to ‘being there’? Soccer & Society 8, 2-3 (2007), 399–414. http://www.tandfonline.com/doi/abs/10.1080/14660970701224665
- Zhou et al. (2017) Rui Zhou, Jasmine Hentschel, and Neha Kumar. 2017. Goodbye Text, Hello Emoji: Mobile Communication on WeChat in China. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 748–759. http://dl.acm.org/citation.cfm?id=3025800