Emotion is often defined as an individual’s mental state associated with thoughts, feelings and behaviour. Stoics like Cicero organized emotions into four categories - metus (fear), aegritudo (pain), libido (lust) and laetitia (pleasure). Later, evolutionary theory of emotion were initiated in the late 19th century by Charles Darwin . He hypothesized that emotions evolved through natural selection and, hence, have cross-culturally universal counterparts. In recent times, Plutchik  categorized emotion into eight primary types, visualized by the wheel of emotions (Fig. 4). Further, Ekman  argued the correlation between emotion and facial expression.
Natural language is often indicative of one’s emotion. Hence, emotion recognition has been enjoying popularity in the field of NLP [4, 5], due to its widespread applications in opinion mining, recommender systems, health-care, and so on. Strapparava and Mihalcea 
addressed the task of emotion detection on news headlines. A number of emotion lexicons[7, 8] have been developed to tackle the textual emotion recognition problem.
Only in the past few years has emotion recognition in conversation (ERC) gained attention from the NLP community [9, 10, 11, 12] due to the increase of public availability of conversational data. ERC can be used to analyze conversations that take place on social media. It can also aid in analyzing conversations in real times, which can be instrumental in legal trials, interviews, e-health services and more.
Unlike vanilla emotion recognition of sentences/utterances, ERC ideally requires context modeling of the individual utterances. This context can be attributed to the preceding utterances, and relies on the temporal sequence of utterances. Compared to the recently published works on ERC [10, 11, 12], both lexicon-based [13, 8, 14]
and modern deep learning-based[4, 5] vanilla emotion recognition approaches fail to work well on ERC datasets as these works ignore the conversation specific factors such as the presence of contextual cues, the temporality in speakers’ turns, or speaker-specific information. Fig. 4(a) and Fig. 4(b) show an example where the same utterance changes its meaning depending on its preceding utterance.
Task definition — Given the transcript of a conversation along with speaker information of each constituent utterance, the ERC task aims to identify the emotion of each utterance from several pre-defined emotions. Fig. 2 illustrates one such conversation between two people, where each utterance is labeled by the underlying emotion. Formally, given the input sequence of number of utterances , where each utterance consists of words and spoken by party , the task is to predict the emotion label of each utterance .
Controlling variables in conversations — Conversations are broadly categorized into two categories: task oriented and chit-chat (also called as non-task oriented). Both kinds of conversation are governed by different factors or pragmatics , such as topic, interlocutors’ personality, argumentation logic, viewpoint, intent , and so on. Fig. 1 depicts how these factors play out in a dyadic conversation. Firstly, topic () and interlocutor personality () always influence the conversation, irrespective of the time. A speaker makes up his/her mind () about the reply () based on the contextual preceding utterances () from both speaker and listener, the previous utterance being the most important one since it usually makes the largest change in the joint task model (for task-oriented conversations) or the speaker’s emotional state (for chit-chat). Delving deeper, the pragmatic features, as explained by Hovy , like argumentation logic, interlocutor viewpoint, inter-personal relationship and dependency, situational awareness are encoded in speaker state (). Intent () of the speaker is decided based on previous intent and speaker state , as the interlocutor may change his/her intent based on the opponent’s utterance and current situation. Then, the speaker formulates appropriate emotion for the response based on the state and intent . Finally, the response is produced based on the speaker state , intent and emotion . We surmise that considering these factors would help representing the argument and discourse structure of the conversation, which leads to improved conversation understanding, including emotion recognition.
Early computational work on dialogue focused mostly on task-oriented cases, in which the overall conversational intent and step-by-step sub-goals played a large part [17, 18]. Cohen and Levesque  developed a model and logic to represent intentions and their connections to utterances, whose operators explicate the treatment of beliefs about the interlocutor’s beliefs and vice versa, recursively. Emotion however played no role in this line of research. In more recent work, chatbots and chit-chat dialogue have become more prominent, in part due to the use of distributed (such as embedding) representations that do not readily support logical inference.
On conversational setting, K. D’Mello et al.  and Yang et al.  worked with small datasets with three and four emotion labels, respectively. This was followed by Phan et al. , where emotion detection on conversation transcript was attempted. Recently, several works [23, 24]
have devised deep learning-based techniques for ERC. These works are crucial as we surmise an instrumental role of ERC in emotion-aware a.k.a. affective dialogue generation which has fallen within the topic of “text generation under pragmatics constriants” as proposed byHovy . Fig. 3 illustrates one such conversation between a human (user) and a medical chatbot (health-assistant). The assistant responds with emotion based on the user’s input. Depending on whether the user suffered an injury earlier or not, the health-assistant responds with excitement (evoking urgency) or happiness (evoking relief).
As ERC is a new research field, outlining research challenges, available datasets, and benchmarks can potentially aid future research on ERC. In this paper, we aim to serve this purpose by discussing various factors that contribute to the emotion dynamics in a conversation. We surmise that this paper will not only help the researchers to better understand the challenges and recent works on ERC but also show possible future research directions. The rest of the paper is organized as follows: Section II presents the key research challenges; Section III and IV cover the datasets and recent progress in this field; finally Section V concludes the paper.
Ii Research Challenges
Categorization of emotions
Emotion is defined using two type of models — categorical and dimensional. Categorical model classifies emotion into a fixed number of discrete categories. In contrast, dimensional model describes emotion as a point in a continuous multi-dimensional space.
In the categorical front, Plutchik ’s wheel of emotions (Fig. 4) defines eight discrete primary emotion types, each of which has finer related subtypes. On the other hand, Ekman  concludes six basic emotions — anger, disgust, fear, happiness, sadness and surprise.
adopt two dimensions — valence and arousal. Valence represents the degree of emotional positivity, and arousal represents the intensity of the emotion. In contrast with the categorical models, dimensional models map emotion into a continuous spectrum rather than hard categories. This enables easy and intuitive comparison of two emotional states using vector operations, whereas comparison is non-trivial for categorical models. As there are multiple categorization and dimensional taxonomies available, it is challenging to select one particular model for annotation. Choosing a simple categorization model e.g., Ekman’s model has a major drawback as these models are unable to ground complex emotions. On the other hand, complex emotion models such as Plutchik’s model make it very difficult for the annotators to discriminate between the related emotions, e.g., discerning anger from rage. Complex emotion models also increase the risk of obtaining a lower inter-annotator agreement.
The popular ERC dataset IEMOCAP  adopted both categorical and dimensional models. However, newer ERC datasets like DailyDialogue  have employed only categorical model due to its more intuitive nature. Most of the available datasets for emotion recognition in conversation adopted simple taxonomies, which are slight variants of Ekman’s model. Each emotional utterance in the EmoContext dataset is labeled with one of the following emotions: happiness, sadness and anger. The majority of the utterances in EmoContext do not elicit any of these three emotions and are annotated with an extra label: others. Naturally, the inter-annotator agreement for the EmoContext dataset is higher due to its simplistic emotion taxonomy. However, the short context length and simple emotion taxonomy make ERC on this dataset less challenging.
Basis of emotion annotation
Annotation with emotion labels is challenging as the label depends on the annotators perspective. Self-assessment by the interlocutors in a conversation is arguably the best way to annotate utterances. However, in practice it is unfeasible as real-time tagging of unscripted conversations will impact the conversation flow. Post-conversation self-annotation could be an option, but it has not been done yet.
As such, many ERC datasets  are scripted and annotated by a group of people uninvolved with the script and conversation. The annotators are given the context of the utterances as prior knowledge for accurate annotation. Often pre-existing transcripts are annotated for quick turn-around, as in EmotionLines .
The annotators also need to be aware of the interlocutors perspective for situation-aware annotation. For example, the emotion behind the utterance “Lehman Brothers’ stock is plummeting!!” depends on whether the speaker benefits from the crash. The annotators should be aware of the nature of association between the speaker and Lehman Brothers for accurate labeling.
Conversational context modeling
Context is at the core of the NLP research. According to several recent studies [29, 30], contextual sentence and word embeddings can improve the performance of the state-of-the-art NLP systems by a significant margin.
The notion of context can vary from problem to problem. For example, while calculating word representations, the surrounding words carry contextual information. Likewise, to classify a sentence in a document, other neighboring sentences are considered as its context. In Poria et al. , surrounding utterances are treated as context and they experimentally show that contextual evidence indeed aids in classification.
Similarly in conversational emotion-detection, to determine the emotion of an utterance at time , the preceding utterances at time can be considered as its context. However, computing this context representation often exhibits major difficulties due to emotional dynamics.
Emotional dynamics of conversations consists of two important aspects: self and inter-personal dependencies . self-dependency, also known as emotional inertia, deals with the aspect of emotional influence that speakers have on themselves during conversations . On the other hand, inter-personal dependencies relate to the emotional influences that the counterparts induce into a speaker. Conversely, during the course of a dialogue, speakers also tend to mirror their counterparts to build rapport . This phenomenon is illustrated in Fig. 2. Here, is frustrated over her long term unemployment and seeks encouragement (). , however, is pre-occupied and replies sarcastically (). This enrages to appropriate an angry response (). In this dialogue, emotional inertia is evident in who does not deviate from his nonchalant behavior. , however, gets emotionally influenced by . Modeling self and inter-personal relationship and dependencies may also depend on the topic of the conversation as well as various other factors like argument structure, interlocutors’ personality, intents, viewpoints on the conversation, attitude towards each other etc.. Hence, analyzing all these factors are key for a true self and inter-personal dependency modeling that can lead to enriched context understanding.
The contextual information can come from both local and distant conversational history. While the importance of local context is more obvious, as stated in recent works, distant context often plays a less important role in ERC. Distant contextual information is useful mostly in the scenarios when a speaker refers to earlier utterances spoken by any of the speakers in the conversational history.
The usefulness of context is more prevalent in classifying short utterances, like “yeah”, “okay”, “no”, that can express different emotions depending on the context and discourse of the dialogue. The examples in Fig. 4(a) and Fig. 4(b) explain this phenomenon. The emotions expressed by the same utterance “Yeah” in both these examples differ from each other and can only be inferred from the context.
Finding contextualized conversational utterance representations is an active area of research. Leveraging such contextual clues is a difficult task. Memory networks, RNNs, and attention mechanisms have been used in previous works e.g., HRLCE or DialogueRNN, to grasp information from the context.
Speaker specific modeling
Individuals have their own subtle way of expressing emotions. For instance, some individuals are more sarcastic than others. For such cases, the usage of certain words would vary depending on if they are being sarcastic. Let’s consider this example, “The order has been cancelled.”, “This is great!”. If is a sarcastic person, then his response would express negative emotion to the order being canceled through the word great. On the other hand, ’s response, great, could be taken literally if the canceled order is beneficial to (perhaps cannot afford the product he ordered). Since, necessary background information is often missing from the conversations, speaker profiling based on preceding utterances often yields improved results.
Listener specific modeling
During a conversation, the listeners make up their mind about the speaker’s utterance as it’s spoken. However, there is no textual data on the listener’s reaction to the speaker. A model must resort to visual modality to model the listener’s facial expression to capture the listener’s reaction. However, according to DialogueRNN, capturing listener reaction does not yield any improvement as the listener’s subsequent utterance carries their reaction. Moreover, of the listener never speaks in a conversation, his/her reaction remains irrelevant. Nonetheless, listener modeling can be useful in the scenarios where continuous emotion recognition of every moment of the conversation is necessary, like audience reaction during a political speech, as opposed to emotion recognition of each utterance.
Presence of emotion shift
Due to emotional inertia, participants in a conversation tend to stick a particular emotional state, unless some external stimuli, usually the other participants, invoke a change. This is illustrated in Fig. 6, where Joey changes his emotion from neutral to anger due to the last utterance of Chandler, which was unexpected and rather shocking to Joey. This is a hard problem to solve, as the state-of-the-art ERC model, DialogueRNN is more accurate in emotion detection for the utterances without emotional shift or when the shift is to a similar emotion (e.g., from fear to sad). It is equally interesting and challenging to track the development, shift and dynamics of the participants’ opinions on the topic of discussion in the conversation.
Fine-grained emotion recognition
Fine-grained emotion recognition aims at recognizing emotion expressed on the explicit and implicit topics. It involves a deeper understanding of the topic of the conversation, interlocutor opinion and stand. For example, in Fig. 7, while both persons take a supportive stand for the government’s bill, they use completely opposite emotions to express it. It is not possible for a vanilla emotion recognizer to understand the positive emotion of both the interlocutors on the aspect of government’s bill. Only by interpreting Person 2’s frustration about the opposition’s protest against the bill can a classifier infer Person 2’s support for the bill. On the other hand, even though Person 1 does not explicitly express his/her opinion on the opposition, from the discourse of the conversation, it can be inferred that Person 1 holds a negative opinion on the opposition.
In a multiparty conversation, more than two participants are involved. Naturally, emotion recognition in such conversations is more challenging in comparison with dyadic conversations due to the difficulty in tracking individual speaker states and handling co-references.
Presence of sarcasm
Sarcasm is a linguistic tool that uses irony to express contempt. An ERC system incapable of detecting sarcasm mostly fails to predict emotion of the sarcastic utterances correctly. Sarcasm detection in a conversation largely depends on the context and discourse of the conversation. For example, the utterance “The part where Obama signed it” can only be detected as sarcastic if we look at the previous utterance “What part of this would be unconstitutional?”. Sarcastic nature is also person dependent, which again warrants speaker profiling in the conversation.
The ability to reason is necessary for any explainable AI system. In the context of ERC, it is often desired to understand the cause of an expressed emotion by a speaker. As an example, we can refer to Fig. 2. An ideal ERC system, with the ability of emotion reasoning, should perceive the reason for ’s anger, expressed in of Fig. 2. It is evident upon observation that this anger is caused by the persistent nonchalant behavior of . Readers should not conflate emotion reasoning with context modeling, which we discuss earlier in this section. Unlike context modeling, emotion reasoning does not only find the contextual utterances in conversational history that triggers the emotion of an utterance, but also determines the function of those contextual utterances on the target utterance. In Fig. 2, it is the indifference of , reflected by and , that makes angry. Similarly, in Fig. 6, Joey expresses anger once he ascertains Chandler’s deception in the previous utterance. It is hard to define a taxonomy or tagset for emotion reasoning. At present, there is no available dataset which contains such rich annotations. Building such dataset would enable future dialogue systems to frame meaningful argumentation logic and discourse structure, taking one step closer to human-like conversation.
In the last few years, emotion recognition in conversation has gained major research interest, mainly because of its potential application in dialogue systems to generate emotion-aware and empathetic dialogues . The primary goal of ERC task is to label each utterance in the conversation with an emotion label. In this section, we discuss the publicly available ERC datasets as well as the shortcomings of these datasets.
There are a few publicly available datasets for ERC - IEMOCAP , SEMAINE , Emotionlines , MELD , DailyDialog  and EmoContext . A detailed comparison of these datasets is drawn in Table II. Out of these five datasets, IEMOCAP, SEMAINE and MELD are multimodal (containing acoustic, visual and textual information) and the remaining two are textual. Apart from SEMAINE dataset, rest of the datasets contains categorical emotion labels. In contrast, each utterance of SEMAINE dataset is annotated with four real valued affective attributes: valence (), arousal (), expectancy () and power (). We also show the emotion label distribution of these datasets in Table I. In EmoContext dataset, an emotion label is assigned to only the last utterance of each dialogue. None of these datasets can be used for emotion reasoning as they lack necessary annotation details required for the reasoning task. Readers should also note that, all these datasets do not contain fine-grained and topic level emotion annotation.
Iv Recent advances
In this section we give a brief introduction to the recent work on this topic. We also compare the approaches and report their drawbacks.
As depicted in Fig. 1, recognizing emotion of an utterance in a conversation primarily depends on these following three factors:
the utterance itself and its context defined by the interlocutors’ preceding utterances in the conversation, as well as intent and the topic of the conversation,
the speaker’s state comprising variables like personality and argumentation logic and,
emotions expressed in the preceding utterances.
Although, IEMOCAP and SEMAINE have been developed almost a decade ago, most of the works that used these two datasets did not consider the aforementioned factors.
Benchmarks and their drawbacks
Based on these factors, a number of approaches to address the ERC problem have been proposed recently. Conversational memory network (CMN), proposed by Hazarika et al.  for dyadic dialogues, is one of the first ERC approaches that utilizes distinct memories for each speaker for speaker-specific context modeling. Later, Hazarika et al.  improved upon this approach with interactive conversational memory network (ICON), which interconnects these memories to model self and inter-speaker emotional influence. None of these two methods actually exploit the speaker information of the target utterance for classification. This makes the model blind to speaker-specific nuances. Recently, Yeh et al.  proposed an ERC method called Interaction-aware Attention Network (IANN) by leveraging inter-speaker relation modeling. Similar to ICON and CMN, IANN utilises distinct memories for each speaker.
DialogueRNN  aims to solve this issue by considering the speaker information of the target utterance and, further, modeling self and inter-speaker emotional influence with a hierarchical multi-stage RNN with attention mechanism. On both IEMOCAP and SEMAINE datasets, DialogueRNN outperformed (Table III and Table IV) the other two approaches.
as in our experiment, we disregard their contextual feature extraction and pre-processing part. and More details can be found inMajumder et al. .
All of these models affirm that contextual history, modeling self and inter-speaker influence are beneficial to ERC (shown in Fig. 8 and Fig. 9). Further, DialogueRNN shows that the nearby utterances are generally more context rich and ERC performance improves when the future utterances, at time , are available. This is indicated by Fig. 9, where DialogueRNN uses both past and future utterances as context with roughly the same frequency. Also, the distant utterances are used less frequently than the nearby utterances. On the other hand, CMN and ICON do not use future utterances as context at all. However, for real-time applications, systems cannot rely on future utterances. In such cases, CMN, ICON and DialogueRNN with fixed context window would be befitting.
All these networks, namely CMN, ICON, IANN and DialogueRNN, perform poorly on the utterances with emotion shift. In particular, the cases where the emotion of the target utterance differs from the previous utterance, DialogueRNN could only correctly predict instances. This stands less as compared to the success-rate that it achieves at the regions of no emotional-shift.
Among these three approaches, only DialogueRNN is capable of handling multiparty conversations on large scale. However, on the multiparty conversational dataset MELD, only a little performance improvement (shown in Table V) is observed by DialogueRNN compared to bc-LSTM which depicts a future research direction on multiparty ERC. ICON and CMN are designed to detect emotions in dyadic dialogues. Adapting ICON and CMN to apply on multiparty conversational dataset MELD can cause scalability issue in situations when number speakers participating in a conversation in the test data is more than the training data.
Test-set F-score results of bc-LSTM and DialogueRNN for emotion classification in MELD. Note:w-avg denotes weighted-average. text-CNN: CNN applied on text, contextual information were not used.
Due to the sequential nature of the utterances in conversations, RNNs are used for context generation in the aforementioned models. However, there is ample room for improvement, as the RNN-based context representation methods perform poorly in grasping long distant contextual information.
Recently, two shared tasks - EmotionX (co-located with SocialNLP workshop) and EmoContext 111https://www.humanizing-ai.com/emocontext.html (co-located with Semeval 2019) have been organized to address the ERC problem. EmoContext shared task has garnered more than 500 participants, affirming the growing popularity of this research field. Compared to other datasets, EmoContext dataset  has very short conversations consisting only three utterances where the goal is to label the 3rd utterance as shown in Fig. 10.
Emotion labels of the previous utterances are not present in the EmoContext dataset. The key works [24, 39, 37] on this dataset have mainly leveraged on context modeling using bc-LSTM architecture  that encapsulates the temporal order of the utterances using an LSTM. A common trend can be noticed in these works, where traditional word embeddings, such as Glove , are combined with contextualized word embeddings, such as ELMo  to improve the performance.
In Fig. 11, we depict the HRLCE framework, proposed by Huang et al. , that comprises of an utterance encoder and a context encoder that takes input from the utterance encoder. To represent each utterance, HRLCE utilizes ELMo , Glove  and Deepmoji .
The context encoder in HRLCE adapts the bc-LSTM framework followed by a multi-head attention layer. Huang et al.  applied HRLCE framework only on the EmoContext dataset. However, HRLCE can be easily adapted to apply on other ERC datasets. It should be noted that none of the works on the EmoContext dataset utilize speaker information. In fact, in our experiments, we found that DialogueRNN, which makes use of the speaker information, performs similar (Table VI) to Bae et al. , Huang et al.  and Chatterjee et al.  on EmoContext dataset. One possible reason for this could be the presence of very short context history in the dataset that renders speaker information inconsequential.
Emotion recognition in conversation has been gaining popularity among NLP researchers. In this work, we summarize the recent advances in ERC and highlight several key research challenges associated with this research area. Further, we point out how current work has partly addressed these challenges, while also presenting some shortcomings. Overall, we surmise that an effective emotion-shift recognition model and context encoder can yield significant performance improvement over chit-chat dialogue, and even improve some aspects of task-oriented dialogue. Moreover, challenges like topic-level speaker-specific emotion recognition, ERC on multiparty conversations, and conversational sarcasm detection can form new research directions. Additionally, fine-grained speaker-specific continuous emotion recognition may become of interest for the purpose of tracking emotions during long monologue. We believe that addressing each of the challenges outlined in this paper will not only enhance AI-enabled conversation understanding, but also improve the performance of dialogue systems by catering to affective information.
- Darwin and Prodger  C. Darwin and P. Prodger, The expression of the emotions in man and animals. Oxford University Press, USA, 1998.
- Plutchik  R. Plutchik, “A psychoevolutionary theory of emotions,” Social Science Information, vol. 21, no. 4-5, pp. 529–553, 1982.
- Ekman  P. Ekman, “Facial expression and emotion.” American psychologist, vol. 48, no. 4, p. 384, 1993.
- Kratzwald et al.  B. Kratzwald, S. Ilic, M. Kraus, S. Feuerriegel, and H. Prendinger, “Decision support with text-based emotion recognition: Deep learning for affective computing,” arXiv preprint arXiv:1803.06397, 2018.
- Colneriĉ and Demsar  N. Colneriĉ and J. Demsar, “Emotion recognition on twitter: comparative study and training a unison model,” IEEE Transactions on Affective Computing, 2018.
- Strapparava and Mihalcea  C. Strapparava and R. Mihalcea, “Semeval-2007 task 14: Affective text,” in Proceedings of the 4th International Workshop on Semantic Evaluations, ser. SemEval ’07. Stroudsburg, PA, USA: Association for Computational Linguistics, 2007, pp. 70–74. [Online]. Available: http://dl.acm.org/citation.cfm?id=1621474.1621487
- Strapparava et al.  C. Strapparava, A. Valitutti et al., “Wordnet affect: an affective extension of wordnet.” in Lrec, vol. 4, no. 1083-1086. Citeseer, 2004, p. 40.
- Mohammad and Turney  S. M. Mohammad and P. D. Turney, “Emotions evoked by common words and phrases: Using mechanical turk to create an emotion lexicon,” in Proceedings of the NAACL HLT 2010 workshop on computational approaches to analysis and generation of emotion in text. Association for Computational Linguistics, 2010, pp. 26–34.
- Yeh et al.  S.-L. Yeh, Y.-S. Lin, and C.-C. Lee, “An interaction-aware attention network for speech emotion recognition in spoken dialogs,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 6685–6689.
- Chen et al.  S.-Y. Chen, C.-C. Hsu, C.-C. Kuo, L.-W. Ku et al., “Emotionlines: An emotion corpus of multi-party conversations,” arXiv preprint arXiv:1802.08379, 2018.
- Majumder et al.  N. Majumder, S. Poria, D. Hazarika, R. Mihalcea, A. Gelbukh, and E. Cambria, “Dialoguernn: An attentive rnn for emotion detection in conversations,” arXiv preprint arXiv:1811.00405, 2018.
- Zhou et al.  H. Zhou, M. Huang, T. Zhang, X. Zhu, and B. Liu, “Emotional chatting machine: Emotional conversation generation with internal and external memory,” in Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
- Wu et al.  C.-H. Wu, Z.-J. Chuang, and Y.-C. Lin, “Emotion recognition from text using semantic labels and separable mixture models,” ACM transactions on Asian language information processing (TALIP), vol. 5, no. 2, pp. 165–183, 2006.
- Shaheen et al.  S. Shaheen, W. El-Hajj, H. Hajj, and S. Elbassuoni, “Emotion recognition from text based on automatically generated rules,” in 2014 IEEE International Conference on Data Mining Workshop. IEEE, 2014, pp. 383–392.
- Hovy  E. Hovy, “Generating natural language under pragmatic constraints,” Journal of Pragmatics, vol. 11, no. 6, pp. 689–719, 1987.
- Schlöder and Fernández  J. J. Schlöder and R. Fernández, “Clarifying intentions in dialogue: A corpus study,” in Proceedings of the 11th International Conference on Computational Semantics, 2015, pp. 46–51.
- Grosz and Sidner  B. J. Grosz and C. L. Sidner, “Attention, intentions, and the structure of discourse,” Computational linguistics, vol. 12, no. 3, pp. 175–204, 1986.
- Appelt and Appelt  D. E. Appelt and D. E. Appelt, Planning english sentences. Cambridge University Press, 1992.
- Cohen and Levesque  P. R. Cohen and H. J. Levesque, “Speech acts and rationality,” in Proceedings of the 23rd annual meeting on Association for Computational Linguistics. Association for Computational Linguistics, 1985, pp. 49–60.
- K. D’Mello et al.  S. K. D’Mello, S. Craig, J. Sullins, and A. Graesser, “Predicting affective states expressed through an emote-aloud procedure from autotutor’s mixed-initiative dialogue.” I. J. Artificial Intelligence in Education, vol. 16, pp. 3–28, 01 2006.
- Yang et al.  C. Yang, K. H. Lin, and H. Chen, “Emotion classification using web blog corpora,” in IEEE/WIC/ACM International Conference on Web Intelligence (WI’07), Nov 2007, pp. 275–278.
- Phan et al.  D. A. Phan, H. Shindo, and Y. Matsumoto, “Multiple emotions detection in conversation transcripts,” in Proceedings of the 30th Pacific Asia Conference on Language, Information and Computation, Seoul, South Korea, Oct. 2016, pp. 85–94. [Online]. Available: https://www.aclweb.org/anthology/Y16-2006
- Hazarika et al. [2018a] D. Hazarika, S. Poria, R. Mihalcea, E. Cambria, and R. Zimmermann, “Icon: Interactive conversational memory network for multimodal emotion detection,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 2594–2604.
- Bae et al.  S. Bae, J. Choi, and S.-g. Lee, “Snu_ids at semeval-2019 task 3: Addressing training-test class distribution mismatch in conversational classification,” arXiv preprint arXiv:1903.02163, 2019.
- Russell  J. Russell, “A circumplex model of affect,” Journal of Personality and Social Psychology, vol. 39, pp. 1161–1178, 12 1980.
- Mehrabian  A. Mehrabian, “Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament,” Current Psychology, vol. 14, no. 4, pp. 261–292, Dec 1996. [Online]. Available: https://doi.org/10.1007/BF02686918
- Busso et al.  C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, “Iemocap: Interactive emotional dyadic motion capture database,” Language resources and evaluation, vol. 42, no. 4, p. 335, 2008.
- Li et al.  Y. Li, H. Su, X. Shen, W. Li, Z. Cao, and S. Niu, “Dailydialog: A manually labelled multi-turn dialogue dataset,” arXiv preprint arXiv:1710.03957, 2017.
- Peters et al.  M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, “Deep contextualized word representations,” arXiv preprint arXiv:1802.05365, 2018.
- Devlin et al.  J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
Poria et al. 
S. Poria, E. Cambria, D. Hazarika, N. Majumder, A. Zadeh, and L.-P. Morency, “Context-dependent sentiment analysis in user-generated videos,” inACL 2017, vol. 1, 2017, pp. 873–883.
- Morris and Keltner  M. W. Morris and D. Keltner, “How emotions work: The social functions of emotional expression in negotiations,” Research in organizational behavior, vol. 22, pp. 1–50, 2000.
- Kuppens et al.  P. Kuppens, N. B. Allen, and L. B. Sheeber, “Emotional inertia and psychological maladjustment,” Psychological Science, vol. 21, no. 7, pp. 984–991, 2010.
- Navarretta et al.  C. Navarretta, K. Choukri, T. Declerck, S. Goggi, M. Grobelnik, and B. Maegaard, “Mirroring facial expressions and emotions in dyadic conversations.” in LREC, 2016.
- McKeown et al.  G. McKeown, M. Valstar, R. Cowie, M. Pantic, and M. Schroder, “The semaine database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent,” IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 5–17, 2012.
- Poria et al.  S. Poria, D. Hazarika, N. Majumder, G. Naik, E. Cambria, and R. Mihalcea, “Meld: A multimodal multi-party dataset for emotion recognition in conversations,” arXiv preprint arXiv:1810.02508, 2018.
- Chatterjee et al.  A. Chatterjee, U. Gupta, M. K. Chinnakotla, R. Srikanth, M. Galley, and P. Agrawal, “Understanding emotions in text using deep learning and big data,” Computers in Human Behavior, vol. 93, pp. 309–317, 2019.
- Hazarika et al. [2018b] D. Hazarika, S. Poria, A. Zadeh, E. Cambria, L.-P. Morency, and R. Zimmermann, “Conversational memory network for emotion recognition in dyadic dialogue videos,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), vol. 1, 2018, pp. 2122–2132.
- Huang et al.  C. Huang, A. Trabelsi, and O. R. Zaïane, “Ana at semeval-2019 task 3: Contextual emotion detection in conversations through hierarchical lstms and bert,” arXiv preprint arXiv:1904.00132, 2019.
- Pennington et al.  J. Pennington, R. Socher, and C. Manning, “Glove: Global vectors for word representation,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1532–1543.
- Felbo et al.  B. Felbo, A. Mislove, A. Søgaard, I. Rahwan, and S. Lehmann, “Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm,” arXiv preprint arXiv:1708.00524, 2017.