Using Hadoop for Large Scale Analysis on Twitter: A Technical Report

02/03/2016 ∙ by Nikolaos Nodarakis, et al. ∙ Ionian University Computer Technology Institute 0

Sentiment analysis (or opinion mining) on Twitter data has attracted much attention recently. One of the system's key features, is the immediacy in communication with other users in an easy, user-friendly and fast way. Consequently, people tend to express their feelings freely, which makes Twitter an ideal source for accumulating a vast amount of opinions towards a wide diversity of topics. This amount of information offers huge potential and can be harnessed to receive the sentiment tendency towards these topics. However, since none can invest an infinite amount of time to read through these tweets, an automated decision making approach is necessary. Nevertheless, most existing solutions are limited in centralized environments only. Thus, they can only process at most a few thousand tweets. Such a sample, is not representative to define the sentiment polarity towards a topic due to the massive number of tweets published daily. In this paper, we go one step further and develop a novel method for sentiment learning in the MapReduce framework. Our algorithm exploits the hashtags and emoticons inside a tweet, as sentiment labels, and proceeds to a classification procedure of diverse sentiment types in a parallel and distributed manner. Moreover, we utilize Bloom filters to compact the storage size of intermediate data and boost the performance of our algorithm. Through an extensive experimental evaluation, we prove that our solution is efficient, robust and scalable and confirm the quality of our sentiment identification.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Twitter is one of the most popular social network websites and launched in 2006. Since then, it has grown at a very fast pace and at the time speaking numbers 316 million monthly active users, while 500 millions tweets are sent on a daily basis111https://about.twitter.com/company (Visited 19/9/2015). Naturally, it is a wide spreading instant messaging platform and people use it to get informed about world news, videos that have become viral, discussions over recently released products or technological advancements, etc. Inevitably, a cluster of different opinions, that carry rich sentiment information and concern a variety of entities or topics, is formed. Sentiment is defined as "A thought, view, or attitude, especially one based mainly on emotion instead of reason"222http://www.thefreedictionary.com/sentiment and describes someone’s mood or judge towards a specific entity. User-generated content that captures sentiment information has proved to be valuable and its use is widespread among many internet applications and information systems, such as search engines.

Knowing the overall sentiment inclination towards a topic, provides very useful information and can be captivating in certain cases. For instance, Google would like to know what their users think about the latest Android 5.0 update, in order to proceed to further development and bug fixing until the operating system works smoothly and meets the needs of the users. Thus, it is clear that a concise sentiment analysis towards the topic during a time period is needed. Two of the most known websites that perform sentiment analysis on Twitter are Topsy333http://topsy.com/ and Sentiment140444http://www.sentiment140.com/.

In the context of this work, we utilize hashtags and emoticons as sentiment labels to perform classification of diverse sentiment types. Hashtags are a convention for adding additional context and metadata to tweets. They are created by users as a way to categorize their message and/or highlight a topic and are extensively utilized in tweets [18]. Moreover, they provide the ability to people to search tweets that refer to a common subject. The creation of a hashtag is achieved by prefixing a word with a hash symbol (e.g. #love). Emoticon refers to a digital icon or a sequence of keyboard symbols that serves to represent a facial expression, as :-) for a smiling face555http://dictionary.reference.com/browse/emoticon. Both, hashtags and emoticons, provide a fine-grained sentiment learning at tweet level which makes them suitable to be leveraged for opinion mining.

Although the problem of sentiment analysis has been studied extensively during recent years, existing solutions suffer from certain limitations. One problem is that the majority of approaches is bounded in centralized environments. Moreover, sentiment analysis is based on, it terms of methodology, natural language processing techniques and machine learning approaches. However, this kind of techniques are time-consuming and spare many computational resources. Consequently, at most a few thousand records can be processed by such techniques without exceeding the capabilities of a single server. Since millions of tweets are published daily on Twitter, it is more than clear that underline solutions are not sufficient. Consequently, high scalable implementations are required in order to acquire a much better overview of sentiment tendency towards a topic. Cloud computing technologies provide tools and infrastructure to create such solutions and manage the input data in a distributed way among multiple servers. The most popular and notably efficient tool is the MapReduce

[7] programming model, developed by Google, for processing large-scale data.

In this paper, we propose MR-SAT: a novel MapReduce Algorithm for Big Data Sentiment Analysis on Twitter implemented in Hadoop [17, 19]

, the open source MapReduce implementation. Our algorithm exploits the hashtags and emoticons inside a tweet, as sentiment labels, in order to avoid the time-intensive manual annotation task. After that, we build the feature vectors of training and test set and proceed to a classification procedure in a fully distributed manner. Additionally, we encode features using Bloom filters to compress the storage space of the feature vectors. We adapt an existing MapReduce classification algorithm based on A

NN queries to achieve the desirable outcome. Through an extensive experimental evaluation we study various parameters that can affect the total computation cost and classification performance. We prove that our solution is efficient, robust and scalable and confirm the quality of our sentiment identification.

The rest of the paper is organized as follows: in Section 2 we discuss related work and in Section 3 we present how our algorithm works. More specifically, we explain how to build the feature vectors (for both the training and test dataset), we briefly describe the Bloom filter integration and display our ANN based classification algorithm. After that, we proceed to the experimental evaluation of our approach in Section 4, while in Section 5 we conclude the paper and present future steps.

2 Related Work

The domain of sentiment analysis, or opinion mining, has been studied extensively in literature during decent years. Early studies focus on document level sentiment analysis concerning movie or product reviews [9, 25] and posts published on webpages or blogs [24]. Respectively, many efforts have been made towards the sentence level sentiment analysis [20, 21, 23] which examines phrases and assigns to each one of them a sentiment polarity (positive, negative, neutral). A less investigated area is the topic-based sentiment analysis [12, 13] due to the difficulty to provide an adequate definition of topic and how to incorporate the sentiment factor into the opinion mining task.

Many researchers confront the problem of sentiment analysis by applying machine learning approaches and/or natural language processing techniques. In [16]

, the authors employ three machine learning techniques to classify movie reviews as positive or negative. On the other hand, Nasukawa and Yi

[14]

investigate the proper identification of semantic relationships between the sentiment expressions and the subject, in order to enhance the accuracy of sentiment analysis within webpages and online articles. Their approach utilizes a syntactic parser and a sentiment lexicon. Moreover, Ding and Liu

[8] propose a set of linguistic rules together with a new opinion aggregation function to detect sentiment orientations in online product reviews.

Nowadays, Twitter has received much attention for sentiment analysis, as it provides a source of massive user-generated content that captures a wide aspect of published opinions. In [2], the authors propose a 2-step classifier that separates messages as subjective and objective, and further distinguishes the subjective tweets as positive or negative. Davidov et al. [6] exploit the hashtags and smileys in tweets and evaluate the contribution of different features (e.g. unigrams) together with a NN classifier. In this paper, we adopt this approach and create a parallel and distributed version of the algorithm for large scale Twitter data. Agarwal et al. [1] explore the use of a tree kernel model for detecting sentiment orientation in tweets. A three-step classifier is proposed in [10] that follows a target-dependent sentiment classification strategy by incorporating target-dependent features and taking related tweets into consideration. Moreover, the authors in [18] perform a topic sentiment analysis in Twitter data through a graph-based model. A more recent approach [22], investigates the role of emoticons for multidimensional sentiment analysis of Twitter by constructing a sentiment and emoticon lexicon. A large scale solution is presented in [11] where the authors build a sentiment lexicon and classify tweets using a MapReduce algorithm and a distributed database model. Although the classification performance is quite good, the construction of sentiment lexicon needs a lot of time. Our approach is much simpler and, to our best knowledge, we are the first to present a robust large scale approach for opinion mining on Twitter data without the need of building a sentiment lexicon or proceeding to any manual data annotation.

3 MR-SAT Approach

We begin this section by providing a formal definition of the problem we try to tackle and then we present the features we use for sentiment classification. Finally, we describe our algorithm using pseudo-codes and proceed to a step by step explanation of each pseudo-code. Assume a set of hashtags and a set of emoticons associated with a set of tweets (training set). Each carries only one sentiment label from . This means that tweets containing more that one labels from are not candidates for , since their sentiment tendency may be vague. Given a set of unlabelled tweets (test set), we aim to infer the sentiment polarities for , where and means that the tweet carries no sentiment information. We build a tweet-level classifier and adopt a NN strategy to decide the sentiment tendency . We implement by adapting an existing MapReduce classification algorithm based on ANN queries [15], as described in Subsection 3.3.

3.1 Feature Description

In this subsection, we present in details the features used in order to build classifier . For each tweet we combine its features in one feature vector. We apply the features proposed in [6] with some necessary modifications to avoid the production of an exceeding amount of calculations, thus boosting the running performance of our algorithm.

3.1.1 Word and N-Gram Features

We treat each word in a tweet as a binary feature. Respectively, we consider 2-5 consecutive words in a sentence as a binary n-gram feature. If

is a word or n-gram feature, then is the weight of in the feature vector, is the number of times appear in the tweet and declares the count of in the Twitter corpus. Consequently, rare words and n-grams have a higher weight than common words and have a greater effect on the classification task. Moreover, we consider sequences of two or more punctuation symbols as word features. Unlike what authors propose in [6], we do not include the substituted meta-words for URLs, references and hashtags (URL, REF and TAG respectively) as word features (see and Section 4). Also, the common word RT, which means "retweet", does not constitute a feature. The reason for omission of these words from the feature list lies in the fact that they appear in the majority of tweets inside the dataset. So, their contribution as features is negligible, whilst they lead to a great computation burden during the classification task.

3.1.2 Pattern Features

This is the main feature type and we apply the pattern definitions given in [5] for automated pattern extractions. We classify words into three categories: high-frequency words (HFWs), content words (CWs) and regular words (RWs). A word whose corpus frequency is more (less) than (

) is considered to be a HFW (CW). The rest of the words are characterized as RWs. The word frequency is estimated from the training set rather than from an external corpus. In addition, we treat as HFWs all consecutive sequences of punctuation characters as well as URL, REF, TAG and RT meta-words for pattern extraction. We define a pattern as an ordered sequence of HFWs and slots for content words. The upper bound for

is set to 1000 words per million and the lower bound for is set to 100 words per million. Observe that the and bounds allow overlap between some HFWs and CWs. To address this issue, we follow a simple strategy as described next. Assume is the frequency of a word in the corpus; if the word is classified as HFW, else if the word is classified as CW.

We seek for patterns containing 2-6 HFWs and 1-5 slots for CWs. Moreover, we require patterns to start and to end with a HFW, thus a minimal pattern is of the form [HFW][CW slot][HFW]. Additionally, we allow approximate pattern matching in order to enhance the classification performance. Approximate pattern matching is the same as exact matching, with the difference that an arbitrary number of RWs can be inserted between the pattern components. Since the patterns can be quite long and diverse, exact matches are not expected in a regular base. So, we permit approximate matching in order to avoid large sparse feature vectors. The weight

of a pattern feature is defined as in case of exact pattern matching and as in case of approximate pattern matching, where in all experiments.

3.1.3 Punctuation Features

The last feature type is divided into five generic features as follows: 1) tweet length in words, 2) number of exclamation mark characters in the tweet, 3) number of question mark characters in the tweet, 4) number of quotes in the tweet and 5) number of capital/capitalized words in the tweet. The weight of a punctuation feature is defined as , where is the number of times feature appears in the tweet, is the maximal observed value of in the twitter corpus and declare the maximal values for word, n-gram and pattern feature groups, respectively. So, is normalized by averaging the maximal weights of the other feature types.

3.2 Bloom Filter Integration

Bloom filters are data structures proposed by Bloom [3] for checking element membership in any given set. A Bloom filter is a bit vector of length , where initially all the bits are set to 0. We can map an element into the domain between 0 and of the Bloom filter, using independent hash functions . In order to store each element into the Bloom filter, is encoded using the hash functions and all bits having index positions for are set to 1.

Bloom filters are quite useful and they compress the storage space needed for the elements, as we can insert multiple objects inside a single Bloom filter. In the context of this work, we employ Bloom filters to transform our features to numbers, thus reducing the space needed to store our feature vectors. More precisely, instead of storing a feature we store the index positions in the Bloom filter that are set to 1. Nevertheless, it is obvious that the usage of Bloom filters may impose errors when checking for element membership, since two different elements may end up having exactly the same bits set to 1. The error probability is decreased as the number of bits and hash functions used grows. As shown in the experimental evaluation, the side effects of Bloom filters are negligible and boost the performance of our algorithm.

3.3 kNN Classification Algorithm

In order to assign a sentiment label for each tweet in , we apply a NN strategy. Initially, we build the feature vectors for all tweets inside the training and test datasets ( and respectively). Then, for each feature vector in we find all the feature vectors in that share at least one word/n-gram/pattern feature with (matching vectors). After that, we calculate the Euclidean distance and keep the lowest values, thus forming and each has an assigned sentiment label . Finally, we assign the label of the majority of vectors in . If no matching vectors exist for , we assign a "neutral" label. We build by adjusting an already implemented ANN classifier in MapReduce to meet the needs of opinion mining problem.

3.4 Algorithmic Description

In this subsection, we describe in detail the sentiment classification process as implemented in the Hadoop framework. We adjust an already implemented MapReduce ANN classifier to meet the needs of opinion mining problem. Our approach consists of a series of four MapReduce jobs, with each job providing input to the next one in the chain. These MapReduce jobs can be summarized as follows: 1) Feature Extraction: Extract the features from all tweets in and , 2) Feature Vector Construction: Build the feature vectors and respectively, 3) Distance Computation: For each vector find the matching vectors (if any exist) in , calculate the Euclidean distance and form , 4) Sentiment Classification: Assign a sentiment label .

The records provided as input to our algorithm have the format tweet_id, class, text , where class refers either to a sentiment label for tweets in either to a no-sentiment flag for tweets in . In the following subsections, we describe each MapReduce job separately and analyze the Map and Reduce functions that take place in each one of them.

3.4.1 Feature Extraction

1:function Map()
2:     
3:     
4:     for all  do BF is BloomFilter
5:         
6:     end for
7:end function
8:
9:function Reduce()
10:     
11:     for all  do
12:         
13:     end for
14:     
15:     for all  do
16:         
17:         
18:     end for
19:     
20:end function
MapReduce Job 1

In this MapReduce job, we extract the features, as described in Subsection 3.1, of tweets in and and calculate their weights. The output of the job is an inverted index, where the key is the feature itself and the value is a list of tweets that contain it. In the MapReduce Job 1 pseudo-code, we sum up the Map and Reduce functions of this process.

The Map function takes as input the records from and , extracts the features of tweets. Afterwards, for each feature it outputs a key-value record, where the feature itself is the key and the value consists of the id of the tweet, the class of the tweet and the number of times the feature appears inside the sentence. The Reduce function receives the key-value pairs from the Map function and calculates the weight of a feature in each sentence. Then, it forms a list with the format , where is the id of the -th tweet, is the weight of the feature for this tweet and is its class. For each key-value pair, the Reduce function outputs a record where the feature is the key and the value is list .

3.4.2 Feature Vector Construction

In this step, we build the feature vectors and needed for the subsequent distance computation process. To achieve this, we combine all features of a tweet into one single vector. Moreover, we generate a list () of tweets in that share at least one word/n-gram/pattern feature. The Map and Reduce functions are outlined in the following MapReduce Job 2 pseudo-code.

1:function Map()
2:     
3:     
4:     for all  do
5:         
6:         if  then
7:              
8:         else
9:              
10:         end if
11:     end for
12:     for all  do
13:         
14:     end for
15:end function
16:
17:function Reduce()
18:     
19:     for all  do
20:         if  then
21:              
22:         else
23:              
24:         end if
25:     end for
26:     if  then
27:         
28:     else
29:         
30:     end if
31:end function
MapReduce Job 2

Initially, the Map function separates the tweets that contain into two lists, and respectively. Also, it outputs a key-value record, where the key is the tweet id that contains and the value consists of and weight of . Next, it generates a record where the key is the id of and the value is the list. The Reduce function gathers key-value pairs with the same key and build and . For each tweet () it outputs a record where key is the id of () and the value is its feature vector (feature vector together with the list).

3.4.3 Distance Computation

In MapReduce Job 3, we create pairs of matching vectors between and and compute their Euclidean distance. The Map and Reduce functions are depicted in the pseudo-code that follows.

1:function Map()
2:     
3:     
4:     if  then
5:         for all  do
6:              
7:         end for
8:     else
9:         
10:     end if
11:end function
12:
13:function Reduce()
14:     
15:     for all  do
16:         if  then
17:              
18:         else
19:              
20:         end if
21:     end for
22:     for all  do
23:         
24:     end for
25:end function
MapReduce Job 3

For each feature vector , the Map function outputs all pairs of vectors in list of . The output key-value record has as key the id of and the value consists of the class of , the id of and the itself. Moreover, the Map function outputs all feature vectors in . The Reduce function concentrates all matching vectors in and computes the Euclidean distances between pairs of vectors. The Reduce function produces key-value pairs where the key is the id of and the value comprises of the id of , its class and the Euclidean distance between the vectors.

3.4.4 Sentiment Classification

This is the final step of our proposed approach. In this job, we aggregate for all feature vectors in the test set, the vectors with the lowest Euclidean distance to , thus forming . Then, we assign to the label (class) of the majority of , or the label if . The MapReduce Job 4 pseudo-code is given below.

1:function Map()
2:     
3:     
4:end function
5:
6:function Reduce()
7:     
8:     
9:     
10:     
11:     for all  do
12:         if  then
13:              
14:              
15:         end if
16:     end for
17:     
18:end function
MapReduce Job 4

The Map function is very simple and it just dispatches the key-values pairs it receives to the Reduce function. For each feature vector in the test set, the Reduce function keeps the feature vectors with the lowest distance to and then estimates the prevailing sentiment label (if exists) among these vectors. Finally, it assigns to the label .

4 Experimental Evaluation

In this section, we conduct a series of experiments to evaluate the performance of our method under many different perspectives. More precisely, we take into consideration the effect of and Bloom filters, the space compaction ratio and the size of the dataset in the performance of our solution.

Our cluster includes 4 computing nodes (VMs), each one of which has four 2.4 GHz CPU processors, 11.5 GB of memory, 45 GB hard disk and the nodes are connected by 1 gigabit Ethernet. On each node, we install Ubuntu 14.04 operating system, Java 1.7.0_51 with a 64-bit Server VM, and Hadoop 1.2.1. Moreover, we apply the following changes to the default Hadoop configurations: the replication factor is set to 1; the maximum number of Map and Reduce tasks in each node is set to 3 (consequently we set the number of Reduce tasks to 12), the DFS chunk size is 64 MB and the size of virtual memory for each Map and Reduce task is set to 512 MB.

We evaluate our method using two Twitter datasets (one for hashtags and one for emoticons) we have collected through the Twitter Search API666https://dev.twitter.com/rest/public/search between November 2014 to August 2015. We have used two human judges to create a list of hashtags and a list emoticons that express strong sentiment (e.g #bored and :)). We performed some experimentation to exclude from the lists the hashtags and emoticons that either were abused by twitter users or returned a very small number of tweets. We ended up with a list of 13 hashtags and a list of 4 emoticons. We preprocessed the datasets we collected and kept only the English tweets which contained 5 or more proper English words777To identify the proper English word we used an available WN-based English dictionary and do not contain two or more hashtags or emoticons from the aforementioned lists. Moreover, during preprocessing we have replaced URL links, hashtags and references by URL/REF/TAG meta-words as stated in [6]. The final hashtags dataset contains 942188 tweets (72476 tweets for each class) and the final emoticons dataset contains 1337508 tweets (334377 tweets for each class). In both datasets, hashtags and emoticons are used as sentiment labels and for each sentiment label there is an equal amount of tweets. Finally, we produced two no-sentiment datasets by randomly sampling 72476 and 334377 tweets with no hashtags/emoticons from the dataset used in [4] and is publicly available888https://archive.org/details/twitter_cikm_2010. We assume that such random samples are unlikely to contain a significant amount of sentiment sentences. These datasets are used for the binary classification experiments (see Section 4.1).

We assess the classification performance of our algorithm using the 10-fold cross validation method and measuring the harmonic f-score. For the Bloom filter construction we use 999 bits and 3 hash functions. In order to avoid a significant amount of computations that greatly affect the running performance of the algorithm, we define a weight threshold

for feature inclusion in the feature vectors. In essence, we eliminate the most frequent words that have no substantial contribution to the final outcome.

4.1 Classification Performance

In this subsection we measure the classification performance of our solution using the harmonic f-score. We use two experimental settings, the multi-class classification and the binary classification settings. Under multi-class classification we attempt to assign a single label to each of vectors in the test set. In the binary classification experiments, we classified a sentence as either appropriate for a particular label or as not bearing any sentiment. As stated and in [6], the binary classification is a useful application and can be used as a filter that extracts sentiment sentences from a corpus for further processing. We also test how the performance is affected with and without using Bloom filters. The value for the NN classifier is equal to 50. The results of the experiments are displayed in Table 1. In case of binary classification, the results depict the average score for all classes.

Setup BF NBF Random baseline
Multi-class Hashtags 0.32 0.33 0.08
Multi-class Emoticons 0.55 0.56 0.25
Binary Hashtags 0.74 0.53 0.5
Binary Emoticons 0.77 0.69 0.5
Table 1: Classification results for emoticons and hashtags (BF stands for Bloom filter and NBF for no Bloom filter)

For multi-class classification the results are not very good but still they are way above the random baseline. We also observe that the results with and without the Bloom filters are almost the same. Thus, we deduce that for multi-class classification the Bloom filters marginally affect the classification performance. Furthermore, the outcome for emoticons is significantly better than hashtags which is expected due to the lower number of sentiment types. This behavior can also be explained by the ambiguity of hashtags and some overlap of sentiments. In case of binary classification there is a notable difference between the results with and without Bloom filters. These results may be somewhat unexpected but can be explicated when we take a look in Table 2. Table 2 presents the fraction of test set tweets that are classified as neutral because of the Bloom filters and/or the weight threshold (no matching vectors are found). Notice that the integration of Bloom filters, leads to a bigger number of tweets with no matching vectors. Obviously, the excluded tweets have an immediate effect to the performance of the NN classifier in case of binary classification. This happens since the number of tweets in the cross fold validation process is noticeably smaller compared to the multi-class classification. Overall, the results for binary classification with Bloom filters confirm the usefulness of our approach.

Setup BF NBF
Multi-class Hashtags 0.05 0.01
Multi-class Emoticons 0.05 0.02
Binary Hashtags 0.05 0.03
Binary Emoticons 0.08 0.06
Table 2: Fraction of tweets with no matching vectors

4.2 Effect of k

In this subsection, we attempt to alleviate the problem of our approach’s low performance for binary classification without Bloom filters. To achieve this we measure the effect of in the classification performance of the algorithm. We test four different configurations where . The outcome of this experimental evaluation is demonstrated in Table 3. For both binary and multi-class classification, increasing affects slightly (or not at all) the harmonic f-score when we embody Bloom filters. The same thing does not apply when we do not use Bloom filters. More specifically, there is a great enhancement in the binary classification performance for hashtags and emoticons and a smaller improvement in case of multi-class classification. The inference of this experiment, is that larger values of can provide a great impulse in the performance of the algorithm when not using Bloom filters.

Setup
Multi-class Hashtags BF 0.32 0.32 0.32 0.32
Multi-class Hashtags NBF 0.33 0.35 0.37 0.37
Multi-class Emoticons BF 0.55 0.55 0.55 0.55
Multi-class Emoticons NBF 0.56 0.58 0.6 0.6
Binary Hashtags BF 0.74 0.75 0.75 0.75
Binary Hashtags NBF 0.53 0.62 0.68 0.72
Binary Emoticons BF 0.77 0.77 0.78 0.78
Binary Emoticons NBF 0.69 0.75 0.78 0.79
Table 3: Effect of in classification performance

4.3 Space Compression

As stated and above, the Bloom filters can compact the space needed to store a set of elements, since more than one object can be stored to the bit vector. In this subsection, we elaborate on this aspect and present the compression ratio in the feature vectors when exploiting Bloom filters (in the way presented in Section 3.2) in our framework. The outcome of this measurement is depicted in Fig. 1. In all cases, the Bloom filters manage to diminish the storage space required for the feature vectors by a fraction between 15-20%. According to the analysis made so far, the importance of Bloom filters in our solution is twofold. They manage to both preserve a good classification performance, despite any errors they impose, and compact the storage space of the feature vectors. Consequently, we deduce that Bloom filters are very beneficial when dealing with large scale sentiment analysis data, that generate an exceeding amount of features during the feature vector construction step.

Figure 1: Space compression of feature vector

4.4 Running Time and Scalability

In this final experiment, we compare the running time for multi-class and binary classification and measure the scalability of our approach. Initially, we calculate the execution time in all cases in order to detect if the Bloom filters speedup or slow down the running performance of our algorithm. The results when are presented in Fig. 2. It is worth noted that in the majority of cases, Bloom filters slightly boost the execution time performance. Despite needing more preprocessing time to produce the features with Bloom filters, in the end they pay off since the feature vector is smaller in size. This leads to lower I/O cost between the Map and Reduce tasks and consequently to less processing time. Multi-class classification for emoticons constitutes the only exception in our example.

Figure 2: Running time

Finally, we investigate the scalability of our approach. We test the scalability only for the multi-class classification case since the produced feature vector in much bigger compared to the binary classification case. We create new chunks smaller in size that are a fraction of the original datasets, where {0.2, 0.4, 0.6, 0.8}. Moreover, we set the value of to 50. Figure 3 presents the scalability results of our approach. From the outcome, we deduce that our algorithm scales almost linear as the data size increases in all cases. This proves that our solution is efficient, robust, scalable and therefore appropriate for big data sentiment analysis.

Figure 3: Scalability

5 Conclusions and Future Work

In the context of this work, we presented a novel method for sentiment learning in the MapReduce framework. Our algorithm exploits the hashtags and emoticons inside a tweet, as sentiment labels, and proceeds to a classification procedure of diverse sentiment types in a parallel and distributed manner. Moreover, we utilize Bloom filters to compact the storage size of intermediate data and boost the performance of our algorithm. We conduct a variety of experiments to test the efficiency of our method. Through this extensive experimental evaluation we prove that our system is efficient, robust and scalable.

In the near future, we plan to extend and improve our framework by exploring more features that may be added in the feature vector and will increase the classification performance. Furthermore, we wish to explore more strategies for and bounds in order to achieve better separation between the HFWs and CWs. Finally, we plan to implement our solution in other platforms (e.g. Spark) and compare the performance with the current implementation.

References

  • [1] A. Agarwal, B. Xie, I. Vovsha, O. Rambow, and R. Passonneau. Sentiment analysis of twitter data. In Proceedings of the Workshop on Languages in Social Media, pages 30–38, 2011.
  • [2] L. Barbosa and J. Feng. Robust sentiment detection on twitter from biased and noisy data. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 36–44, 2010.
  • [3] B. H. Bloom. Space/time trade-offs in hash coding with allowable errors. Commun. ACM, 13(7):422–426, 1970.
  • [4] Z. Cheng, J. Caverlee, and K. Lee. You are where you tweet: A content-based approach to geo-locating twitter users. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management, pages 759–768, 2010.
  • [5] D. Davidov and A. Rappoport. Efficient unsupervised discovery of word categories using symmetric patterns and high frequency words. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pages 297–304, 2006.
  • [6] D. Davidov, O. Tsur, and A. Rappoport. Enhanced sentiment learning using twitter hashtags and smileys. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 241–249, 2010.
  • [7] J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation, pages 137–150, 2004.
  • [8] X. Ding and B. Liu. The utility of linguistic rules in opinion mining. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 811–812, 2007.
  • [9] M. Hu and B. Liu. Mining and summarizing customer reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 168–177, 2004.
  • [10] L. Jiang, M. Yu, M. Zhou, X. Liu, and T. Zhao. Target-dependent twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, pages 151–160, 2011.
  • [11] V. N. Khuc, C. Shivade, R. Ramnath, and J. Ramanathan. Towards building large-scale distributed systems for twitter sentiment analysis. In Proceedings of the 27th Annual ACM Symposium on Applied Computing, pages 459–464, 2012.
  • [12] C. Lin and Y. He. Joint sentiment/topic model for sentiment analysis. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, pages 375–384, 2009.
  • [13] Q. Mei, X. Ling, M. Wondra, H. Su, and C. Zhai. Topic sentiment mixture: Modeling facets and opinions in weblogs. In Proceedings of the 16th International Conference on World Wide Web, pages 171–180, 2007.
  • [14] T. Nasukawa and J. Yi. Sentiment analysis: Capturing favorability using natural language processing. In Proceedings of the 2Nd International Conference on Knowledge Capture, pages 70–77, 2003.
  • [15] N. Nodarakis, E. Pitoura, S. Sioutas, A. K. Tsakalidis, D. Tsoumakos, and G. Tzimas. Efficient multidimensional aknn query processing in the cloud. In Database and Expert Systems Applications - 25th International Conference, DEXA 2014, Munich, Germany, September 1-4, 2014. Proceedings, Part I, pages 477–491, 2014.
  • [16] B. Pang, L. Lee, and S. Vaithyanathan. Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - Volume 10, pages 79–86, 2002.
  • [17] The apache software foundation: Hadoop homepage. http://hadoop.apache.org/, 2013. [Online; accessed 20-September-2015].
  • [18] X. Wang, F. Wei, X. Liu, M. Zhou, and M. Zhang. Topic sentiment analysis in twitter: A graph-based hashtag sentiment classification approach. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management, pages 1031–1040, 2011.
  • [19] T. White. Hadoop: The Definitive Guide, 3rd Edition. O’Reilly Media / Yahoo Press, 2012.
  • [20] T. Wilson, J. Wiebe, and P. Hoffmann. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 347–354, 2005.
  • [21] T. Wilson, J. Wiebe, and P. Hoffmann. Recognizing contextual polarity: An exploration of features for phrase-level sentiment analysis. Comput. Linguist., 35(3):399–433, Sept. 2009.
  • [22] Y. Yamamoto, T. Kumamoto, and A. Nadamoto. Role of emoticons for multidimensional sentiment analysis of twitter. In Proceedings of the 16th International Conference on Information Integration and Web-based Applications & Services, pages 107–115, 2014.
  • [23] H. Yu and V. Hatzivassiloglou. Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pages 129–136, 2003.
  • [24] W. Zhang, C. Yu, and W. Meng. Opinion retrieval from blogs. In Proceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management, pages 831–840, 2007.
  • [25] L. Zhuang, F. Jing, and X.-Y. Zhu. Movie review mining and summarization. In Proceedings of the 15th ACM International Conference on Information and Knowledge Management, pages 43–50, 2006.