Artificial Intelligence: Deep Learning Based Sentiment Analysis using Ensembles (Short Review)

In this blog we discuss an research paper by (Araque et al) [1] named ” Enhancing deep learning sentiment analysis with ensemble techniques in social applications”  in which Artificial Intelligence techniques  for sentiment analysis and deep learning is used with advanced classification technique of ensemble based learning.   This article [1] propose, analyze the various ways in which an ensemble based  technique can be used for sentiment analysis using deep learning techniques, which had not been done before to this much exhaustive levels.  

Firstly, the three important things in this article are explained in my terminology follows:

1. Sentiment Analysis: Every text or review or even tweets to mention a few can carry a positive sentiment or a neural message or may be of a negative outlook. The positivity and negativity in emotions emphasised is expressed in Natural Language Processing as measure of sentiments.
2. Ensemble based learning: Typically, when the number of features is high. A subset from original problem is selected and considered for classification, many such classifications are performed on several such sub-problems. The final classification is performed with any of the possible techniques such as (i) majority voting ( the maximum of all the classes for a particular instance of testing set) (ii) weighted average of classes (iii) scoring, to mention a few. This is called an ensemble. The process I have described here is called bagging, as is evident from name bag of classifiers are used to predict the class of a sample of data.
3. The third prominent thing in this research is Deep Learning. Deep learning techniques are latest state-of-art algorithms using Artificial Intelligence. It uses learning as is performed by human brains.
4. Word Embeddings: These are representations of words in a particular domain of knowledge which are learned by training data. The word is represented by weights of final neural network by using the context. That is the representation of word depends on representation on words surrounding it in a domain of training data. This is CBOG model. In which the word is predicted using words surrounding it which other (on which GLOVE is based) is the skip gram model wherein the word predicts its neighboring words.  The technique requires word embeddings. These are representations of text (a sentence or document, both can be considered) in a smaller dimension typically, 50, 100, 200 or 300 dimensional vectors. These are learned by deep learning techniques. The embeddings learned on a particular domain or data set, leads to much better word embedding as context is understood by this and better representations can be made for the words and hence the textual data.   [more in separate article]

——————————————–REVIEW———————————————–

Here starts the short review of paper. The points the authors [1] have emphasized in article are [1] Surface level features, such as POS tags, word boundaries, .. ,words specifically trained for sentiment analysis tasks and generic word embeddings vectors. Several research have been performed before this research paper, but not a comprehensive analysis have been performed on various combinations of these feature selection techniques of sentiment analysis and how to use ensembles over it with deep learning. The paper has a comprehensive analysis of ensemble techniques and its comparison on performance for features extracted by deep learning models, traditional features and various combinations of these.  These are the key novelties of the paper and it deals with how these are selected and how various models are combined.

Once word embedding has been made for a review or a tweet and its class—positive or negative is fed to supervised classifier. The problem takes a form of a typical numerical classification form. From this various bags of classifiers can be derived by considering subset of this classifier, typically by selecting lower number of features. From each bag a classifier is generated, the authors [1] have taken logistic regression as the classifies and the final class of the ensemble is derived.

Testing and evaluations have been performed on six public datasets including semeval and IMDB datasets to include domains such as twitter and movie reviews.

Reference

[1] Oscar Araque, Ignacio Corcuera-Platas, J. Fernando Sánchez-Rada, Carlos A. Iglesias. “Enhancing deep learning sentiment analysis with ensemble techniques in social applications”. Expert Systems With Applications 77 (2017) pp. 236-246