CC BY
18
A NOVEL, RELIABLE, ASPECT-BASED SENTIMENT CLASSIFICATION [NRABSC] FOR DIFFERENT INDUSTRY DOMAINS USING HYBRID DEEP LEARNING MODELS
Dhaval Bhoi1*, Amit Thakkar2
1 *
U & P U. Patel Department of Computer Engineering, Chandubhai S. Patel Institute of Technology, Faculty of Technology & Engineering, Charotar University of Science and Technology (CHARUSAT), Changa - 388421, Gujarat, India dhavalbhoi.ce@charusat.ac.in
2
Department of Computer Science & Engineering, Chandubhai S. Patel Institute of Technology, Faculty of Technology & Engineering, Charotar University of Science and Technology (CHARUSAT), Changa - 388421, Gujarat, India amitthakkar.it@charusat.ac.in
Abstract
Customers nowadays are more opinionated than they have ever been. They appreciate interacting with industries or businesses and providing feedback like positive, neutral and negative. Customers leave a plethora of information every time they connect with a company, whether it is through a mention or a review, letting businesses know what they are doing well and wrong. However, wading through all of this data by hand may be laborious task. Aspect-based sentiment analysis, on the other hand, can assist you in overcoming this issue. Because of their inherent competency in the semantic synchronization among aspects with associated contextual terms, attention mechanisms and Convolutional Neural Networks (CNNs) are often utilized for aspect-based sentiment categorization. However, because these models lack a mechanism for accounting for important syntactical restrictions and long-range word dependencies, they may incorrectly identify syntactically irrelevant contextual terms as hints for determining aspect emotion. To solve this problem, we suggest establishing a Graph Convolutional Network (GCN) over a sentence's dependency tree, which is generated using bidirectional Long Short Term Memory (Bi-LSTM). A new aspect-specific sentiment categorization system is proposed as a result of it. Studies on several testing sets show how our suggested approach is on par with a number of state-of-the-art deep learning models in terms of reliable performance efficacy, and that the graph convolution structure appropriately captures both syntactic and semantic data and lengthy-term associations to perform reliable sentiment classification based on the aspects present in the review sentences.
Keywords: Aspect Based Sentiment Classification, Performance Reliability Analysis, Deep Learning
1. Introduction
Our lives have become increasingly reliant on social media. People nowadays are not only limited to using such platforms to communicate with others, but they are also active in expressing their opinions on any event they feel worthy of discussion. This shows the need for analysing such vast
amount of multimedia data [1]. Sentiment analysis can be undertaken at several levels, including document, phrase, and feature/aspect levels. [2]. Statement level sentiment analysis is used to categorise a sentence into one of three categories: negative, positive, or neutral. To categorise a document, sentiment analysis at the document level is used. While sentiment analysis at the aspect level is used to categorise each element of an aspects and entity mentioned in a review. As a consequence, decision makers in any sector, business, or institution may classify their clients' likes and dislikes in more detail. They may then make better decisions about various marketing tactics based on this information. It is possible to boost the potential of a rising industry growth performance. Different granularity level of sentiment classification is shown in Figure 1 below.
Sentiment classification relying on aspects (often referred to as aspect-level classification) seeks to find out the appropriate sentiment polarity of aspects expressly stated in sentences. For instance, the sentiment polarities for a couple of features of the pizzas and staff are positive and negative, correspondingly, in a review regarding a restaurant claiming "The pizzas were delicious, but the staff took very long to serve." This task is usually stated as guessing the polarity of a given (sentence, aspect) pair combination. It includes to identify aspect in a given sentence and to classify the statement according to the aspect into positive, negative or neutral category.
From idea through disposal, all facets of the asset lifetime are crucial to a company's or any specific industry success [3]. When using aspect-based sentiment categorization to tackle problems in diverse industries, such as the computer business and the restaurant industry, there are numerous hurdles needed to be faced. Aspect Term Extraction (ATE), Aspect Category Detection (ACD), and Aspect Based Sentiment Classification (ABSC) are among the issues. ABSC also aims to find reliable and scalable solutions to the problem automatically. To accomplish aspect-based sentiment classification in this study, we employed a novel and hybrid dependable approach named Bidirectional Long Short Term Memory (Bi-LSTM) and Graph Convolutional Network (GCN). This proposed technique has the potential to improve ABSC's performance, and in turn, it will aid several industrial domains in understanding client feedback. The proposed approach aids in the identification of emotion expressed by industry stakeholders. We have currently limited our research to perform aspect level sentiment classification on laptop domain reviews and restaurant reviews expressed by people who have used their services.
Granularity of Sentiment Analysis
Coarse-grained Sentiment Analysis (Document-level
or Sentence-level)
Fine-grained Sentiment Analysis (Phraselevel or Aspect-level)
Figure 1: Sentiment Analysis: Granularity
2. Related Work
A variety of ways for dealing with ABSC challenges have been proposed in recent years, including traditional machine learning and neural network methods. This section will cover the related study of aspect-based sentiment classification.
2.1. Machine Learning Methods
Existing machine learning techniques for the ABSC problem are largely focused on feature engineering. This also indicates that a large amount of work is spent gathering and assessing data, defining features based on the dataset's attributes, and obtaining sufficient language resources for lexicon creation. The Support Vector Machine (SVM) [4] is a well-known machine learning algorithm that gets decent results when dealing with aspect-level sentiment classification when incorporated implicit aspects into account, their aspect extraction accuracy improved significantly. However, like with other classic machine learning approaches, manually designing features is time consuming and inefficient [5]. Furthermore, the method's performance suffers significantly when the dataset changes. As a result, traditional machine learning algorithms are limited in their applicability and difficult to adapt to a variety of datasets. As a result, it is the primary rationale for opting for deep learning over machine learning.
2.2. Deep Learning Methods
Recent work is increasingly integrating with Neural Networks (NN) because NN-based techniques may extract original features and transfer them into continuous and low-dimensional vectors without the requirement for feature engineering.
Recursive Neural Network (RecNN) is a form of neural network used to learn a directed acyclic graph structure from data. Tree based RecNN was introduced by [6] and [7]. To accomplish aspect based sentiment categorization, the authors of [8] employed LSTM, Target Dependent LSTM (TD-LSTM), and Target Connection (TC-LSTM). In addition, the end-to-end training of the LSTM model for ABSC included commonsense knowledge of sentiment-related topics [9]. Bi-RNN was utilised by the authors of 1 to simulate the syntax and semantics in sentences, as well as the relationship between the aspects and their surrounds words. Authors of [10] proposes employing a hierarchical bidirectional LSTM model for ABSC that can learn both intra and inter sentence connections, which uses Hierarchical bi-directional attention-based RNNs (HRNN). These models, on the other hand, are unable to capture aspect level sentiment classification with improved performance. We presented a hybrid deep learning model to overcome the limitations of existing techniques. The next part talks at length about it.
3. Proposed System
For a given (sentence-review, aspect) pair (s, a), where aspect a= [a\, a2, ..an } is a sub-sequence of the sentence of sentence for a given sentence-review s={wi, w2, w3,.., wn}. Sentence s is classified into positive, negative or neutral according to the content of the aspect. To minimise the distance between aspect and opinion words and capture long-term dependencies, a Dependency Tree is employed to express syntactic dependencies paths between review sentence words. Bi-LSTM Model learns hidden state representation Hc = {h1, h2, h3,.., hn }, where hi represents the hidden state vector at different time step t1, t2,..tn in the forward and backward direction for an arbitrary review sentence to be contextualized.
For document-word relationships, tree topologies, link prediction, and relation extraction, GCNs have been applied. For training, GCNs can employ both node features and structure. Each GCN layer uses features from near neighbours to encode and update representations of nodes in the graph. The GCN's primary principle is to take the weighted average of all node attributes of
all neighbours (including itself): The weights of lower-degree nodes are increased. The feature vectors are then fed into a neural network for training.
AX adds up the characteristics of neighbouring nodes but ignores the characteristics of the node itself. The Adjacency Matrix is A, and the feature vector is X. Although Adjacency Matrix A with self loop has been included, they are still not normalised. To avoid numerical instability and vanishing gradient, normalisation is essential. Normalized features are calculated using value CAX, where C is the inverse of D. Our Proposed Approach is presented in the below Figure 2.
This combination allows embeddings of aspect words to have discriminative features
Figure 2: Our Proposed Approach
The sentence's features will be learned using Bi-LSTM, which will then be enhanced by using GCN.As GCN is capable of effectively capturing and encoding syntactical information as well as long-range dependencies. Only aspect vectors encoded with information from opinion words are aggregated in the final representation for the ABSC task. BiLSTM and GCN provide for discriminating features in aspect word embeddings, resulting in improved supervisory signals for the classification process.
hik+ = (p{ tcA (W(k) hf] + b(k)) (1)
j=i
Where, hj is the hidden state representation for given node j at the kth layer of GCN, b(k) is the bias term considered, W(k) is the weight parameter matrix, cl = 1/ dl (di denotes degree of node i in the constructed graph), 0(.) is the relu elementwise non-linear activation function, h(0)
represents the initial embeddings modeled by BiLSTM and h-k+1) is the final output at layer k for node i .
J(01,02) = - E Eyc((a,s))logyc((a,s)) (2)
(a,s)eD ceC
Where, D is the collection of aspect-sentence pairs datasets, C is the collection of distinct sentiment classes such as Positive, Negative and Neutral, yc((a,s)) is the actual ground truth for (a,s) while yc((a,s)) is the model prediction for givne (a,s), while (01,02) are the trainable parameters for BiLSTM and GCN.
3.1. Dataset
To asses the effectiveness and reliability of our proposed approach We have used two different kinds of datasets of computer industry and restaurant industries. Laptop14 data belongs to computer industry contains review related to laptops while Restaurant14, Restaurant15 and Restaurant16 belong to restaurant industry.
Table 1: Dataset Details
Dataset Size of the Dataset
Positive Negative Neutral
Restaurant2014-training Dataset 2164 807 637
Restaurant2014-testing Dataset 728 196 196
Laptop2014-training Dataset 994 870 464
Laptop2014-testing Dataset 341 128 169
Restaurant2015-training Dataset 1178 382 50
Restaurant2015-testing Dataset 439 328 35
Restaurant2016-training Dataset 1620 709 88
Restaurant2016-testing Dataset 597 190 38
In our study, we have included review sentences containing positive, negative and neutral sentiment classification. Dataset detail and dataset preview are shown in Table 1 and Table 2 respectively. Aspect term is denoted between two $ symbols while type of sentiment is denoted using 1, 0 and -1 for positive, neutral and negative respectively.
Table 2: Dataset Preview
Sentence Aspect Term Sentiment Type
$T$ was easy Set up 1 (Positive)
No $T$ is included Installation disk 0 (Neutral)
Did not enjoy the new arrival windows 10 and $T$ Touchscreen Functionality -1 (Negative)
3.2. Performance Evaluation Measures
To test the effectiveness of our suggested strategy NRABSC, we employed accuracy (A) and F1 score (F1) as performance assessment measures to evaluate the actual prediction performance of our proposed hybrid deep learning approach with existing deep learning models LSTM and CNN. A true positive sample is one in which the selected model predicts the positive class correctly. A true negative sample, on the other hand, is an outcome in which the model makes correct prediction regarding the negative class. A false positive sample is one in which the model estimates the positive class incorrectly. A false negative sample is one in which the model predicts the negative class incorrectly.True Positive samples are represented by TPS, True Negative samples by TNS, False Positive samples by FPS, and False Negative samples by FNS. Calculations for Accuracy, Precision, Recall and F1-Score are done using the formulas as shown in Equations 3, 4, 5 and 6.
A
TPS + FN S
TPS + FPS + TNS + FNS TPS
P
R
TPS+FPS
TPS TPS + FNS
(3)
(4)
(5)
Dhaval Bhoi, Amit Thakkar RT&A, No 3 (69)
NRABSC USING HYBRID DEEP LEARNING MODELS Volume 17, September 2022
F1 = ^ <«>
4. Results and Discussion
Proposed reliable hybrid deep learning approach NRABSC is compared with baseline models LSTM and CNN to prove its effectiveness. Experiments are performed on various standard benchmark dataset Laptop14, Restaurant14, Restaurant15 and Restaurant16. Our proposed approach gives highly reliable results for restaurant laptop and restaurant domain as shown in the below Table 3, 4 ,5, 6 and Figure 3, 4, 5 and 6 for both the performance measures Accuracy and F1-Score.
Table 3: Performance Result for Laptop14 Dataset
Model Accuracy F1-Score
LSTM 69.80 63.64
CNN 74.97 70.80
Our Proposed Model [NRABSC] 75.60 71.87
Table 4: Performance Result for Restaurant14 Dataset
Model Accuracy F1-Score
LSTM 77.98 67.35
CNN 81.04 72.52
Our Proposed Model [NRABSC] 81.37 72.44
Table 5: Performance Result for Restaurant15 Dataset
Model Accuracy F1-Score
LSTM 77.12 55.90
CNN 79.46 59.19
Our Proposed Model [NRABSC] 79.64 62.82
Table 6: Performance Result for Restaurant16 Dataset
Model Accuracy F1-Score
LSTM 86.69 65.33
CNN 87.88 64.88
Our Proposed Model [NRABSC] 88.85 72.10
5. Conclusion and Future Work
In order to increase the performance of the computer and hotel industries, the proposed approach generates extremely trustworthy outcomes. It can then be used to other industrial sectors to produce domain-independent outcomes. We looked at the problems that current systems have with aspect-specific sentiment categorization and found that GCN is well suited to solving them. As a result, we have suggested a new structure that uses GCN to classify sentiment depending on aspects. GCN improves total efficiency by exploiting both syntactical knowledge and long-range phrase relationships, according to testing data. The following components of this study should be
Figure 3: Performance Result on Laptop14 Dataset
Figure 4: Performance Result on Restaurant14 Dataset
enhanced. To begin with, the border knowledge syntactical dependence trees, i.e. the name of each border, is not used in this study. We intend to create a graph neural network that takes the edge labels into account. Existing understanding, on the other hand, could be included. Finally, by incorporating relationships among both the aspects words, the RHABSC model can be expanded to concurrently judge emotions of various features. In the future, we will highlight significant concerns and suggest possible solutions, including new models like as the BERT language model and GANs.
Figure 5: Performance Result on Restaurant15 Dataset
Figure 6: Performance Result on Restaurant16 Dataset
Acknowledgement
The authors would like to express special thanks of gratitude to the Principal (CSPIT) and Dean of the Faculty of Technology and Engineering (FTE). Authors would like to acknowledge the Head of the U & P U. Patel Department of Computer Engineering at CSPIT, Faculty of Technology and Engineering, Charotar University of Science and Technology, Changa, for their continuous suggestions, encouragement, guidance, and support throughout the research. We would like to thank Management for their unwavering moral support and encouragement throughout the years.
Declaration of Conflicting Interests The Author(s) declare(s) that there is no conflict of interest.
References
[1] Wankhade, M., Rao, A.C.S. Kulkarni, C. A survey on sentiment analysis methods, applications, and challenges. Artif Intell Rev (2022). https://doi.org/10.1007/s10462-022-10144-1.
[2] Suhariyanto, R. Abdullah, R. Sarno, and C. Fatichah, Aspect-Based Sentiment Analysis for Sentence Types with Implicit Aspect and Explicit Opinion in Restaurant Review Using Grammatical Rules, Hybrid Approach, and SentiCircle/ Int. J. Intell. Eng. Syst., vol. 14, no. 5, pp. 177-187, 2021, doi: 10.22266/ijies2021.1031.17.
[3] Igor Shubinsky , Alexei Zamyshlaev , Alexander Bochkov (2022). APPLICATION OF ARTIFICIAL INTELLIGENCE IN RUSSIA'S RAILWAY NETWORK ASSET MANAGEMENT. Reliability: Theory Applications, 17 (SI 3 (66)), 42-48. doi: 10.24412/1932-2321-2022-366-4248.
[4] Alqaryouti, O., Siyam, N., Abdel Monem, A. and Shaalan, K. (2020), "Aspect-based sentiment analysis using smart government review data", Applied Computing and Informatics, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1016/jj.aci.2019.11.003.
[5] Anadkat, K., Diwanji, H., Modasiya, S. (2022). Effect of Preprocessing in Human Emotion Analysis Using Social Media Status Dataset. Reliability: Theory Applications, 17(1), 104-112. Retrieved from http://www.gnedenko.net/RTA/index.php/rta/article/view/847.
[6] Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 49-54, Baltimore, Maryland. Association for Computational Linguistics.
[7] Thien Hai Nguyen and Kiyoaki Shirai. 2015. PhraseRNN: Phrase Recursive Neural Network for Aspect-based Sentiment Analysis. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2509"2514, Lisbon, Portugal. Association for Computational Linguistics.
[8] Duyu Tang, Bing Qin, Xiaocheng Feng, Ting Liu. 2016. Effective LSTMs for Target-Dependent Sentiment Classification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3298-3307, Osaka, Japan, December 11-17 2016.
[9] Ma, Y., Peng, H., Cambria, E. (2018). Targeted Aspect-Based Sentiment Analysis via Embedding Commonsense Knowledge into an Attentive LSTM. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1).
[10] Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2016. Gated neural networks for targeted sentiment analysis. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI'16). AAAI Press, 3087-3093.
