MSGAT-Based Sentiment Analysis for E-Commerce
Abstract
:1. Introduction
- A multiscale GAT [12] model based on syntactic dependency trees is proposed to extract sentiment features of sentences as a function of their syntactic structure, which solves the problem of CNN, LSTM, and other models ignoring the syntactic features of sentences.
- A method for constructing graph networks after encoding Chinese utterances is proposed. The representation of node features in the syntactic spanning tree is obtained when syntactic analysis is performed on Chinese utterances.
- Two Chinese e-commerce review datasets are constructed; the proposed model is applied to the datasets, and good results are obtained.
2. Related Work
3. MSGAT
3.1. Model Architecture
3.1.1. Sentence Encoding
3.1.2. Construct Parse Tree
3.1.3. Vertex Representation
3.1.4. Generate Edge List
3.1.5. Feature Extraction
3.1.6. Classification
4. Experiment and Discussion
4.1. Dataset
4.2. Experimental Environment
4.3. Baselines
4.4. Experimental Results and Analysis
4.5. Ablation Study
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Hinton, G.E. Learning distributed representations of concepts. In Proceedings of the Eighth Conference of the Cognitive Science Society, Amherst, MA, USA, 15–17 August 1986; pp. 1–12. [Google Scholar]
- Bengio, Y.; Ducharme, R.; Vincent, P.; Jauvin, C. A neural probabilistic language model. J. Mach. Learn. Res. 2006, 3, 1137–1155. [Google Scholar]
- Mikolov, T.; Chen, K.; Corrado, G.; Dean, J. Efficient Estimation of Word Representations in Vector Space. arXiv 2013, arXiv:1301.3781. [Google Scholar]
- Pennington, J.; Socher, R.; Manning, C. Glove: Global Vectors for Word Representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 26–28 October 2014. [Google Scholar]
- Peters, M.; Neumann, M.; Iyyer, M.; Gardner, M.; Clark, C.; Lee, K.; Zettlemoyer, L. Deep Contextualized Word Representations. arXiv 2018, arXiv:1802.05365. [Google Scholar]
- Devlin, J.; Chang, M.-W.; Lee, K.; Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv 2018, arXiv:1810.04805. [Google Scholar]
- Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; Stoyanov, V. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv 2019, arXiv:1907.11692. [Google Scholar]
- Cui, Y.; Che, W.; Liu, T.; Qin, B.; Wang, S.; Hu, G. Revisiting Pre-Trained Models for Chinese Natural Language Processing. arXiv 2020, arXiv:2004.13922. [Google Scholar]
- MiKolov, T.; Karaflat, M.; Burget, L.; Černocký, J.; Khudanpur, S. Recurrent neural network based language model. In Proceedings of the INTERSPEECH 2010, Conference of the International Speech Communication Association, Chiba, Japan, 26–30 September 2010; pp. 1045–1048. [Google Scholar]
- Kim, Y. Convolutional Neural Networks for Sentence Classification. arXiv 2014, arXiv:1408.5882. [Google Scholar]
- Sundermeyer, M.; Schluter, R.; Ney, H. LSTM Neural Networks for Language Modeling. In Proceedings of the INTERSPEECH, Portland, OR, USA, 9–13 September 2012; pp. 194–197. [Google Scholar]
- Velikovi, P.; Cucurull, G.; Casanova, A.; Liò, P.; Romero, A. Graph Attention Networks. In Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada, 30 April–3 May 2018. [Google Scholar]
- Wang, T.; Yang, W. A review of text sentiment analysis methods. Comput. Eng. Appl. 2021, 57, 11–24. [Google Scholar]
- Zhou, C.; Li, S.; Xu, Y. Research on text sentiment analysis. In Proceedings of the China Computer Users Association Web Application Branch 2018 22nd Annual Conference on New Technologies and Applications for the Web, Suzhou, China, 8–10 November 2018; pp. 302–305. [Google Scholar]
- Hu, M.; Bing, L. Mining and summarizing customer reviews. In Proceedings of the Tenth ACM Sigkdd International Conference on Knowledge Discovery & Data Mining, Seattle, WA, USA, 22–25 August 2004; pp. 168–177. [Google Scholar]
- Yang, M.; Peng, B.; Chen, Z. A Topic Model for Building Fine-Grained Domain- Specific Emotion Lexicon. In Proceedings of the Association for Computational Linguistics (ACL), Baltimore, MD, USA, 22–27 June 2014; pp. 421–426. [Google Scholar]
- Zhou, Y.; Yang, A.; Lin, J. A method for constructing Chinese microblogging emotion dictionary. J. Shandong Univ. 2014, 44, 36–40. [Google Scholar]
- Pang, B.; Lee, L.; Vaithyanathan, S. Thumbs up: Sentiment Classification Using Machine Learning Techniques. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, Philadelphia, PA, USA, 6 July 2002; pp. 79–86. [Google Scholar]
- Wikarsa, L.; Thahir, S.N. A Text Mining Application of Emotion Classifications of Twitter’s Users Using Naive Bayes Method. In Proceedings of the Wireless and Telematics (ICWT), Manado, Indonesia, 17–18 November 2015. [Google Scholar]
- Su, Y.; Zhang, Y.; Hu, P.; Tu, X.H. Sentiment analysis based on the combination of plain Bayesian and latent Dirichlet distributions. Comput. Appl. 2016, 36, 1613–1618. [Google Scholar]
- Chai, Y. Research on sentiment analysis of book review text based on LSTM and Word2vec. Inf. Technol. 2022, 7, 59–64. [Google Scholar]
- Wang, X.; Liu, Y.; Sun, C.; Wang, B.; Wang, X. Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), Beijing, China, 26–31 July 2015. [Google Scholar]
- Wang, J.; Yu, L.-C.; Lai, R.K.; Zhang, X. Dimensional Sentiment Analysis Using a Regional CNN-LSTM Model. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), Berlin, Germany, 7–12 August 2016. [Google Scholar]
- Feng, X.J.; Zhang, Z.W.; Shi, J.C. Text sentiment analysis based on convolutional neural network and attention model. Comput. Appl. Res. 2018, 35, 1434–1436. [Google Scholar]
- Li, Y.; Dong, H.-B. Text sentiment analysis based on CNN and BiLSTM network feature fusion. Comput. Appl. 2018, 38, 3075–3080. [Google Scholar]
- Gao, H.; Ji, S. Graph U-Nets. Inst. Electr. Electron. Eng. (IEEE) 2022, 44, 4948–4960. [Google Scholar] [CrossRef]
Relation | Description |
---|---|
advmod | Adverbial modifier |
punct | Punctuation |
case | Dependencies |
amod | Adjective modifier |
conj | Parallelism |
nmod | Compound noun modifier |
nsubj | Subject–predicate relationship |
Dataset | Positive Emotions | Negative Emotions | Total | ||
---|---|---|---|---|---|
Train | Test | Train | Test | ||
Hotel review dataset | 3726 | 1596 | 1711 | 732 | 7766 |
Takeaway review dataset | 2800 | 1200 | 5592 | 2394 | 11,986 |
Dataset | Content |
---|---|
Hotel review dataset | (1, Business king room, the room is large, the bed is 2 m wide, the overall feeling of economy is good!) |
(0, I booked a suite in the secondary floor during the National Day, and it was more than a little worse, the furniture was shabby and the TV was incredibly small and unimaginably spartan.) | |
Takeaway review dataset | (1, Delicious! Fast! The packaging has quality too… restaurant food without leaving home!) |
(0, Too bad. I waited 2 h for the beef, and I was about to throw up; never again.) |
Classification | Specific Description |
---|---|
Hardware type | CPU: Intel(R) Xeon(R) W-2255 CPU @ 3.70 GHz GPU: NVIDIA GeForce RTX 3080 Ti Memory: 64 GB Hard disk: 4 TB |
Software version | OS: Windows 10 python: 3.9.12 torch: 1.12.0 pycharm: 2022.1.3 |
Parameter Name | Description | Value |
---|---|---|
Batch size | Volume of data per batch | 32 |
Epoch | Number of times the dataset was learned | 20 |
Learning rate | Learning rate | 10−5 |
Optimizer | Optimizer | AdamW |
Dropout | Random drop rate | 0.5 |
Model | Hotel Review Dataset | Takeaway Review Dataset | ||
---|---|---|---|---|
Accuracy | F1 | Accuracy | F1 | |
RoBERTa | 90.84 | 93.46 | 90.76 | 85.29 |
RoBERTa + BiLSTM | 93.49 | 95.27 | 91.99 | 87.73 |
RoBERTa + TextCNN | 93.62 | 95.31 | 92.51 | 88.62 |
RoBERTa + MSGAT | 94.79 | 96.23 | 93.72 | 90.44 |
Number of GAT Layers | Accuracy | F1 |
---|---|---|
Single GAT | 94.01 | 95.63 |
Double GAT | 94.79 | 96.23 |
Triple GAT | 94.36 | 95.88 |
Global Aggregate Function | Accuracy | F1 |
---|---|---|
global_mean_pool | 94.79 | 96.23 |
global_max_pool | 94.45 | 95.96 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Jiang, T.; Sun, W.; Wang, M. MSGAT-Based Sentiment Analysis for E-Commerce. Information 2023, 14, 416. https://doi.org/10.3390/info14070416
Jiang T, Sun W, Wang M. MSGAT-Based Sentiment Analysis for E-Commerce. Information. 2023; 14(7):416. https://doi.org/10.3390/info14070416
Chicago/Turabian StyleJiang, Tingyao, Wei Sun, and Min Wang. 2023. "MSGAT-Based Sentiment Analysis for E-Commerce" Information 14, no. 7: 416. https://doi.org/10.3390/info14070416
APA StyleJiang, T., Sun, W., & Wang, M. (2023). MSGAT-Based Sentiment Analysis for E-Commerce. Information, 14(7), 416. https://doi.org/10.3390/info14070416