JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE) ›› 2018, Vol. 53 ›› Issue (3): 13-23.doi: 10.6040/j.issn.1671-9352.0.2017.064

Previous Articles     Next Articles

Deep representative learning based sentiment analysis in the cross-lingual environment

YU Chuan-ming1, FENG Bo-lin1, TIAN Xin1, AN Lu2*   

  1. 1. School of Information and Safety Engineering, Zhongnan University of Economics and Law, Wuhan 430073, Hubei, China;
    2. School of Information Management, Wuhan University, Wuhan 430072, Hubei, China
  • Received:2017-04-15 Online:2018-03-20 Published:2018-03-13

PDF (PC)

1610

Abstract: Transfer learning focuses on solving the problem that it is difficult for supervised learning to obtain good classification results with the small training sets. Compared with the traditional supervised methods, it does not require the training and testing sets follow the same or similar data distributions. The model can be trained on the language which has rich data and labeling resources(source language), and the source and target language documents can be projected into the same feature space. In this way the large data from the source domain or language can be leveraged to solve the low performance problem in the target language. The reviews of three product categories, i.e. books, DVD and music, from Amazon, which are written in Chinese, English and Japanese, are collected as the experimental data. A novel model, i.e. the Cross Lingual Deep Representation Learning(CLDRL)is proposed and empirical study is conducted upon the experimental data. From the experimental results, it shows that the best performance of CLDRL achieves 78.59%, which prove the effectiveness of the proposed model.

Key words: cross language, deep representation learning, transfer learning, sentiment analysis

CLC Number: 

  • TP391
[1] 余传明. 从产品评论中挖掘观点:原理与算法分析[J]. 情报理论与实践, 2009, 32(7):124-128. YU Chuanming. Mining opinions from product review: principles and algorithm analysis[J]. Information Studies: Theory & Application, 2009, 32(7):124-128.
[2] 余传明, 安璐. 从小数据到大数据——观点检索面临的三个挑战[J]. 情报理论与实践, 2016, 39(2):13-19. YU Chuanming, AN Lu. From small data to big data——the three challenges of opinion retrieval[J]. Information Studies: Theory & Application, 2016, 39(2):13-19.
[3] BLITZER J, DREDZE M, PEREIRA F. Domain adaptation for sentiment classification[EB/OL].[2017-02-16].http://www.cs.jhu.edu/~mdredze/publications/sentiment_acl07.pdf
[4] PAN Jialin, YANG Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge & Data Engineering, 2010, 22(10):1345-1359.
[5] PAN Weike, ZHONG Erheng, YANG qiang. Transfer learning for text mining[M] // Aggarwal C, Zhai C. Mining Text Data. Boston: Springer, 2012: 223-257.
[6] RIGUTINI L, MAGGINI M, LIU Bing. An EM based training algorithm for cross-language text categorization[C] // Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence. Washington: IEEE Computer Society, 2005: 529-535.
[7] WAN Xiaojun. Co-training for cross-lingual sentiment classification[C] // Processing of the AFNLP. Stroudsburg: Association for Computational Linguistics, 2009: 235-243.
[8] DUH K, FUJINO A, NAGATA M. Is machine translation ripe for cross-lingual sentiment classification?[C] // Proceedings of Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2011: 429-433.
[9] PRETTENHOFER P, STEIN B. Cross-lingual adaptation using structural correspondence learning[J]. ACM Transactions on Intelligent Systems & Technology, 2011, 3(1):1-13.
[10] FERNÁNDEZ A M, ESULI A, SEBASTIANI F. Distributional correspondence indexing for cross-lingual and cross-domain sentiment classification[J]. Journal of Articial Intelligence Research, 2015(55):131-163.
[11] DUMAIS S T, LETSCHE T A, LITTMAN M L, et al. Automatic cross-language retrieval using latent semantic indexing[J]. AAAI Symposium on Crosslanguage Text & Speech Retrieval, 1997, 1:51-62.
[12] DEERWESTER S. Indexing by latent semantic analysis[J]. Journal of the Association for Information Science and Technology, 1990, 41(6):391-407.
[13] XIAO Min, GUO Yuhong. A novel two-step method for cross language representation learning[J]. Advances in Neural Information Processing Systems, 2013: 1259-1267.
[14] VULIC I, SMET W D, TANG Jie, et al. Probabilistic topic modeling in multilingual settings: an overview of its methodology and applications[J]. Information Processing & Management, 2015, 51(1):111-147.
[15] GLOROT X, BORDES A, BENGIO Y. Domain adaptation for large-scale sentiment classification: a deep learning approach[C] // Proceedings of the 28th International Conference on International Conference on Machine Learning. Bellevue: Omnipress, 2011: 513-520.
[16] ZHOU Guangyou, ZHU Zhiyuan, HE Tingting, et al. Cross-lingual sentiment classification with stacked autoencoders[J]. Knowledge and Information Systems, 2016, 47(1):27-44.
[17] ZHOU Guangyou, ZENG Zhao, HUANG Xiangji, et al. Transfer learning for cross-lingual sentiment classification with weakly shared deep neural networks[C] // Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2016: 245-254.
[18] KIM Y. Convolutional neural networks for sentence classification[EB/OL].[2017-01-02].arXiv:1408.5882.
[19] KALCHBRENNER N, GREFENSTETTE E, BLUNSOMP. A convolutional neural network for modelling sentences[EB/OL].[2017-01-05]. arXiv: 1404.2188.
[20] COLLOBERT R, WESTON J. A unified architecture for natural language processing: deep neural networks with multitask learning[C] // Proceedings of the 25th International Conference on Machine Learning. New York: ACM, 2008: 160-167.
[21] GAO Jianfeng, PANTEL P, GAMON M, et al. Modeling interestingness with deep neural networks[C] // Proceedings of Annual Conference on Empirical Methods in Natural Language Processing.[S.l.] :[s.n.] ,2014:2-13.
[22] YAN Chao, ZHANG Bailing, COENEN F. Driving posture recognition by convolutional neural networks[C] // Proceedings of the International Conference on Natural Computation. New York: IEEE, 2015: 680-685.
[23] NGIAM J, KOH P, CHEN Zhenghao, et al. Sparse filtering[C] // Proceedings of the 24th International Conference on Neural Information Processing Systems. Granada: Curran Associates Inc, 2011: 1125-1133.
[24] DAHL G E, RANZATO M, MOHAMED A R, et al. Phone recognition with the mean-covariance restricted boltzmann machine[C] // Proceedings of the 23rd International Conference on Neural Information Processing Systems. Granada: Curran Associates Inc, 2010: 469-477.
[25] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[26] BOSER B E, GUYON I M, VAPNIK V N. A training algorithm for optimal margin classifiers[C] // Proceedings of Annual ACM Workshop on Computational Learning Theory. New York: ACM, 1996, 5:144-152.
[27] TANG Yichuan. Deep learning using linear support vector machines[EB/OL].[2017-02-05]. arXiv:1306.0239v4.
[28] DUCHI J, HAZAN E, SINGER Y. Adaptive subgradient methods for online learning and stochastic optimization[J]. Journal of Machine Learning Research, 2011, 12(7):2121-2159.
[29] GLOROT X, BENGIO Y. Understanding the difficulty of training deep feedforward neural networks[J]. Journal of Machine Learning Research, 2010, 9:249-256.
[30] ZHOW Huiwei, CHEN Long, SHI Fulin, et al. Learning bilingual sentiment word embeddings for cross-language sentiment classification[C] // Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing.[S.l.] :[s.n.] , 2015:430-441.
[31] ZHOU Xinjie, WAN Xiaojun, XIAO Jianguo. Cross-lingual sentiment classification with bilingual document representation learning[C] // Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.[S.l.] :[s.n.] , 2016:1403-1412.
[32] GIATSOGLOU M, VOZALIS M G, DIAMANTARAS K, et al. Sentiment analysis leveraging emotions and word embeddings[J]. Expert Systems with Applications, 2017, 69:214-224
[33] RUDER S, GHAFFARI P, BRESLIN J G. NSIGHT-1 at SemEval-2016 Task 5: deep learning for multilingual aspect-based sentiment analysis[C] // International Workshop on Semantic Evaluation.[S.l.] :[s.n.] , 2016: 330-336.
[34] CHEN Xilun, SUN Yu, ATHIWARATKUN B. Adversarial deep averaging networks for cross-lingual sentiment classification[EB/OL].[2016-12-08].arXiv: 1606.01614.
[1] CHEN Xin, XUE Yun, LU Xin, LI Wan-li, ZHAO Hong-ya, HU Xiao-hui. Text feature extraction method for sentiment analysis based on order-preserving submatrix and frequent sequential pattern mining [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2018, 53(3): 36-45.
[2] SUN Shi-chang, LIN Hong-fei, MENG Jia-na, LIU Hong-bo. Model selection with likelihood ratio for sequence transfer learning [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2017, 52(6): 24-31.
[3] WANG Tong, MA Yan-zhou, YI Mian-zhu. Speech recognition of Russian short instructions based on DTW [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2017, 52(11): 29-36.
[4] HE Yan-xiang, LIU Jian-bo, SUN Song-tao, WEN Wei-dong. Product reviews sentiment classification in Micro-blog based on cascaded conditional random field [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2015, 50(11): 67-73.
[5] ZHU Zhu, LI Shou-shan, DAI Min, ZHOU Guo-dong. Opinion target extraction with active-learning and automatic annotation [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2015, 50(07): 38-44.
[6] ZHOU Wen, ZHANG Shu-qing, OUYANG Chun-ping, LIU Zhi-ming, YANG Xiao-hua. Topic sentiment analysis of Chinese news based on emotional dependency tuple [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(12): 1-6.
[7] YANG Jia-neng, YANG Ai-min, ZHOU Yong-mei. Sentiment classification method of Chinese Micro-blog based on semantic analysis [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 14-21.
[8] ZHU Xi, DONG Xi-shuang, GUAN Yi, LIU Zhi-guang. Sentiment analysis of Chinese Micro-blog based on semi-supervised learning [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 37-42.
[9] LIU Ming, ZAN Hong-ying, YUAN Hui-bin. Key sentiment sentence prediction using SVM and RNN [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 68-73.
[10] SUN Song-tao, HE Yan-xiang, CAI Rui, LI Fei, HE Fei-yan. Comparative study of methods for Micro-blog sentiment evaluation tasks [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 43-50.
[11] ZHANG Cheng-gong 1, 2, LIU Pei-yu1, 2*, ZHU Zhen-fang1,2, FANG Ming1,2. A sentiment analysis method based on a polarity lexicon [J]. J4, 2012, 47(3): 47-50.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE)
Supported by Beijing Magtech Co.Ltd