JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE) ›› 2021, Vol. 56 ›› Issue (5): 57-65.doi: 10.6040/j.issn.1671-9352.1.2020.060

Previous Articles    

Short text classification based on semi-supervised graph neural network

ZHANG Bin-yan, ZHU Xiao-fei*, XIAO Zhao-hui, HUANG Xian-ying, WU Jie   

  1. College of Computer Science and Engineering, Chongqing University of Technology, Chongqing 400054, China
  • Published:2021-05-13

PDF (PC)

430

Abstract: This paper proposes to introduce the global structural relationship between terms and terms and between terms and documents in the process of short text modeling to enhance the representation of short text. Due to the lack of labeled training data, existing global structural relationship modeling methods, such as TextGCN, cannot learn high-quality terms and document global structure representations. Therefore, we further propose to adopt the idea of semi-supervised learning to solve the problem of insufficient training data. On the benchmark dataset MEDUI, we compare with the existing related models. The experimental results show that the method proposed in this paper improves the F1 index by 1.91% compared with the best benchmark model.

Key words: graph neural network, semi-supervised learning, short text classification

CLC Number: 

  • TP391
[1] WANG Fang, WANG Zhongyuan, LI Zhoujun, et al. Concept-based short text classification and ranking[C] // Proceedings of the 23rd ACM International Conference on Information and Knowledge Management. New York: ACM, 2014: 1069-1078.
[2] LEE Jiyoung, DERNONCOURT F. Sequential short-text classification with recurrent and convolutional neural networks[C] // Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego: ACL, 2016: 515-520.
[3] CAI Hongyun, ZHENG V W, CHANG Kevin Chen-chuan. A comprehensive survey of graph embedding: problems, techniques and applications[J]. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(9):1616-1637.
[4] ZHANG Dongwen, XU Hua, SU Zengcai, et al. Chinese comments sentiment classification based on word2vec and SVMperf[J]. Expert Systems with Applications, 2015, 42(4):1857-1863.
[5] YANG Zichao, YANG Diyi, DYER Chris, et al. Hierarchical attention networks for document classification[C] //Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego: ACL, 2016: 1480-1489.
[6] KIM Yoon. Convolutional neural networks for sentence classification[J]. arXiv, 2014: 1746-1751. https://arxiv.org/abs/1408.5882.
[7] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[8] BATTAGLIA P W, HAMRICK J B, BAPST V, et al. Relational inductive biases, deep learning, and graph networks[J]. arXiv, 2018. https://arxiv.org/pdf/1806.01261.pdf.
[9] PENG Hao, LI Jianxin, HE Yu, et al. Large-scale hierarchical text classification with recursively regularized deep graph-cnn[C] // Proceedings of the 2018 World Wide Web Conference. Lyon: WWW, 2018: 1063-1072.
[10] YAO Liang, MAO Chensheng, LUO Yuan. Graph convolutional networks for text classification[R/OL]. AAAI, 2019: 7370-7377. https://arxiv.org/abs/1809.05679.
[11] WANG Sida, MANNING Christopher D. Baselines and bigrams: simple, good sentiment and topic classification[C] // Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. Jeju: ACL, 2012: 90-94.
[12] LAZARIDOU A, TITOV I, SPORLEDER C. A bayesian model for joint unsupervised induction of sentiment, aspect and discourse representations[C] //Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Sofia: ACL, 2013: 1630-1639.
[13] POST Matt, BERGSMA Shane. Explicit and implicit syntactic features for text classification[C] //Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Sofia: ACL, 2013: 866-872.
[14] LAI Siwei, XU Liheng, LIU Kang, et al. Recurrent convolutional neural networks for text classification[C] //Proceedings of the 29th AAAI Conference on Artificial Intelligence.[S.L] :[s.n.] , 2015: 2267-2273.
[15] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[R/OL]. 2017. https://arxiv.org/pdf/1609.02907v4.pdf
[16] MARCHEGGIANI D, TITOV I. Encoding sentences with graph convolutional networks for semantic role labeling[C] // Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.[S.L] : [s.n.] , 2017: 1506-1515.
[17] LI Yifu, JIN Ran, LUO Yuan. Classifying relations in clinical narratives using segment graph convolutional and recurrent neural networks(Seg-Gcrns)[J]. Journal of the American Medical Informatics Association, 2019, 26(3):262-268.
[18] BASTINGS J, TITOV I, AZIZ W, et al. Graph convolutional encoders for syntax-aware neural machine translation[C] // Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen: ACL, 2017: 1957-1967.
[19] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. Computer Science, 2013. https://arxiv.org/pdf/1301.3781v3.pdf.
[20] HAO Yanchao, ZHANG Yuanzhe, LIU Kang, et al. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge[C] // Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Vancouver:ACL, 2017: 221-231.
[21] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] //Proceedings of the 31st Conference on Neural Information Processing Systems(NIPS 2017). Long Beach:[s.n.] , 2017.
[22] 吴洁, 朱小飞, 张宜浩, 等. 基于用户情感倾向感知的微博情感分析方法[J].山东大学学报(理学版), 2019, 54(3):46-55. WU Jie, ZHU Xiaofei, ZHANG Yihao, et al. Microblog sentiment analysis method based on users emotional orientation perception[J]. Journal of Shandong University(Natural Science), 2019, 54(3):46-55.
[23] KINGMA D, BA J. Adam: a method for stochastic optimization[R/OL]. 2017. https://arxiv.org/pdf/1412.6980.pdf.
[24] LILLEBERG Joseph, ZHU Yun, ZHANG Yanqing. Support vector machines and word2vec for text classification with senmantic features[C] // Proceedings of the 2015 IEEE 14th International Conference on Cognitive Informatics & Cognitive Computing. Beijing: IEEE, 2015: 136-140.
[25] DAI Yuanfei, GUO Wenzhong, CHEN Xing, et al. Relation classification via LSTMs based on sequence and tree structure[J]. IEEE Access, 2018, 6:64927-64937.
[26] YANG Xiaoyilei, XU Shuaijing, WU Hao, et al. Sentiment analysis of weibo comment texts based on extended vocabulary and convolutional neural network[J]. Procedia Computer Science, 2019, 147:361-368.
[27] WU Chuhan, WU Fangzhao, AN Mingxiao, et al. NPA: neural news recommendation with personalized attention[C] //Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.[S.l.] : ACM, 2019: 2576-2584.
[1] YANG Ting, ZHU Heng-dong, MA Ying-cang, WANG Yi-rui, YANG Xiao-fei. Semi-supervised spectral clustering algorithm based on L2,1 norm and manifold regularization terms [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2021, 56(3): 67-76.
[2] ZHANG Peng, WANG Su-ge, LI De-yu, WANG Jie. A semi-supervised spam review classification method based on heuristic rules [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2017, 52(7): 44-51.
[3] SU Feng-long, XIE Qing-hua, HUANG Qing-quan, QIU Ji-yuan, YUE Zhen-jun. Semi-supervised method for attribute extraction based on transductive learning [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(3): 111-115.
[4] DU Hong-le, ZHANG Yan, ZHANG Lin. Intrusion detection on imbalanced dataset [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2016, 51(11): 50-57.
[5] MA Cheng-long, JIANG Ya-song, LI Yan-ling, ZHANG Yan, YAN Yong-hong. Short text classification based on word embedding similarity [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(12): 18-22.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE)
Supported by Beijing Magtech Co.Ltd