基于半监督图神经网络的短文本分类

doi:10.6040/j.issn.1671-9352.1.2020.060

摘要/Abstract

摘要： 文中提出了在短文本建模过程中引入词项与词项之间、词项与文档之间的全局结构关系来增强短文本的表示。由于有标签训练数据的缺乏,使得现有的全局结构关系建模方法,如TextGCN,无法学习到高质量的词项和文档全局结构表示,因此,文中进一步提出采用半监督学习思想来解决有标签训练数据不足的问题。实验结果表明,在基准数据集MEDUI上,与现有相关模型进行对比,文中提出的方法比最好的基准模型在F1指标上提高了1.91%。

关键词: 图神经网络, 半监督学习, 短文本分类

Abstract: This paper proposes to introduce the global structural relationship between terms and terms and between terms and documents in the process of short text modeling to enhance the representation of short text. Due to the lack of labeled training data, existing global structural relationship modeling methods, such as TextGCN, cannot learn high-quality terms and document global structure representations. Therefore, we further propose to adopt the idea of semi-supervised learning to solve the problem of insufficient training data. On the benchmark dataset MEDUI, we compare with the existing related models. The experimental results show that the method proposed in this paper improves the F₁ index by 1.91% compared with the best benchmark model.

Key words: graph neural network, semi-supervised learning, short text classification

中图分类号:

TP391

张斌艳,朱小飞,肖朝晖,黄贤英,吴洁. 基于半监督图神经网络的短文本分类[J]. 《山东大学学报(理学版)》, 2021, 56(5): 57-65.

ZHANG Bin-yan, ZHU Xiao-fei, XIAO Zhao-hui, HUANG Xian-ying, WU Jie. Short text classification based on semi-supervised graph neural network[J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2021, 56(5): 57-65.

参考文献

[1] WANG Fang, WANG Zhongyuan, LI Zhoujun, et al. Concept-based short text classification and ranking[C] // Proceedings of the 23rd ACM International Conference on Information and Knowledge Management. New York: ACM, 2014: 1069-1078.
[2] LEE Jiyoung, DERNONCOURT F. Sequential short-text classification with recurrent and convolutional neural networks[C] // Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego: ACL, 2016: 515-520.
[3] CAI Hongyun, ZHENG V W, CHANG Kevin Chen-chuan. A comprehensive survey of graph embedding: problems, techniques and applications[J]. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(9):1616-1637.
[4] ZHANG Dongwen, XU Hua, SU Zengcai, et al. Chinese comments sentiment classification based on word2vec and SVMperf[J]. Expert Systems with Applications, 2015, 42(4):1857-1863.
[5] YANG Zichao, YANG Diyi, DYER Chris, et al. Hierarchical attention networks for document classification[C] //Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego: ACL, 2016: 1480-1489.
[6] KIM Yoon. Convolutional neural networks for sentence classification[J]. arXiv, 2014: 1746-1751. https://arxiv.org/abs/1408.5882.
[7] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[8] BATTAGLIA P W, HAMRICK J B, BAPST V, et al. Relational inductive biases, deep learning, and graph networks[J]. arXiv, 2018. https://arxiv.org/pdf/1806.01261.pdf.
[9] PENG Hao, LI Jianxin, HE Yu, et al. Large-scale hierarchical text classification with recursively regularized deep graph-cnn[C] // Proceedings of the 2018 World Wide Web Conference. Lyon: WWW, 2018: 1063-1072.
[10] YAO Liang, MAO Chensheng, LUO Yuan. Graph convolutional networks for text classification[R/OL]. AAAI, 2019: 7370-7377. https://arxiv.org/abs/1809.05679.
[11] WANG Sida, MANNING Christopher D. Baselines and bigrams: simple, good sentiment and topic classification[C] // Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. Jeju: ACL, 2012: 90-94.
[12] LAZARIDOU A, TITOV I, SPORLEDER C. A bayesian model for joint unsupervised induction of sentiment, aspect and discourse representations[C] //Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Sofia: ACL, 2013: 1630-1639.
[13] POST Matt, BERGSMA Shane. Explicit and implicit syntactic features for text classification[C] //Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Sofia: ACL, 2013: 866-872.
[14] LAI Siwei, XU Liheng, LIU Kang, et al. Recurrent convolutional neural networks for text classification[C] //Proceedings of the 29th AAAI Conference on Artificial Intelligence.[S.L] :[s.n.] , 2015: 2267-2273.
[15] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[R/OL]. 2017. https://arxiv.org/pdf/1609.02907v4.pdf
[16] MARCHEGGIANI D, TITOV I. Encoding sentences with graph convolutional networks for semantic role labeling[C] // Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.[S.L] : [s.n.] , 2017: 1506-1515.
[17] LI Yifu, JIN Ran, LUO Yuan. Classifying relations in clinical narratives using segment graph convolutional and recurrent neural networks(Seg-Gcrns)[J]. Journal of the American Medical Informatics Association, 2019, 26(3):262-268.
[18] BASTINGS J, TITOV I, AZIZ W, et al. Graph convolutional encoders for syntax-aware neural machine translation[C] // Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen: ACL, 2017: 1957-1967.
[19] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. Computer Science, 2013. https://arxiv.org/pdf/1301.3781v3.pdf.
[20] HAO Yanchao, ZHANG Yuanzhe, LIU Kang, et al. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge[C] // Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Vancouver:ACL, 2017: 221-231.
[21] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] //Proceedings of the 31st Conference on Neural Information Processing Systems(NIPS 2017). Long Beach:[s.n.] , 2017.
[22] 吴洁, 朱小飞, 张宜浩, 等. 基于用户情感倾向感知的微博情感分析方法[J].山东大学学报(理学版), 2019, 54(3):46-55. WU Jie, ZHU Xiaofei, ZHANG Yihao, et al. Microblog sentiment analysis method based on users emotional orientation perception[J]. Journal of Shandong University(Natural Science), 2019, 54(3):46-55.
[23] KINGMA D, BA J. Adam: a method for stochastic optimization[R/OL]. 2017. https://arxiv.org/pdf/1412.6980.pdf.
[24] LILLEBERG Joseph, ZHU Yun, ZHANG Yanqing. Support vector machines and word2vec for text classification with senmantic features[C] // Proceedings of the 2015 IEEE 14th International Conference on Cognitive Informatics & Cognitive Computing. Beijing: IEEE, 2015: 136-140.
[25] DAI Yuanfei, GUO Wenzhong, CHEN Xing, et al. Relation classification via LSTMs based on sequence and tree structure[J]. IEEE Access, 2018, 6:64927-64937.
[26] YANG Xiaoyilei, XU Shuaijing, WU Hao, et al. Sentiment analysis of weibo comment texts based on extended vocabulary and convolutional neural network[J]. Procedia Computer Science, 2019, 147:361-368.
[27] WU Chuhan, WU Fangzhao, AN Mingxiao, et al. NPA: neural news recommendation with personalized attention[C] //Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.[S.l.] : ACM, 2019: 2576-2584.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed