您的位置:山东大学 -> 科技期刊社 -> 《山东大学学报(理学版)》

《山东大学学报(理学版)》 ›› 2021, Vol. 56 ›› Issue (5): 57-65.doi: 10.6040/j.issn.1671-9352.1.2020.060

• • 上一篇    

基于半监督图神经网络的短文本分类

张斌艳,朱小飞*,肖朝晖,黄贤英,吴洁   

  1. 重庆理工大学 计算机科学与工程学院, 重庆 400054
  • 发布日期:2021-05-13
  • 作者简介:张斌艳(1996— ),女,硕士研究生,研究方向为机器学习和自然语言处理. E-mail: 1576579348@qq.com*通信作者简介:朱小飞(1979— ),男,博士,教授,研究方向为机器学习、信息检索和自然语言处理. E-mail: zxf@cqut.edu.cn
  • 基金资助:
    国家自然科学基金资助项目(61702063,61502065);重庆市基础科学与前沿技术研究项目(cstc2017jcyjBX0059,cstc2017jcyjAX0339);重庆市教委语言文字科研项目重点项目(yyk20103)

Short text classification based on semi-supervised graph neural network

ZHANG Bin-yan, ZHU Xiao-fei*, XIAO Zhao-hui, HUANG Xian-ying, WU Jie   

  1. College of Computer Science and Engineering, Chongqing University of Technology, Chongqing 400054, China
  • Published:2021-05-13

摘要: 文中提出了在短文本建模过程中引入词项与词项之间、词项与文档之间的全局结构关系来增强短文本的表示。由于有标签训练数据的缺乏,使得现有的全局结构关系建模方法,如TextGCN,无法学习到高质量的词项和文档全局结构表示,因此,文中进一步提出采用半监督学习思想来解决有标签训练数据不足的问题。实验结果表明,在基准数据集MEDUI上,与现有相关模型进行对比,文中提出的方法比最好的基准模型在F1指标上提高了1.91%。

关键词: 图神经网络, 半监督学习, 短文本分类

Abstract: This paper proposes to introduce the global structural relationship between terms and terms and between terms and documents in the process of short text modeling to enhance the representation of short text. Due to the lack of labeled training data, existing global structural relationship modeling methods, such as TextGCN, cannot learn high-quality terms and document global structure representations. Therefore, we further propose to adopt the idea of semi-supervised learning to solve the problem of insufficient training data. On the benchmark dataset MEDUI, we compare with the existing related models. The experimental results show that the method proposed in this paper improves the F1 index by 1.91% compared with the best benchmark model.

Key words: graph neural network, semi-supervised learning, short text classification

中图分类号: 

  • TP391
[1] WANG Fang, WANG Zhongyuan, LI Zhoujun, et al. Concept-based short text classification and ranking[C] // Proceedings of the 23rd ACM International Conference on Information and Knowledge Management. New York: ACM, 2014: 1069-1078.
[2] LEE Jiyoung, DERNONCOURT F. Sequential short-text classification with recurrent and convolutional neural networks[C] // Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego: ACL, 2016: 515-520.
[3] CAI Hongyun, ZHENG V W, CHANG Kevin Chen-chuan. A comprehensive survey of graph embedding: problems, techniques and applications[J]. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(9):1616-1637.
[4] ZHANG Dongwen, XU Hua, SU Zengcai, et al. Chinese comments sentiment classification based on word2vec and SVMperf[J]. Expert Systems with Applications, 2015, 42(4):1857-1863.
[5] YANG Zichao, YANG Diyi, DYER Chris, et al. Hierarchical attention networks for document classification[C] //Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego: ACL, 2016: 1480-1489.
[6] KIM Yoon. Convolutional neural networks for sentence classification[J]. arXiv, 2014: 1746-1751. https://arxiv.org/abs/1408.5882.
[7] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[8] BATTAGLIA P W, HAMRICK J B, BAPST V, et al. Relational inductive biases, deep learning, and graph networks[J]. arXiv, 2018. https://arxiv.org/pdf/1806.01261.pdf.
[9] PENG Hao, LI Jianxin, HE Yu, et al. Large-scale hierarchical text classification with recursively regularized deep graph-cnn[C] // Proceedings of the 2018 World Wide Web Conference. Lyon: WWW, 2018: 1063-1072.
[10] YAO Liang, MAO Chensheng, LUO Yuan. Graph convolutional networks for text classification[R/OL]. AAAI, 2019: 7370-7377. https://arxiv.org/abs/1809.05679.
[11] WANG Sida, MANNING Christopher D. Baselines and bigrams: simple, good sentiment and topic classification[C] // Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. Jeju: ACL, 2012: 90-94.
[12] LAZARIDOU A, TITOV I, SPORLEDER C. A bayesian model for joint unsupervised induction of sentiment, aspect and discourse representations[C] //Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Sofia: ACL, 2013: 1630-1639.
[13] POST Matt, BERGSMA Shane. Explicit and implicit syntactic features for text classification[C] //Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Sofia: ACL, 2013: 866-872.
[14] LAI Siwei, XU Liheng, LIU Kang, et al. Recurrent convolutional neural networks for text classification[C] //Proceedings of the 29th AAAI Conference on Artificial Intelligence.[S.L] :[s.n.] , 2015: 2267-2273.
[15] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[R/OL]. 2017. https://arxiv.org/pdf/1609.02907v4.pdf
[16] MARCHEGGIANI D, TITOV I. Encoding sentences with graph convolutional networks for semantic role labeling[C] // Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.[S.L] : [s.n.] , 2017: 1506-1515.
[17] LI Yifu, JIN Ran, LUO Yuan. Classifying relations in clinical narratives using segment graph convolutional and recurrent neural networks(Seg-Gcrns)[J]. Journal of the American Medical Informatics Association, 2019, 26(3):262-268.
[18] BASTINGS J, TITOV I, AZIZ W, et al. Graph convolutional encoders for syntax-aware neural machine translation[C] // Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen: ACL, 2017: 1957-1967.
[19] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. Computer Science, 2013. https://arxiv.org/pdf/1301.3781v3.pdf.
[20] HAO Yanchao, ZHANG Yuanzhe, LIU Kang, et al. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge[C] // Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Vancouver:ACL, 2017: 221-231.
[21] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] //Proceedings of the 31st Conference on Neural Information Processing Systems(NIPS 2017). Long Beach:[s.n.] , 2017.
[22] 吴洁, 朱小飞, 张宜浩, 等. 基于用户情感倾向感知的微博情感分析方法[J].山东大学学报(理学版), 2019, 54(3):46-55. WU Jie, ZHU Xiaofei, ZHANG Yihao, et al. Microblog sentiment analysis method based on users emotional orientation perception[J]. Journal of Shandong University(Natural Science), 2019, 54(3):46-55.
[23] KINGMA D, BA J. Adam: a method for stochastic optimization[R/OL]. 2017. https://arxiv.org/pdf/1412.6980.pdf.
[24] LILLEBERG Joseph, ZHU Yun, ZHANG Yanqing. Support vector machines and word2vec for text classification with senmantic features[C] // Proceedings of the 2015 IEEE 14th International Conference on Cognitive Informatics & Cognitive Computing. Beijing: IEEE, 2015: 136-140.
[25] DAI Yuanfei, GUO Wenzhong, CHEN Xing, et al. Relation classification via LSTMs based on sequence and tree structure[J]. IEEE Access, 2018, 6:64927-64937.
[26] YANG Xiaoyilei, XU Shuaijing, WU Hao, et al. Sentiment analysis of weibo comment texts based on extended vocabulary and convolutional neural network[J]. Procedia Computer Science, 2019, 147:361-368.
[27] WU Chuhan, WU Fangzhao, AN Mingxiao, et al. NPA: neural news recommendation with personalized attention[C] //Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.[S.l.] : ACM, 2019: 2576-2584.
[1] 杨婷,朱恒东,马盈仓,汪义瑞,杨小飞. 基于L2,1范数和流形正则项的半监督谱聚类算法[J]. 《山东大学学报(理学版)》, 2021, 56(3): 67-76.
[2] 张鹏,王素格,李德玉,王杰. 一种基于启发式规则的半监督垃圾评论分类方法[J]. 山东大学学报(理学版), 2017, 52(7): 44-51.
[3] 苏丰龙,谢庆华,黄清泉,邱继远,岳振军. 基于直推式学习的半监督属性抽取[J]. 山东大学学报(理学版), 2016, 51(3): 111-115.
[4] 杜红乐,张燕,张林. 不均衡数据集下的入侵检测[J]. 山东大学学报(理学版), 2016, 51(11): 50-57.
[5] 马成龙, 姜亚松, 李艳玲, 张艳, 颜永红. 基于词矢量相似度的短文本分类[J]. 山东大学学报(理学版), 2014, 49(12): 18-22.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!