您的位置：山东大学 -> 科技期刊社 -> 《山东大学学报（理学版）》

《山东大学学报(理学版)》 ›› 2023, Vol. 58 ›› Issue (1): 48-58.doi: 10.6040/j.issn.1671-9352.2.2021.035

• • 上一篇    

基于条件变分自编码器的问题生成方法

刘东,洪宇*,苏玉兰,张民   

  1. 苏州大学计算机科学与技术学院, 江苏 苏州 215006
  • 发布日期:2023-02-12
  • 作者简介:刘东(1997— ),男,硕士研究生,研究方向为生成式问答.E-mail:liudong.young1127@gmail.com*通信作者简介:洪宇(1978— ),男,教授,研究方向为自然语言处理.E-mail:tianxianer@gmail.com
  • 基金资助:
    国家重点研发计划资助项目(2020YFB1313601);国家自然科学基金资助项目(62076174)

Question generation method based on conditional variational autoencoder

LIU Dong, HONG Yu*, SU Yu-lan, ZHANG Min   

  1. School of Computer Science and Technology, Soochow University, Suzhou 215006, Jiangsu, China
  • Published:2023-02-12

PDF (PC)

146

摘要： 将条件变分自编码器作为辅助模块,引入预训练语言模型的编码解码过程,通过数据增强(潜在的语义扩充)以提高模型的鲁棒性。通过建立陈述句与疑问句之间的高维分布联系,由分布采样实现一对多的问题生成。结果表明,融合条件变分自编码器不仅能生成多样性的问题,也有助于提升问题生成的模型性能。在基于SQuAD数据集划分的2个答案可知问题生成数据集Split1和Split2上,BLEU-4值分别被提升到20.75%和21.61%。

关键词: 条件变分自编码器, 问题生成, 预训练语言模型

Abstract: The conditional variational autoencoder, as an auxiliary module, is introduced into the encoding and decoding process of the pre-trained language model. It improves the robustness of the model through data augmentation(potential semantic expansion), and establishes a high-dimensional distribution connection between declarative sentences and interrogative sentences, which implements one-to-many question generation by sampling from the distribution. The results show that the fusion of conditional variational autoencoder can not only generate diverse questions, but also help to improve the performance of the question generation model. On the two answer-known question generation datasets Split1 and Split2, which are based on the SQuAD dataset, the BLEU-4 score is improved to 20.75% and 21.61%, respectively.

Key words: conditional variational autoencoder, question generation, pre-trained language model

中图分类号: 

  • TP391
[1] PAN L, LEI W, CHUA T S, et al. Recent advances in neural question generation[J/OL]. arXiv, 2019. https://arxiv.org/abs/1905.08949v3.
[2] QIU X, SUN T, XU Y, et al. Pre-trained models for natural language processing: a survey[J]. Science China Technological Sciences, 2020, 63(10):1-26.
[3] BAO H, DONG L, WEI F, et al. Unilmv2: pseudo-masked language models for unified language model pre-training[C] //International Conference on Machine Learning. New York: PMLR, 2020: 642-652.
[4] WANG T, WAN X. T-CVAE: transformer-based conditioned variational autoencoder for story completion[C] //International Joint Conferences on Artificial Intelligence.[S.l.] : IJCAI, 2019: 5233-5239.
[5] SERBAN I, SORDONI A, LOWE R, et al. A hierarchical latent variable encoder-decoder model for generating dialogues[C] //Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2017.
[6] RAJPURKAR P, ZHANG J, LOPYREVK, et al. Squad: 100,000+ questions for machine comprehension of text[J/OL]. arXiv, 2016. https://arxiv.org/abs/1606.05250v2.
[7] DU X, SHAO J, CARDIEC. Learning to ask: neural question generation for reading comprehension[J/OL]. arXiv, 2017. https://arxiv.org/abs/1705.00106v1.
[8] ZHOU Q, YANG N, WEI F, et al. Neural question generation from text: a preliminary study[C] //National CCF Conference on Natural Language Processing and Chinese Computing. Cham: Springer, 2017: 662-671.
[9] BOWMAN S R, VILNIS L, VINYALS O, et al. Generating sentences from a continuous space[J/OL]. arXiv, 2015. https://arxiv.org/abs/1511.06349v3.
[10] SUN X, LIU J, LYU Y, et al. Answer-focused and position-aware neural question generation[C] //Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2018: 3930-3939.
[11] GU J, LU Z, LI H, et al. Incorporating copying mechanism in sequence-to-sequence learning[J/OL]. arXiv, 2016. https://arxiv.org/abs/1603.06393v3.
[12] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] //Advances in Neural Information Processing Systems. [S.l.] :Neural information processing systems foundation, 2017: 5998-6008.
[13] CHAN Y H, FAN Y C. A recurrent BERT-based model for question generation[C] //Proceedings of the 2nd Workshop on Machine Reading for Question Answering. Stroudsburg:Association for Computational Linguistics, 2019: 154-162.
[14] VARANASI S, AMIN S, NEUMANN G.CopyBERT: a unified approach to question generation with self-attention[C] //Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI. Stroudsburg:Association for Computational Linguistics, 2020: 25-31.
[15] DONG L, YANG N, WANG W, et al. Unified language model pre-training for natural language understanding and generation[J/OL]. arXiv, 2019. https://arxiv.org/abs/1905.03197.
[16] 黄民烈. 现代自然语言生成[M]. 北京:电子工业出版社,2021: 98-100. HUANG Minlie. Modern natural language generation[M]. Beijing:Publishing House of Electronics Industry of China, 2021: 98-100.
[17] SHARMA S, ASRI L E, SCHULZH, et al. Relevance of unsupervised metrics in task-oriented dialogue for evaluating natural language generation[J/OL]. arXiv, 2017. https://arxiv.org/abs/1706.09799.
[18] PAPINENI K, ROUKOS S, WARD T, et al. Bleu: a method for automatic evaluation of machine translation[C] //Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2002: 311-318.
[19] DENKOWSKI M, LAVIEA. Meteor universal: language specific translation evaluation for any target language[C] //Proceedings of the Ninth Workshop on Statistical Machine Translation. Stroudsburg:Association for Computational Linguistics, 2014: 376-380.
[20] LIN C Y. Rouge: a package for automatic evaluation of summaries[C] //Proceedings of Workshop on Text Summarization of ACL. Stroudsburg:Association for Computational Linguistics, 2004.
[21] GU X D, CHO K H, HA J W, et al. Dialogwae: multimodal response generation with conditional wasserstein auto-encoder[J/OL]. arXiv, 2018. https://arxiv.org/pdf/1805.12352.pdf.
[22] ZHAO Y, NI X, DING Y, et al. Paragraph-level neural question generation with maxout pointer and gated self-attention networks[C] //Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2018: 3901-3910.
[1] 谭金源,刁宇峰,杨亮,祁瑞华,林鸿飞. 基于BERT-SUMOPN模型的抽取-生成式文本自动摘要[J]. 《山东大学学报(理学版)》, 2021, 56(7): 82-90.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2018 《山东大学学报(理学版)》编辑部　
地址: 山东省济南市山大南路27号山东大学中心校区（250100）　电话: +86-0531-88366917　E-mail: xblxb@sdu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发