您的位置：山东大学 -> 科技期刊社 -> 《山东大学学报（理学版）》

《山东大学学报(理学版)》 ›› 2021, Vol. 56 ›› Issue (3): 44-53.doi: 10.6040/j.issn.1671-9352.0.2020.346

• • 上一篇    

基于双重多路注意力匹配的观点型阅读理解

鲍亮1,2,3,陈志豪1,2,3,陈文章1,2,3,叶锴1,2,3,廖祥文1,2,3*   

  1. 1.福州大学数学与计算机科学学院, 福建 福州 350116;2.福州大学福建省网络计算与智能信息处理重点实验室, 福建 福州 350116;3.数字福建金融大数据研究所, 福建 福州 350116
  • 发布日期:2021-03-16
  • 作者简介:鲍亮(1996— ),男,硕士研究生,研究方向为机器阅读理解. E-mail:n180320044@fzu.edu.cn*通信作者简介:廖祥文(1980— ),男,博士,教授,博士生导师,研究方向为观点挖掘、情感分析、自然语言处理等. E-mail:liaoxw@fzu.edu.cn
  • 基金资助:
    国家自然科学基金资助项目(61976054,61772135,U1605251);国家自然科学基金青年科学基金资助项目(41801324);福建省自然科学基金面上项目(2017J01755);模式识别国家重点实验室开放课题基金项目(201900041)

Dual co-matching network with multiway attention for opinion reading comprehension

BAO Liang1,2,3, CHEN Zhi-hao1,2,3, CHEN Wen-zhang1,2,3, YE Kai1,2,3, LIAO Xiang-wen1,2,3*   

  1. 1. School of Mathematics and Computer Science, Fuzhou University, Fuzhou 350116, Fujian, China;
    2. Fujian Provincial Key Laboratory of Networking Computing and Intelligent Information Processing, Fuzhou University, Fuzhou 350116, Fujian, China;
    3. Digital Fujian Institute of Financial Big Data, Fuzhou 350116, Fujian, China
  • Published:2021-03-16

PDF (PC)

287

摘要： 观点型阅读理解旨在对于给定的问题和答案段落摘要,判断答案段落摘要相对于问题的观点倾向。现有基于神经网络的模型主要依靠精心设计的匹配网络来捕获各部分文本之间的关系,往往只考虑单向的匹配关系或是采用单一的交互机制对各个文本对之间的关系进行建模,难以有效捕捉观点型问题中问题和答案段落摘要二者之间的潜在关系。为此,提出一种基于双重多路注意力的匹配方法。该方法对<问题,答案段落摘要>二元组从2个方向同时进行匹配,并采用多种注意力机制学习二者的协同注意力表示,通过双向多视角的交互为最后的观点推断提供更丰富的推理证据。在观点型阅读理解数据集DureaderOpinion上的实验表明,该方法相对于基准模型取得了更好的效果。

关键词: 机器阅读理解, 观点挖掘, 注意力机制

Abstract: Opinion-based reading comprehension aims to judge the opinion polarity of the answer span to the given question. Previous approaches usually rely on a matching network to capture the relationship between texts, however either only modeling the relationship in a uni-direction, or only calculating the interactive representations with a single attention mechanism, which cannot effectively capture the correlation between the opinion-based question and the answer span. In this work, we propose dual co-matching with multiway attention(DCMA)matching method, which models the relationship between question and answer bidirectionally, and three attention mechanisms are employed to calculated the co-attention representations of question and answer. Experimental result on the opinion-based reading comprehension dataset DureaderOpinion demonstrates our model obtains state-of-the-art performance.

Key words: machine reading comprehension, opinion mining, attention mechanism

中图分类号: 

  • TP391
[1] RAJPURKAR P, ZHANG J, LOPYREV K, et al. SQuAD: 100, 000+ questions for machine comprehension of text[C] //Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2016: 2383-2392.
[2] HE W, LIU K, LIU J, et al. DuReader: a Chinese machine reading comprehension dataset from real-world applications[C] //Proceedings of the Workshop on Machine Reading for Question Answering. Stroudsburg: ACL, 2018: 37-46.
[3] SEO M, KEMBHAVI A, FARHADIA, et al. Bidirectional attention flow for machine comprehension[J/OL]. arXiv Preprint arXiv:1611.01603[cs]. 2016.
[4] HERMANN K M, KOCISKY T, GREFENSTETTE E, et al. Teaching machines to read and comprehend[C] //Advances in Neural Information Processing Systems. New York: Curran Associates, 2015: 1693-1701.
[5] WANG Shuohang, JIANG Jing. Machine comprehension using match-LSTM and answer pointer[J/OL]. arXiv Preprint arXiv:1608.07905[CS]. 2016.
[6] WANG W, YANG N, WEI F, et al. Gated self-matching networks for reading comprehension and question answering[C] //Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Stroudsburg: ACL, 2017: 189-198.
[7] DEVLIN J, CHANG M W,LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C] //Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: ACL, 2019: 4171-4186.
[8] YANG Z, DAI Z, YANG Y, et al. Xlnet: generalized autoregressive pretraining for language understanding[C] //Advances in Neural Information Processing Systems. New York: Curran Associates, 2019: 5753-5763.
[9] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J/OL]. arXiv Preprint arXiv:1409.0473[cs]. 2014.
[10] CHEN D, BOLTON J, MANNING C D. A thorough examination of the CNN/daily mail reading comprehension task[C] //Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Stroudsburg: ACL, 2016: 2358-2367.
[11] CUI Y, CHEN Z, WEI S, et al. Attention-over-attention neural networks for reading comprehension[C] //Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Stroudsburg: ACL, 2017: 593-602.
[12] ZHANG S, ZHAO H, WU Y, et al. DCMN+: dual co-matching network for multi-choice reading comprehension[J/OL]. arXiv Preprint arXiv: 1908.11511[cs]. 2019.
[13] DUAN X, WANG B, WANG Z, et al. Cjrc: a reliable human-annotated benchmark dataset for Chinese judicial reading comprehension[C] //China National Conference on Chinese Computational Linguistics. Berlin: Springer, 2019: 439-451.
[14] BAJAJ P, CAMPOS D, CRASWELL N, et al. Ms marco: a human generated machine reading comprehension dataset[J/OL]. arXiv Preprint arXiv: 1611.09268[cs]. 2016.
[15] TAN C, WEI F, WANG W, et al. Multiway attention networks for modeling sentence pairs[C] //Proceedings of the 27th International Joint Conference on Artificial Intelligence. Menlo Park: AAAI, 2018: 4411-4417.
[16] ZHU H, WEI F, QIN B, et al. Hierarchical attention flow for multiple-choice reading comprehension[C] //Thirty-Second AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2018: 6077-6084.
[17] LIU Y, OTT M, GOYAL N, et al. Roberta: a robustly optimized bert pretraining approach[J/OL]. arXiv Preprint arXiv:1907.11692[cs]. 2019.
[18] LAN Z, CHEN M, GOODMAN S, et al. Albert: a lite bert for self-supervised learning of language representations[J/OL]. arXiv Preprint arXiv: 1909.11942[cs]. 2019.
[19] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] //Advances in Neural Information Processing Systems. New York: Curran Associates, 2017: 5998-6008.
[20] LAI G, XIE Q, LIU H, et al. RACE: large-scale reading comprehension dataset from examinations[C] //Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2017: 785-794.
[21] PETERS M, NEUMANN M, IYYER M, et al. Deep contextualized word representations[C] //Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1(Long Papers). Stroudsburg: ACL, 2018: 2227-2237.
[22] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[J/OL]. https://www.cs.ubc.ca/~amuham01/LING530/papers/radford2018improving.pdf,2018.
[23] ROCKTÄSCHEL T, GREFENSTETTE E, HERMANN K M, et al. Reasoning about entailment with neural attention[J/OL]. arXiv Preprint arXiv: 1509.06664[cs]. 2015.
[24] WANG Z, MI H, ITTYCHERIAH A. Semi-supervised clustering for short text via deep representation learning[C] //Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning. Berlin: ACL, 2016: 31-39.
[25] SUN Y, WANG S, LI Y, et al. Ernie: enhanced representation through knowledge integration[J/OL]. arXiv Preprint arXiv:1904.09223[cs]. 2019.
[1] 郝长盈,兰艳艳,张海楠,郭嘉丰,徐君,庞亮,程学旗. 基于拓展关键词信息的对话生成模型[J]. 《山东大学学报(理学版)》, 2019, 54(7): 68-76.
[2] 罗毅, 李利, 谭松波, 程学旗. 基于中文微博语料的情感倾向性分析[J]. 山东大学学报（理学版）, 2014, 49(11): 1-7.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2018 《山东大学学报(理学版)》编辑部　
地址: 山东省济南市山大南路27号山东大学中心校区（250100）　电话: +86-0531-88366917　E-mail: xblxb@sdu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发