JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE) ›› 2021, Vol. 56 ›› Issue (3): 44-53.doi: 10.6040/j.issn.1671-9352.0.2020.346

Previous Articles    

Dual co-matching network with multiway attention for opinion reading comprehension

BAO Liang1,2,3, CHEN Zhi-hao1,2,3, CHEN Wen-zhang1,2,3, YE Kai1,2,3, LIAO Xiang-wen1,2,3*   

  1. 1. School of Mathematics and Computer Science, Fuzhou University, Fuzhou 350116, Fujian, China;
    2. Fujian Provincial Key Laboratory of Networking Computing and Intelligent Information Processing, Fuzhou University, Fuzhou 350116, Fujian, China;
    3. Digital Fujian Institute of Financial Big Data, Fuzhou 350116, Fujian, China
  • Published:2021-03-16

PDF (PC)

287

Abstract: Opinion-based reading comprehension aims to judge the opinion polarity of the answer span to the given question. Previous approaches usually rely on a matching network to capture the relationship between texts, however either only modeling the relationship in a uni-direction, or only calculating the interactive representations with a single attention mechanism, which cannot effectively capture the correlation between the opinion-based question and the answer span. In this work, we propose dual co-matching with multiway attention(DCMA)matching method, which models the relationship between question and answer bidirectionally, and three attention mechanisms are employed to calculated the co-attention representations of question and answer. Experimental result on the opinion-based reading comprehension dataset DureaderOpinion demonstrates our model obtains state-of-the-art performance.

Key words: machine reading comprehension, opinion mining, attention mechanism

CLC Number: 

  • TP391
[1] RAJPURKAR P, ZHANG J, LOPYREV K, et al. SQuAD: 100, 000+ questions for machine comprehension of text[C] //Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2016: 2383-2392.
[2] HE W, LIU K, LIU J, et al. DuReader: a Chinese machine reading comprehension dataset from real-world applications[C] //Proceedings of the Workshop on Machine Reading for Question Answering. Stroudsburg: ACL, 2018: 37-46.
[3] SEO M, KEMBHAVI A, FARHADIA, et al. Bidirectional attention flow for machine comprehension[J/OL]. arXiv Preprint arXiv:1611.01603[cs]. 2016.
[4] HERMANN K M, KOCISKY T, GREFENSTETTE E, et al. Teaching machines to read and comprehend[C] //Advances in Neural Information Processing Systems. New York: Curran Associates, 2015: 1693-1701.
[5] WANG Shuohang, JIANG Jing. Machine comprehension using match-LSTM and answer pointer[J/OL]. arXiv Preprint arXiv:1608.07905[CS]. 2016.
[6] WANG W, YANG N, WEI F, et al. Gated self-matching networks for reading comprehension and question answering[C] //Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Stroudsburg: ACL, 2017: 189-198.
[7] DEVLIN J, CHANG M W,LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C] //Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: ACL, 2019: 4171-4186.
[8] YANG Z, DAI Z, YANG Y, et al. Xlnet: generalized autoregressive pretraining for language understanding[C] //Advances in Neural Information Processing Systems. New York: Curran Associates, 2019: 5753-5763.
[9] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J/OL]. arXiv Preprint arXiv:1409.0473[cs]. 2014.
[10] CHEN D, BOLTON J, MANNING C D. A thorough examination of the CNN/daily mail reading comprehension task[C] //Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Stroudsburg: ACL, 2016: 2358-2367.
[11] CUI Y, CHEN Z, WEI S, et al. Attention-over-attention neural networks for reading comprehension[C] //Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Stroudsburg: ACL, 2017: 593-602.
[12] ZHANG S, ZHAO H, WU Y, et al. DCMN+: dual co-matching network for multi-choice reading comprehension[J/OL]. arXiv Preprint arXiv: 1908.11511[cs]. 2019.
[13] DUAN X, WANG B, WANG Z, et al. Cjrc: a reliable human-annotated benchmark dataset for Chinese judicial reading comprehension[C] //China National Conference on Chinese Computational Linguistics. Berlin: Springer, 2019: 439-451.
[14] BAJAJ P, CAMPOS D, CRASWELL N, et al. Ms marco: a human generated machine reading comprehension dataset[J/OL]. arXiv Preprint arXiv: 1611.09268[cs]. 2016.
[15] TAN C, WEI F, WANG W, et al. Multiway attention networks for modeling sentence pairs[C] //Proceedings of the 27th International Joint Conference on Artificial Intelligence. Menlo Park: AAAI, 2018: 4411-4417.
[16] ZHU H, WEI F, QIN B, et al. Hierarchical attention flow for multiple-choice reading comprehension[C] //Thirty-Second AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2018: 6077-6084.
[17] LIU Y, OTT M, GOYAL N, et al. Roberta: a robustly optimized bert pretraining approach[J/OL]. arXiv Preprint arXiv:1907.11692[cs]. 2019.
[18] LAN Z, CHEN M, GOODMAN S, et al. Albert: a lite bert for self-supervised learning of language representations[J/OL]. arXiv Preprint arXiv: 1909.11942[cs]. 2019.
[19] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] //Advances in Neural Information Processing Systems. New York: Curran Associates, 2017: 5998-6008.
[20] LAI G, XIE Q, LIU H, et al. RACE: large-scale reading comprehension dataset from examinations[C] //Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2017: 785-794.
[21] PETERS M, NEUMANN M, IYYER M, et al. Deep contextualized word representations[C] //Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1(Long Papers). Stroudsburg: ACL, 2018: 2227-2237.
[22] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[J/OL]. https://www.cs.ubc.ca/~amuham01/LING530/papers/radford2018improving.pdf,2018.
[23] ROCKTÄSCHEL T, GREFENSTETTE E, HERMANN K M, et al. Reasoning about entailment with neural attention[J/OL]. arXiv Preprint arXiv: 1509.06664[cs]. 2015.
[24] WANG Z, MI H, ITTYCHERIAH A. Semi-supervised clustering for short text via deep representation learning[C] //Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning. Berlin: ACL, 2016: 31-39.
[25] SUN Y, WANG S, LI Y, et al. Ernie: enhanced representation through knowledge integration[J/OL]. arXiv Preprint arXiv:1904.09223[cs]. 2019.
[1] Chang-ying HAO,Yan-yan LAN,Hai-nan ZHANG,Jia-feng GUO,Jun XU,Liang PANG,Xue-qi CHENG. Dialogue generation model based on extended keywords information [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2019, 54(7): 68-76.
[2] LUO Yi, LI Li, TAN Song-bo, CHENG Xue-qi. Sentiment analysis on Chinese Micro-blog corpus [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 1-7.
[3] ZHENG Yan, PANG Lin, BI Hui, LIU Wei, CHENG Gong. Feature selection algorithm based on sentiment topic model [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 74-81.
[4] XIA Meng-nan, DU Yong-ping, ZUO Ben-xin. Micro-blog opinion analysis based on syntactic dependency and feature combination [J]. JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE), 2014, 49(11): 22-30.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of JOURNAL OF SHANDONG UNIVERSITY(NATURAL SCIENCE)
Supported by Beijing Magtech Co.Ltd