基于胶囊的英文文本蕴含识别方法

doi:10.13190/j.jbupt.2018-289

北京邮电大学学报 ›› 2019, Vol. 42 ›› Issue (3): 21-28.doi: 10.13190/j.jbupt.2018-289

基于胶囊的英文文本蕴含识别方法

朱皓, 谭咏梅

北京邮电大学智能科学与技术中心, 北京 100876

收稿日期:2018-11-16 出版日期:2019-06-28 发布日期:2019-06-20
作者简介:朱皓(1994-),男,硕士生,E-mail:hzhu@bupt.edu.cn;谭咏梅(1975-),女,副教授.

English Textual Entailment Recognition Using Capsules

ZHU Hao, TAN Yong-mei

Intelligence Science and Technology Center, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2018-11-16 Online:2019-06-28 Published:2019-06-20

摘要/Abstract

摘要： 提出了一种基于胶囊的英文文本蕴含识别方法.分别为每一种蕴含关系构建一个胶囊，用于模拟此蕴含关系的识别，并将其指定为该胶囊的属性.给定两段文本，经过highway编码层和序列编码层获取语义表示，分别输入胶囊中，依次通过其内部的交互模块、比较模块和聚合模块.交互模块利用交互注意力机制提取文本间的局部交互特征，比较模块和聚合模块使用前馈神经网络进行语义信息比较和聚合.最后对所有胶囊的输出归一化，得到两段文本的蕴含关系.该方法在SNLI测试集上的准确率为89.2%，在MultiNLI匹配测试集和不匹配测试集上的准确率分别为77.4%和76.4%.对交互模块中注意力关系矩阵的可视化分析结果验证了胶囊在英文文本蕴含识别任务中的有效性.

关键词: 文本蕴含识别, 胶囊, 交互注意力机制

Abstract: An English textual entailment recognition method using capsules is presented. This method builds a capsule for each relationship to model the recognition of this relationship and assigns it as the attribute of the capsule. Given two texts, they are first encoded by highway encoding layer and sequence encoding layer to obtain the semantic representations, and then fed into each capsule, passing through its internal interaction module, comparison module and aggregation module in turn. The interaction module uses the inter-attention mechanism to extract local interactive features between texts. The comparison module and the aggregation module use the feedforward network to compare and aggregate the semantic information. Finally, the output of all capsules is normalized to obtain the relationship between the two texts. The accuracy on SNLI test dataset is 89.2%, the accuracy on MultiNLI matched and mismatched test dataset is 77.4% and 76.4%. The visual analysis of the attentional relationship matrix of interaction module also verifies the effectiveness of capsules in the English textual entailment recognition task.

Key words: textual entailment recognition, capsules, inter-attention mechanism

中图分类号:

TN929.53

朱皓, 谭咏梅. 基于胶囊的英文文本蕴含识别方法[J]. 北京邮电大学学报, 2019, 42(3): 21-28.

ZHU Hao, TAN Yong-mei. English Textual Entailment Recognition Using Capsules[J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2019, 42(3): 21-28.

参考文献

[1] Bowman S R, Angeli G, Potts C, et al.A large annotated corpus for learning natural language inference[C]//Conference on Empirical Methods in Natural Language Processing.Lisbon:ACL Press, 2015:632-642.
[2] Williams A, Nangia N, Bowman S R.A broad-coverage challenge corpus for sentence understanding through inference[C]//Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies:Volume 1, Long Papers.New Orleans:ACL Press, 2018:1112-1122.
[3] Rocktäschel T, Grefenstette E, Hermann K M, et al.Reasoning about entailment with neural attention[EB/OL].2016(2016-03-01)[2018-11-16].https://arxiv.org/abs/1509.06664.
[4] Parikh A P, Täckström O, Das D, et al.A decomposable attention model for natural language inference[C]//Conference on Empirical Methods in Natural Language Processing.Austin:ACL Press, 2016:2249-2255.
[5] Sabour S, Frosst N, Hinton G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems.Long Beach:MIT Press, 2017:3859-3869.
[6] Wang Y, Sun A, Han J, et al.Sentiment analysis by capsules[C]//Proceedings of the 2018 World Wide Web Conference.Lyon:International World Wide Web Conferences Steering Committee, 2018:1165-1174.
[7] Liu Y, Sun C, Lin L, et al.Learning natural language inference using bidirectional LSTM model and inner-attention[EB/OL].2016(2016-05-30)[2018-11-16].https://arxiv.org/abs/1605.09090.
[8] Munkhdalai T, Yu H.Neural semantic encoders[C]//Proceedings of the 15^th Conference of the European Chapter of the Association for Computational Linguistics.Valencia:ACL Press, 2017:397-407.
[9] Wang S, Jiang J.Learning natural language inference with LSTM[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.San Diego:ACL Press, 2015:1442-1451.
[10] Munkhdalai T, Yu H.Neural tree indexers for text understanding[C]//Proceedings of the 15^th Conference of the European Chapter of the Association for Computational Linguistics.Valencia:ACL Press, 2017:11-21.
[11] Sha L, Chang B, Sui Z, et al.Reading and thinking:re-read LSTM unit for textual entailment recognition[C]//International Conference on Computational Linguistics:Technical Papers.Osaka:The COLING 2016 Organizing Committee, 2016:2870-2879.
[12] Gong Y, Luo H, Zhang J.Natural language inference over interaction space[EB/OL].2018(2018-05-26)[2018-11-16].https://arxiv.org/abs/1709.04348.
[13] Srivastava R K, Greff K, Schmidhuber J.Highway networks[EB/OL].2015(2015-11-03)[2018-11-16].https://arxiv.org/abs/1505.00387.
[14] Bowman S R, Gauthier J, Rastogi A, et al.A fast unified model for parsing and sentence understanding[C]//Proceedings of the 54^th Annual Meeting of the Association for Computational Linguistics.Berlin:ACL Press, 2016:1466-1477.
[15] Mou L, Men R, Li G, et al.Natural language inference by tree-based convolution and heuristic matching[C]//Proceedings of the 54^th Annual Meeting of the Association for Computational Linguistics.Berlin:ACL Press, 2016:130-136.
[16] Shen T, Zhou T, Long G, et al.DiSAN:directional self-attention network for RNN/CNN-free language understanding[C]//The 32^th AAAI Conference on Artificial Intelligence.New Orleans:AAAI Press, 2018:5446-5455.
[17] Chen Q, Zhu X, Ling Z, et al.Enhanced LSTM for natural language inference[C]//Proceedings of the 55^th Annual Meeting of the Association for Computational Linguistics.Vancouver:ACL Press, 2016:1657-1668.
[18] Balazs J A, Marrese-Taylor E, Loyola P, et al.Refining raw sentence representations for textual entailment recognition via attention[C]//Proceedings of the 2^nd Workshop on Evaluating Vector Space Representations for NLP.Copenhagen:ACL Press, 2017:51-55.
[19] Nie Y, Bansal M.Shortcut-stacked sentence encoders for multi-domain inference[C]//Proceedings of the 2^nd Workshop on Evaluating Vector Space Representations for NLP.Copenhagen:ACL Press, 2017:41-45.

基于胶囊的英文文本蕴含识别方法

English Textual Entailment Recognition Using Capsules

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价