一种基于对抗训练的医疗问答匹配方法

doi:10.13190/j.jbupt.2021-242

北京邮电大学学报 ›› 2022, Vol. 45 ›› Issue (4): 37-43.doi: 10.13190/j.jbupt.2021-242

一种基于对抗训练的医疗问答匹配方法

付洁琼^1,2, 孙亚伟^1,2, 刘建毅³, 李金斌⁴

1. 北京邮电大学计算机学院(国家示范性软件学院), 北京 100876;
2. 北京邮电大学可信分布式计算与服务教育部重点实验室, 北京 100876;
3. 北京邮电大学网络空间安全学院, 北京 100876;
4. 中国医学科学院国家人口健康科学数据中心, 北京 100005

收稿日期:2021-10-18 出版日期:2022-08-28 发布日期:2022-06-26
通讯作者: 孙亚伟(1992—),男,博士生,邮箱:sunyawei@bupt.edu.cn。 E-mail:sunyawei@bupt.edu.cn
作者简介:付洁琼(1998—),女,硕士生。
基金资助:
国家自然科学基金项目（U1936216）

Medical Question-Answer Matching Base on Adversarial Training

FU Jieqiong^1,2, SUN Yawei^1,2, LIU Jianyi³, LI Jinbin⁴

1. School of Computer Science(National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing 100876, China;
2. Key Laboratory of Trustworthy Distributed Computing and Service(Ministry of Education), Beijing University of Posts and Telecommunications, Beijing 100876, China;
3. School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing 100876, China;
4. National Population Health Data Center, Chinese Academy of Medical Sciences, Beijing 100005, China

Received:2021-10-18 Online:2022-08-28 Published:2022-06-26

摘要/Abstract

摘要： 相较于英文开放领域的问答匹配，中文专业医疗领域的问答匹配任务更具有挑战性。针对中文语义和医疗数据的复杂、多样，大多数研究人员都专注于设计繁杂的神经网络来探索更深层次的文本语义，工作思路较为单一，同时神经网络模型很容易因为微小扰动而误判，模型的泛化能力较差。为此，提出了一种基于对抗训练的问答匹配模型，利用双向预训练编码器来捕获问答句的语义信息，从而得到对应的向量表征；再通过在词嵌入表示上添加扰动因子生成对抗样本；最后将初始样本和对抗样本共同输入带有线性层的模型中进行分类预测。在cMedQA V2.0数据集上通过对比实验证明了对抗训练可以有效提升问答匹配模型的性能。

关键词: 医疗问答, 对抗训练, 自然语言处理

Abstract: Compared with the question-answer matching task in the English open domain, the task in the Chinese professional medical field is more challenging.In view of the complexity of Chinese semantics and the diversity of medical data,most researchers focus on designing complex neural networks to explore deeper text semantics, and this kind of idea is relatively simple.At the same time, the neural network model is easy to make misjudgments due to small disturbances, and the poor generalization ability of the model.To solve these issues,a question-answer matching model is proposed based on adversarial training. A bidirectional pre-training encoder is used in this model to capture the semantic information of question-answer sentences and to obtain the corresponding vector representation. Then, adversarial samples are generated by adding a disturbance factor to the word embedding representation.Finally, the initial samples and adversarial samples are jointly input into the model with linear layers for classification prediction.Comparative experiments demonstrate that adversarial training can effectively improve the performance of the question-answer matching model on the cMedQA V2.0 dataset.

Key words: medical question-answer, adversarial training, natural language processing

中图分类号:

TP311.5

付洁琼, 孙亚伟, 刘建毅, 李金斌. 一种基于对抗训练的医疗问答匹配方法[J]. 北京邮电大学学报, 2022, 45(4): 37-43.

FU Jieqiong, SUN Yawei, LIU Jianyi, LI Jinbin. Medical Question-Answer Matching Base on Adversarial Training[J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(4): 37-43.

参考文献

[1] LIU Y H, OTT M, GOYAL N, et al. RoBERTa:a robustly optimized BERT pretraining approach[EB/OL]. (2019-07-26)[2021-08-26]. https://arxiv.org/pdf/1907.11692.pdf.
[2] ZHANG S, ZHANG X, WANG H, et al. Chinese medical question answer matching using end-to-end character-level multi-scale CNNs[J/OL]. Applied Sciences, 2017, 7(8)[2021-08-26]. https://www.mdpi.com/2076-3417/7/8/767.
[3] ZHANG S, ZHANG X, WANG H, et al. Multi-scale attentive interaction networks for Chinese medical question answer selection[J]. IEEE Access, 2018, 6:74061-74071.
[4] TIAN Y H, MA W C, XIA F, et al. ChiMed:a Chinesemedical corpus for question answering[C]//Proceedings of the 18th BioNLP Workshop and Shared Task.[S.l.]:Association for Computational Linguistics, 2019:250-260.
[5] HE J Q, FU M M, TU M S. Applying deep matching networks to Chinese medical question answering:a study and a dataset[J]. BMC Medical Informatics and Decision Making, 2019, 19(Sup 2):52.
[6] CUI X T, HAN J G. Chinese medical question answer matching based on interactive sentence representation learning[J]. Computer Science & Information Technology, 2020, 8(5):93-109.
[7] GOODFELLOW I, JEANP-A, MEHDIM, et al. Generative adversarial nets[C]//Neural Information Processing Systems. Montreal:[s.n.], 2014:27-36.
[8] SZEGEDY C, ZAREMBA W, SUTSKEVER I, et al. Intriguing properties of neural networks:proceedings of "International Conference on Learning Representations"[C/OL].[S.l.]:ICLR,2014[2021-08-26]. http://arxiv.org/abs/1312.6199.
[9] MIYATO T, DAI A M, GOODFELLOW I. Adversarial training methods for semi-supervised text classification:proceedings of "International Conference on Learning Representations"[C/OL].[S.l.]:ICLR, 2017[2021-08-26]. https://arxiv.org/abs/1605.07725.
[10] ZHU C, CHENG Y, GAN Z, et al. FreeLB:enhanced adversarial training for language understanding:proceedings of "International Conference on Learning Representations"[C/OL].[S.l.]:ICLR, 2020[2021-08-26]. https://openreview.net/forum?id=Fk_47QnlVy.
[11] JU Y, ZHAO F B, CHEN S J, et al. Technical report on conversational question answering[EB/OL]. (2019-09-24)[2021-08-26]. https://arxiv.org/pdf/1909.10772.pdf.
[12] DEVLINJ, CHANGMW, LEEK, et al. BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volumn 1(Long and Short Papers). Minneapolis:NAACL, 2019:4171-4186.
[13] CUI Y M, CHE W X, LIU T, et al. Revisiting pre-trained models for Chinese natural language processing[C]//Conference on Empirical Methods in Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2020:657-668.
[14] GOODFELLOW I, JONATHON S, CHRISTIAN S. Explaining and harnessing adversarial examples[EB/OL]. (2015-03-20)[2021-08-26]. https://arxiv.org/pdf/1412.6572.pdf.
[15] XIE C H, WU Y X, MAATEN L, et al. Feature denoising for improving adversarial robustness[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE Press, 2019:501-509.

一种基于对抗训练的医疗问答匹配方法

Medical Question-Answer Matching Base on Adversarial Training

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 7

编辑推荐

Metrics

本文评价

[1]	郭知鑫, 邓小龙. 基于BERT-BiLSTM-CRF的法律案件实体智能识别方法[J]. 北京邮电大学学报, 2021, 44(4): 129-134.
[2]	黄诗琳郑小林陈德人. 针对产品命名实体识别的半监督学习方法[J]. 北京邮电大学学报, 2013, 36(2): 20-23.
[3]	李荣军, 王小捷, 周延泉. PageRank模型在中文情感词极性判别中的应用[J]. 北京邮电大学学报, 2010, 33(5): 141-144.
[4]	王大亮1, 张德政1, 涂序彦1, 郑雪峰1, 佟子健2. 基于相对条件熵的搭配抽取方法[J]. 北京邮电大学学报, 2007, 30(6): 40-45.
[5]	刘耀　穗志方胡永伟冀铁亮. 领域Ontology自动构建研究[J]. 北京邮电大学学报, 2006, 29(s2): 65-69.
[6]	李蕾, 许晶, 龚媛, 周延泉, 王枞, 钟义信. 基于全信息自然语言理解的语音识别后文本处理[J]. 北京邮电大学学报, 2006, 29(s2): 192-196.
[7]	李蕾,孙春葵,杨晓兰,钟义信. 一种特定领域中文自动摘要系统[J]. 北京邮电大学学报, 2000, 23(1): 6-10.