Medical Question-Answer Matching Base on Adversarial Training

doi:10.13190/j.jbupt.2021-242

Journal of Beijing University of Posts and Telecommunications ›› 2022, Vol. 45 ›› Issue (4): 37-43.doi: 10.13190/j.jbupt.2021-242

• Special Topics on Intelligent Medical • Previous Articles Next Articles

Medical Question-Answer Matching Base on Adversarial Training

FU Jieqiong^1,2, SUN Yawei^1,2, LIU Jianyi³, LI Jinbin⁴

1. School of Computer Science(National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing 100876, China;
2. Key Laboratory of Trustworthy Distributed Computing and Service(Ministry of Education), Beijing University of Posts and Telecommunications, Beijing 100876, China;
3. School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing 100876, China;
4. National Population Health Data Center, Chinese Academy of Medical Sciences, Beijing 100005, China

Received:2021-10-18 Online:2022-08-28 Published:2022-06-26

Abstract

Abstract: Compared with the question-answer matching task in the English open domain, the task in the Chinese professional medical field is more challenging.In view of the complexity of Chinese semantics and the diversity of medical data,most researchers focus on designing complex neural networks to explore deeper text semantics, and this kind of idea is relatively simple.At the same time, the neural network model is easy to make misjudgments due to small disturbances, and the poor generalization ability of the model.To solve these issues,a question-answer matching model is proposed based on adversarial training. A bidirectional pre-training encoder is used in this model to capture the semantic information of question-answer sentences and to obtain the corresponding vector representation. Then, adversarial samples are generated by adding a disturbance factor to the word embedding representation.Finally, the initial samples and adversarial samples are jointly input into the model with linear layers for classification prediction.Comparative experiments demonstrate that adversarial training can effectively improve the performance of the question-answer matching model on the cMedQA V2.0 dataset.

Key words: medical question-answer, adversarial training, natural language processing

CLC Number:

TP311.5

FU Jieqiong, SUN Yawei, LIU Jianyi, LI Jinbin. Medical Question-Answer Matching Base on Adversarial Training[J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(4): 37-43.

References

[1] LIU Y H, OTT M, GOYAL N, et al. RoBERTa:a robustly optimized BERT pretraining approach[EB/OL]. (2019-07-26)[2021-08-26]. https://arxiv.org/pdf/1907.11692.pdf.
[2] ZHANG S, ZHANG X, WANG H, et al. Chinese medical question answer matching using end-to-end character-level multi-scale CNNs[J/OL]. Applied Sciences, 2017, 7(8)[2021-08-26]. https://www.mdpi.com/2076-3417/7/8/767.
[3] ZHANG S, ZHANG X, WANG H, et al. Multi-scale attentive interaction networks for Chinese medical question answer selection[J]. IEEE Access, 2018, 6:74061-74071.
[4] TIAN Y H, MA W C, XIA F, et al. ChiMed:a Chinesemedical corpus for question answering[C]//Proceedings of the 18th BioNLP Workshop and Shared Task.[S.l.]:Association for Computational Linguistics, 2019:250-260.
[5] HE J Q, FU M M, TU M S. Applying deep matching networks to Chinese medical question answering:a study and a dataset[J]. BMC Medical Informatics and Decision Making, 2019, 19(Sup 2):52.
[6] CUI X T, HAN J G. Chinese medical question answer matching based on interactive sentence representation learning[J]. Computer Science & Information Technology, 2020, 8(5):93-109.
[7] GOODFELLOW I, JEANP-A, MEHDIM, et al. Generative adversarial nets[C]//Neural Information Processing Systems. Montreal:[s.n.], 2014:27-36.
[8] SZEGEDY C, ZAREMBA W, SUTSKEVER I, et al. Intriguing properties of neural networks:proceedings of "International Conference on Learning Representations"[C/OL].[S.l.]:ICLR,2014[2021-08-26]. http://arxiv.org/abs/1312.6199.
[9] MIYATO T, DAI A M, GOODFELLOW I. Adversarial training methods for semi-supervised text classification:proceedings of "International Conference on Learning Representations"[C/OL].[S.l.]:ICLR, 2017[2021-08-26]. https://arxiv.org/abs/1605.07725.
[10] ZHU C, CHENG Y, GAN Z, et al. FreeLB:enhanced adversarial training for language understanding:proceedings of "International Conference on Learning Representations"[C/OL].[S.l.]:ICLR, 2020[2021-08-26]. https://openreview.net/forum?id=Fk_47QnlVy.
[11] JU Y, ZHAO F B, CHEN S J, et al. Technical report on conversational question answering[EB/OL]. (2019-09-24)[2021-08-26]. https://arxiv.org/pdf/1909.10772.pdf.
[12] DEVLINJ, CHANGMW, LEEK, et al. BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volumn 1(Long and Short Papers). Minneapolis:NAACL, 2019:4171-4186.
[13] CUI Y M, CHE W X, LIU T, et al. Revisiting pre-trained models for Chinese natural language processing[C]//Conference on Empirical Methods in Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2020:657-668.
[14] GOODFELLOW I, JONATHON S, CHRISTIAN S. Explaining and harnessing adversarial examples[EB/OL]. (2015-03-20)[2021-08-26]. https://arxiv.org/pdf/1412.6572.pdf.
[15] XIE C H, WU Y X, MAATEN L, et al. Feature denoising for improving adversarial robustness[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE Press, 2019:501-509.

Medical Question-Answer Matching Base on Adversarial Training

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 6

Recommended Articles

Metrics

Comments

[1]	GUO Zhi-xin, DENG Xiao-long. Intelligent Identification Method of Legal Case Entity Based on BERT-BiLSTM-CRF [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(4): 129-134.
[2]	HUANG Shi-lin, ZHENG Xiao-lin, CHEN De-ren. A Semi-Supervised Learning Method for Product Named Entity Recognition [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2013, 36(2): 20-23.
[3]	LI Rong-Jun, WANG Xiao-Cha, ZHOU Yan-Quan. Semantic Orientation Computing Using PageRank Model [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2010, 33(5): 141-144.
[4]	LIU Yao, SUI Zhifang, HU yongwei, Ji tieliang. Automatic Construction On of Domain Ontology [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2006, 29(s2): 65-69.
[5]	LI Lei, Xu Jing, Gong Yuan, ZHOU Yan-quan, Wang Cong, ZHONG Yi-xin. CI-NLU Based Text Processing After Speech Recognition [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2006, 29(s2): 192-196.
[6]	LI Lei, SUN Chun-kui, YANG Xiao-lan, ZHONG Yi-xin. Understanding-Based Chinese Automatic Abstracting System in Special Domain [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2000, 23(1): 6-10.