基于预训练语言模型的中医症状标准化方法

doi:10.13190/j.jbupt.2021-191

北京邮电大学学报 ›› 2022, Vol. 45 ›› Issue (4): 13-18,57.doi: 10.13190/j.jbupt.2021-191

基于预训练语言模型的中医症状标准化方法

谢永红^1,2, 陶浒^1,2, 贾麒^1,2, 杨石兵^1,2, 韩辛亮²

1. 北京科技大学计算机与通信工程学院, 北京 100083;
2. 材料领域知识工程北京市重点实验室, 北京 100083

收稿日期:2021-09-01 出版日期:2022-08-28 发布日期:2022-09-03
作者简介:谢永红(1970—),女,副教授,邮箱:xieyh@ustb.edu.cn。
基金资助:
国家重点研发计划项目（2018YFC1707410）

Traditional Chinese Medicine Symptom Normalization Approach Based on Pre-Trained Language Models

XIE Yonghong^1,2, TAO Hu^1,2, JIA Qi^1,2, YANG Shibing^1,2, HAN Xinliang²

1. School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China;
2. Beijing Key Laboratory of Knowledge Engineering for Materials Science, Beijing 100083, China

Received:2021-09-01 Online:2022-08-28 Published:2022-09-03

摘要/Abstract

摘要： 为了解决中医症状描述词的异名同义、一对多的问题，提出了一种基于预训练语言模型的2阶段症状标准化框架：第1阶段，生成候选标准症状词，参考中医症状词的定义与分类，利用多标签分类思想对原始症状词进行语义划分，进而得到相应语义标签下的候选标准症状词；第2阶段，对候选标准症状词进行排序，使用匹配模型对第1阶段得到的候选标准症状词集进行评分与排序，同时用策略对结果进行二次召回以提高症状标准化框架的性能，由此得到最终的标准化结果。实验结果表明，提出的症状标准化方法与传统方法相比能够更有效地处理症状标准化的问题。通过对比分析不同预训练语言模型在症状标准化任务上的性能，进一步说明了所提框架和策略的有效性。

关键词: 中医, 症状标准化, 实体匹配, 语义分类, 预训练语言模型

Abstract: To solve the issue in traditional Chinese medicine that one symptom has different literal descriptions and one symptom corresponds to multiple normalized descriptions, a two-stage framework based on pre-trained language models is proposed. In the first step, according to the definition and classification of symptoms, a multi-label text classification model is adopted to semantically divide the symptom descriptions to obtain candidate normalization symptom words. In the second step, we score and sort the candidate normalization symptom words with an entity matching model, and some strategies are designed to perform a second recall of the results to improve performance. After that, the candidate word with the highest score in each semantic label is regarded as the normalization result. Experiments results show that the proposed method performs better than traditional methods on solving the symptom normalization problem. Furthermore, the research compares and analyzes the results using different pre-trained language models on the symptom normalization task to verify the effectiveness of the proposed method.

Key words: traditional Chinese medicine, symptom normalization, entity matching, semantic classification, pre-trained language model

中图分类号:

TP391.1

谢永红, 陶浒, 贾麒, 杨石兵, 韩辛亮. 基于预训练语言模型的中医症状标准化方法[J]. 北京邮电大学学报, 2022, 45(4): 13-18,57.

XIE Yonghong, TAO Hu, JIA Qi, YANG Shibing, HAN Xinliang. Traditional Chinese Medicine Symptom Normalization Approach Based on Pre-Trained Language Models[J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(4): 13-18,57.

参考文献

[1] QIU X P, SUN T X, XU Y G, et al. Pre-trained models for natural language processing:a survey[J]. Science China Technological Sciences, 2020, 63(10):1872-1897.
[2] JIA Q, ZHANG D Z, YANG S B, et al. Traditional Chinese medicine symptom normalization approach leveraging hierarchical semantic information and text matching with attention mechanism[J/OL]. Journal of Biomedical Informatics, 2021, 116(6):103718[2021-07-30]. https://doi.org/10.1016/j.jbi.2021.103718.
[3] TUTUBALINA E, MIFTAHUTDINOV Z, NIKOLENKO S, et al. Medical concept normalization in social media posts with recurrent neural networks[J]. Journal of Biomedical Informatics, 2018, 84:93-102.
[4] DENG P, CHEN H P, HUANG M Y, et al. An ensemble CNN method for biomedical entity normalization[C]//Proceedings of the 5th Workshop on BioNLP Open Shared Tasks. Stroudsburg:ACL, 2019:143-149.
[5] AN Y, WANG J L, ZHANG L, et al. PASCAL:a pseudo cascade learning framework for breast cancer treatment entity normalization in Chinese clinical text[J]. BMC Medical Informatics and Decision Making, 2020, 20(1):1-12.
[6] CHEN Q, ZHU X D, LING Z H, et al. Enhanced LSTM for natural language inference[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers). Stroudsburg:ACL, 2017:1657-1668.
[7] DAI Z Y, XIONG C Y, CALLAN J, et al. Convolutional neural networks for soft-matching n-grams in Ad-hoc search[C]//Proceedings of the 11th ACM International Conference on Web Search and Data Mining. New York:ACM, 2018:126-134.
[8] DEVLIN J, CHANG M W, LEE K, et al. BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of NAACL-HLT. Stroudsburg:ACL, 2019:4171-4186.
[9] LIU Y H, OTT M, GOYAL N, et al. RoBERTa:a robustly optimized BERT pretraining approach[EB/OL]. (2019-07-26)[2021-07-30]. https://arxiv.org/abs/1907.11692.
[10] SUN Y, WANG S H, LI Y K, et al. ERNIE:enhanced representation through knowledge integration[EB/OL]. (2019-04-19)[2021-07-30]. https://arxiv.org/abs/1904.09223.
[11] CLARK K, LUONG M T, LE Q V, et al. ELECTRA:pre-training text encoders as discriminators rather than generators[EB/OL].(2020-03-23)[2021-07-30].https://arxiv.org/abs/2003.10555.

基于预训练语言模型的中医症状标准化方法

Traditional Chinese Medicine Symptom Normalization Approach Based on Pre-Trained Language Models

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价