Abstract: Aiming at the problems that the problems of single diagnosis and low accuracy in the existing osteoporosis assessment, considering the bone image data and questionnaire data, a multi-modal feature fusion osteoporosis evaluation method based on deep neural network was proposed. And, for the characteristics of shallow image and fixed structure of bone image, Unet is used to perform image segmentation preprocessing to remove redundant information. In view of the shortcomings of ordinary convolution operations in grasping the global information, a new convolutional neural network based on non-local module was proposed to further enrich the feature information. Cross-validation shows that the proposed multimodal feature fusion method has obvious advantages compared with the machine learning method using only image data or questionnaire data alone, and the classification accuracy rate is increased by 3.2% and 22.3%.

Key words: osteoporosis, multi-modal fusion, deep neural network