Driver Distraction Recognition Using Bilinear Fusion Networks

doi:10.13190/j.jbupt.2021-171

Journal of Beijing University of Posts and Telecommunications ›› 2022, Vol. 45 ›› Issue (2): 79-84.doi: 10.13190/j.jbupt.2021-171

• PAPERS • Previous Articles Next Articles

Driver Distraction Recognition Using Bilinear Fusion Networks

LIU Changyuan¹, HU Haoyuan¹, BI Xiaojun²

1. School of Measurement and Control Technology and Communication Engineering, Harbin University of Science and Technology, Harbin 150080, China;
2. School of Information Engineering, Minzu University of China, Beijing 100081, China

Received:2021-08-08 Published:2021-12-16

Abstract

Abstract: Accurate recognition of driver's distraction behavior can radically reduce the traffic accidents. Traditional recognition methods have the problems of few classification categories and low accuracy. To solve these problems, residual neural network (ResNet-50) is employed to recognize driver's distraction behavior and improve the network. To further improve the feature extraction ability of the model, bilinear fusion is carried out on the features extracted from the improved ResNet-50 model and EfficientNet-B0 model. Thus, the recognition accuracy of the model can be further improved. The average accuracy of the improved ResNet-50 single model is up to 94.2%, and the average accuracy of the model after fusing the improved ResNet-50 with EfficientNet-B0 is up to 96.7%. Experimental results show that this method has a good classification effect on the detection of driver's distraction behavior.

Key words: distracted behavior recognition, deep learning, ResNet-50, bilinear fusion

CLC Number:

TP391.4

LIU Changyuan, HU Haoyuan, BI Xiaojun. Driver Distraction Recognition Using Bilinear Fusion Networks[J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(2): 79-84.

References

[1] 刘逸琛,孙华志,马春梅,等.一种基于高层特征融合的网络商品分类[J].北京邮电大学学报, 2020, 43(5):98-104, 117. LIU Y C, SUN H Z, MA C M, et al. Commodity classification of online based on high-level feature fusion[J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(5):98-104, 117.
[2] YAN C, COENEN F, YUE Y, et al. Video-based classification of driving behavior using a hierarchical classification system with multiple features[J]. International Journal of Pattern Recognition and Artificial Intelligence, 2016, 30(5):1-33.
[3] 田文洪,曾柯铭,莫中勤,等.基于卷积神经网络的驾驶员不安全行为识别[J].电子科技大学学报, 2019, 48(3):381-387. TIAN W H, ZENG K M, MO Z Q, et al. Recognition of unsafe driving behaviors based on convolutional neural network[J]. Journal of University of Electronic Science and Technology of China, 2019, 48(3):381-387.
[4] YANG X, TANG J L, LIU H, et al. End-to-end driving activities and secondary tasks recognition using deep convolutional neural network and transfer learning[C]//2018 IEEE Intelligent Vehicles Symposium. Piscatway, NJ:IEEE Press, 2018:1626-1631.
[5] LE T H N, ZHENG Y T, ZHU C C, et al. Multiple scale faster-RCNN approach to driver's cell-phone usage and hands on steering wheel detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, NJ:IEEE Press, 2016:46-53.
[6] 熊群芳,林军,岳伟,等.基于深度学习的驾驶员打电话行为检测方法[J].控制与信息技术, 2019(6):53-56, 62. XIONG Q F, LIN J, YUE W. A driver's calling behavior detection method based on deep learning[J]. Control and Information Technology, 2019(6):53-56, 62.
[7] HE K M, ZHANG X G, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ:IEEE Press, 2016:770-778.
[8] 张一帆.基于深度学习的驾驶员分心行为识别研究[D].哈尔滨:哈尔滨理工大学, 2021. ZHANG Y F. Research on driver distraction behavior recognition based on deep learning[D]. Harbin:Harbin University of Science and Technology, 2021.
[9] TAN M X, LE Q V. EfficientNet:rethinking model scaling for convolutional neural networks[C]//International Conference on Machine Learning. Lugano:ICML, 2019:6105-6114.
[10] LIN T Y, ROYCHOWDHURY A, MAJI S. Bilinear convolutional neural networks for fine-grained visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(6):1309-1322.

[1]	LIU Yang, TENG Yinglei, NIU Tao, ZHI Jialin. Filter Pruning Algorithm Based on Deep Reinforcement Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 31-36.
[2]	. Multi-Mode Guided Point Cloud Completion Method in Complex Environment [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 103-108.
[3]	RONG Zhenyu, LIU Jianyi. Retinal Blood Vessel Segmentation Based on Transformer and MLP [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(1): 26-31.
[4]	LIU Jintong, YANG Guoxing, LIU Xiaohong, WANG Guangyu. Smoothing Attack Algorithm Based on Electrocardiogram Classification [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(4): 44-50.
[5]	HUANG Liuting, LIU Kexin, NIU Kai, CHANG Chun, HE Zhiqiang. Automatic Recognition of Mucus Impaction in CT Images of Asthmatic Patients Using Deep Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(4): 58-63.
[6]	ZHAO Haiying, ZHU Hui, HOU Xiaogang. Traditional Custume Image Semantic Segmentation Based on Improved EMA Unit [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(1): 69-74.
[7]	ZHANG Binyu, ZHAO Yanyun, DU Yunhao, WAN Junfeng, TONG Zhihang. Character Detection Method for PCB Image Based on Deep Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(1): 108-114.
[8]	JIA Jun, FENG Chun-yan, XIA Hai-lun, ZHANG Tian-kui, LI Cheng-gang. Communication Networks Fault Prediction Method Based on Sample Equalization and Feature Interaction [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(6): 59-66.
[9]	PU Yun-wei, GUO Jiang, LIU Tao-tao, WU Hai-xiao. A Recognition Method for Radar Emitter Signals Based on Convolutional Neural Network with Multiple Learning Units [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(6): 74-82.
[10]	WANG Yi-fei, MO Shuang, WU Wen-rui, FAN Shao-hua, XIAO Ding. Internal-External Convolutional Networks for Network Intrusion Detection [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(5): 94-100.
[11]	GAO Hui, ZHANG Ji-wei, LAI Yang, WANG Wen-dong. Deep Learning Based Semi-Automatic Labeling System for Human Images [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(1): 104-109.
[12]	PU Yue-yi, WANG Wen-han, ZHU Qiang, CHEN Peng-peng. Urban Short-Term Traffic Flow Prediction Algorithm Based on CNN-ResNet-LSTM Model [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(5): 9-14.
[13]	MA Xiang-liang, LI Bing, YANG Dan, HUANG Ke-zhen, DUAN Xiao-yi. Reverse-Analysis of S-Box for SM4-Like Algorithms Based on Side Channel Technology [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(5): 118-124.
[14]	ZHANG Tian-qi, KANG Bo, MENG Xiang-fei, LIU Yi-lin, ZHOU Ying. U-Net Based Intracranial Hemorrhage Recognition [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(3): 92-98.
[15]	MA Su-gang, ZHAO Xiang-mo, HOU Zhi-qiang, WANG Zhong-min, SUN Han-lin. A Visual Object Tracking Algorithm Based on Features Extracted by Deep Residual Network [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(2): 129-134.

Driver Distraction Recognition Using Bilinear Fusion Networks

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments