双线性融合网络的驾驶员分心行为识别

doi:10.13190/j.jbupt.2021-171

北京邮电大学学报 ›› 2022, Vol. 45 ›› Issue (2): 79-84.doi: 10.13190/j.jbupt.2021-171

双线性融合网络的驾驶员分心行为识别

柳长源¹, 虎浩媛¹, 毕晓君²

1. 哈尔滨理工大学测控技术与通信工程学院, 哈尔滨 150080;
2. 中央民族大学信息工程学院, 北京 100081

收稿日期:2021-08-08 发布日期:2021-12-16
作者简介:柳长源(1970—),男,副教授,硕士生导师,邮箱:liuchangyuan@hrbust.edu.cn。
基金资助:
国家自然科学基金项目(51779050);黑龙江省自然科学基金项目(F2016022)

Driver Distraction Recognition Using Bilinear Fusion Networks

LIU Changyuan¹, HU Haoyuan¹, BI Xiaojun²

1. School of Measurement and Control Technology and Communication Engineering, Harbin University of Science and Technology, Harbin 150080, China;
2. School of Information Engineering, Minzu University of China, Beijing 100081, China

Received:2021-08-08 Published:2021-12-16

摘要/Abstract

摘要： 准确识别驾驶员的分心行为能够从源头上减少交通事故的发生。传统的识别方法类别少,准确率不高,对此,引入并改进残差神经网络(ResNet-50)对驾驶员分心行为进行识别。为了进一步提高模型特征的提取能力,从改进后的ResNet-50和EfficientNet-B0模型中提取特征,并将其双线性融合,从而进一步提高模型的识别准确率。通过对正常驾驶、玩手机、打电话、喝水、向后座拿东西、与副驾交谈6种驾驶员的行为进行测试,改进后ResNet-50模型的平均识别准确率达94.2%。将改进后的ResNet-50与EfficientNet-B0模型进行融合,融合模型的平均识别准确率高达96.7%。实验结果表明,该方法对驾驶员分心行为的检测有较好的分类效果。

关键词: 分心行为识别, 深度学习, ResNet-50, 双线性融合

Abstract: Accurate recognition of driver's distraction behavior can radically reduce the traffic accidents. Traditional recognition methods have the problems of few classification categories and low accuracy. To solve these problems, residual neural network (ResNet-50) is employed to recognize driver's distraction behavior and improve the network. To further improve the feature extraction ability of the model, bilinear fusion is carried out on the features extracted from the improved ResNet-50 model and EfficientNet-B0 model. Thus, the recognition accuracy of the model can be further improved. The average accuracy of the improved ResNet-50 single model is up to 94.2%, and the average accuracy of the model after fusing the improved ResNet-50 with EfficientNet-B0 is up to 96.7%. Experimental results show that this method has a good classification effect on the detection of driver's distraction behavior.

Key words: distracted behavior recognition, deep learning, ResNet-50, bilinear fusion

中图分类号:

TP391.4

柳长源, 虎浩媛, 毕晓君. 双线性融合网络的驾驶员分心行为识别[J]. 北京邮电大学学报, 2022, 45(2): 79-84.

LIU Changyuan, HU Haoyuan, BI Xiaojun. Driver Distraction Recognition Using Bilinear Fusion Networks[J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(2): 79-84.

参考文献

[1] 刘逸琛,孙华志,马春梅,等.一种基于高层特征融合的网络商品分类[J].北京邮电大学学报, 2020, 43(5):98-104, 117. LIU Y C, SUN H Z, MA C M, et al. Commodity classification of online based on high-level feature fusion[J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(5):98-104, 117.
[2] YAN C, COENEN F, YUE Y, et al. Video-based classification of driving behavior using a hierarchical classification system with multiple features[J]. International Journal of Pattern Recognition and Artificial Intelligence, 2016, 30(5):1-33.
[3] 田文洪,曾柯铭,莫中勤,等.基于卷积神经网络的驾驶员不安全行为识别[J].电子科技大学学报, 2019, 48(3):381-387. TIAN W H, ZENG K M, MO Z Q, et al. Recognition of unsafe driving behaviors based on convolutional neural network[J]. Journal of University of Electronic Science and Technology of China, 2019, 48(3):381-387.
[4] YANG X, TANG J L, LIU H, et al. End-to-end driving activities and secondary tasks recognition using deep convolutional neural network and transfer learning[C]//2018 IEEE Intelligent Vehicles Symposium. Piscatway, NJ:IEEE Press, 2018:1626-1631.
[5] LE T H N, ZHENG Y T, ZHU C C, et al. Multiple scale faster-RCNN approach to driver's cell-phone usage and hands on steering wheel detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, NJ:IEEE Press, 2016:46-53.
[6] 熊群芳,林军,岳伟,等.基于深度学习的驾驶员打电话行为检测方法[J].控制与信息技术, 2019(6):53-56, 62. XIONG Q F, LIN J, YUE W. A driver's calling behavior detection method based on deep learning[J]. Control and Information Technology, 2019(6):53-56, 62.
[7] HE K M, ZHANG X G, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ:IEEE Press, 2016:770-778.
[8] 张一帆.基于深度学习的驾驶员分心行为识别研究[D].哈尔滨:哈尔滨理工大学, 2021. ZHANG Y F. Research on driver distraction behavior recognition based on deep learning[D]. Harbin:Harbin University of Science and Technology, 2021.
[9] TAN M X, LE Q V. EfficientNet:rethinking model scaling for convolutional neural networks[C]//International Conference on Machine Learning. Lugano:ICML, 2019:6105-6114.
[10] LIN T Y, ROYCHOWDHURY A, MAJI S. Bilinear convolutional neural networks for fine-grained visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(6):1309-1322.

[1]	刘阳, 滕颖蕾, 牛涛, 郅佳琳. 基于深度强化学习的滤波器剪枝方案[J]. 北京邮电大学学报, 2023, 46(3): 31-36.
[2]	靳梦凡, 黄智濒, 储志强. 复杂环境下多模态指导的点云补全方法[J]. 北京邮电大学学报, 2023, 46(3): 103-108.
[3]	荣震宇刘建毅. 基于Transformer和MLP的眼底血管分割算法[J]. 北京邮电大学学报, 2023, 46(1): 26-31.
[4]	刘金桐, 杨国兴, 刘晓鸿, 王光宇. 针对心电图分类模型的平滑攻击算法[J]. 北京邮电大学学报, 2022, 45(4): 44-50.
[5]	黄柳婷, 刘可欣, 牛凯, 常春, 贺志强. 基于深度学习的哮喘患者CT影像黏液栓自动识别[J]. 北京邮电大学学报, 2022, 45(4): 58-63.
[6]	赵海英, 朱会, 侯小刚. 基于改进EMA单元的传统服饰图像语义分割[J]. 北京邮电大学学报, 2022, 45(1): 69-74.
[7]	张滨宇, 赵衍运, 杜昀昊, 万俊峰, 佟知航. 一种基于深度学习的PCB图像字符检测方法[J]. 北京邮电大学学报, 2022, 45(1): 108-114.
[8]	贾珺, 冯春燕, 夏海轮, 张天魁, 李成钢. 基于样本均衡与特征交互的通信网络故障预测方法[J]. 北京邮电大学学报, 2021, 44(6): 59-66.
[9]	普运伟, 郭江, 刘涛涛, 吴海潇. 基于多学习单元卷积神经网络的雷达辐射源信号识别[J]. 北京邮电大学学报, 2021, 44(6): 74-82.
[10]	王艺霏, 莫爽, 吴文睿, 范少华, 肖丁. 基于内外卷积网络的网络入侵检测[J]. 北京邮电大学学报, 2021, 44(5): 94-100.
[11]	高慧, 张继威, 来扬, 王文东. 深度学习的人体图像半自动标注系统[J]. 北京邮电大学学报, 2021, 44(1): 104-109.
[12]	蒲悦逸, 王文涵, 朱强, 陈朋朋. 基于CNN-ResNet-LSTM模型的城市短时交通流量预测算法[J]. 北京邮电大学学报, 2020, 43(5): 9-14.
[13]	马向亮, 李冰, 杨丹, 黄克振, 段晓毅. 基于深度学习的类SM4算法S盒逆向分析[J]. 北京邮电大学学报, 2020, 43(5): 118-124.
[14]	张天麒, 康波, 孟祥飞, 刘奕琳, 周颖. 基于U-Net的颅内出血识别算法[J]. 北京邮电大学学报, 2020, 43(3): 92-98.
[15]	马素刚, 赵祥模, 侯志强, 王忠民, 孙韩林. 一种基于ResNet网络特征的视觉目标跟踪算法[J]. 北京邮电大学学报, 2020, 43(2): 129-134.

双线性融合网络的驾驶员分心行为识别

Driver Distraction Recognition Using Bilinear Fusion Networks

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价