北京邮电大学学报

  • EI核心期刊

北京邮电大学学报 ›› 2021, Vol. 44 ›› Issue (5): 48-54.doi: 10.13190/j.jbupt.2021-001

• 论文 • 上一篇    下一篇

一种基于滑动窗口分割的中国手语识别系统

王鑫炎1, 王青山1, 马晓迪1, 刘鹏2, 戴海鹏3   

  1. 1. 合肥工业大学 数学学院, 合肥 230601;
    2. 杭州电子科技大学 计算机学院, 杭州 310018;
    3. 南京大学 计算机科学与技术学院, 南京 210023
  • 收稿日期:2021-03-26 出版日期:2021-10-28 发布日期:2021-09-06
  • 通讯作者: 王青山(1975-),男,教授,E-mail:qswang@hfut.edu.cn. E-mail:qswang@hfut.edu.cn
  • 作者简介:王鑫炎(1998-),男,本科生.

A Split Sliding Window-Based Continuous Chinese Sign Language Recognition System

WANG Xin-yan1, WANG Qing-shan1, MA Xiao-di1, LIU Peng2, DAI Hai-peng3   

  1. 1. Institute of Mathematics, Hefei University of Technology, Hefei 230601, China;
    2. School of Computer Science, Hangzhou Dianzi University, Hangzhou 310018, China;
    3. School of Computer Science and Technology, Nanjing University, Nanjing 210023, China
  • Received:2021-03-26 Online:2021-10-28 Published:2021-09-06

摘要: 听力障碍者在全世界残疾人群体中占有较大的比重.他们能通过手语与健全人交流,但因手语不被大众所掌握,导致彼此交流存在较大障碍.为此提出了一种基于滑动窗口分割(SSW)的连续中国手语识别系统来实现手语自动识别.SSW系统将通过滑动窗口选取出来的手语信号平均分割,依次删去其中一组数据,从而得到新的数据,输入手语识别神经网络进行训练,得出单个手语单词手势预测值,最后运用基于阈值的多投票策略对识别出的预测值进行判断,得出识别结果.SSW系统在对20名志愿者采集的30条手语语句上进行训练,结果显示,所提SSW系统自动识别手语的平均准确率在测试集上达到83.9%,较长短期记忆网络模型提高了16.7%.

关键词: 滑动窗口, 双向长短期记忆网络, 阈值, 数据分割, 手语识别

Abstract: A large proportion of the world's disabled population is accounted for the individuals with hearing impairment which can communicate with people through the sign language. However, sign language is not mastered by the public, and there are still big obstacles between the individuals with hearing impairment and the normal people. A continuous Chinese sign language recognition system based on split sli-ding window (SSW) to realize automatic sign language recognition is proposed. The SSW system divides the sign language signal selected through the sliding window, and deletes one group of data to get new data in the original order, which is inputted to the sign language recognition neural network for training to obtain the gesture prediction value of a single sign language word. Finally, the majority voting strategy based on threshold is used to judge the identified prediction values. The SSW system is trained on 30 sign language sentences collected by 20 volunteers. The results show that the average accuracy of the SSW system reachs 83.9% on the test dataset, which is 16.7% higher than the long short-term memory model.

Key words: sliding window, bi-directional long short-term memory network, threshold, data segmentation, sign language recognition

中图分类号: