A Split Sliding Window-Based Continuous Chinese Sign Language Recognition System

doi:10.13190/j.jbupt.2021-001

Journal of Beijing University of Posts and Telecommunications ›› 2021, Vol. 44 ›› Issue (5): 48-54.doi: 10.13190/j.jbupt.2021-001

• PAPERS • Previous Articles Next Articles

A Split Sliding Window-Based Continuous Chinese Sign Language Recognition System

WANG Xin-yan¹, WANG Qing-shan¹, MA Xiao-di¹, LIU Peng², DAI Hai-peng³

1. Institute of Mathematics, Hefei University of Technology, Hefei 230601, China;
2. School of Computer Science, Hangzhou Dianzi University, Hangzhou 310018, China;
3. School of Computer Science and Technology, Nanjing University, Nanjing 210023, China

Received:2021-03-26 Online:2021-10-28 Published:2021-09-06

Abstract

Abstract: A large proportion of the world's disabled population is accounted for the individuals with hearing impairment which can communicate with people through the sign language. However, sign language is not mastered by the public, and there are still big obstacles between the individuals with hearing impairment and the normal people. A continuous Chinese sign language recognition system based on split sli-ding window (SSW) to realize automatic sign language recognition is proposed. The SSW system divides the sign language signal selected through the sliding window, and deletes one group of data to get new data in the original order, which is inputted to the sign language recognition neural network for training to obtain the gesture prediction value of a single sign language word. Finally, the majority voting strategy based on threshold is used to judge the identified prediction values. The SSW system is trained on 30 sign language sentences collected by 20 volunteers. The results show that the average accuracy of the SSW system reachs 83.9% on the test dataset, which is 16.7% higher than the long short-term memory model.

Key words: sliding window, bi-directional long short-term memory network, threshold, data segmentation, sign language recognition

CLC Number:

TP183

WANG Xin-yan, WANG Qing-shan, MA Xiao-di, LIU Peng, DAI Hai-peng. A Split Sliding Window-Based Continuous Chinese Sign Language Recognition System[J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(5): 48-54.

References

[1] Mellon N, Niparko J K, Rathmann C, et al. Should all deaf children learn sign language[J]. Pediatrics, 2015, 136(1):170-176.
[2] Koller O, Zargaran S, Ney H, et al. Deep sign:hybrid CNN-HMM for continuous sign language recognition[C]//BMVC 2016. York:BMVC, 2016:1-12.
[3] Koller O, Zargaran S, Ney H, et al. Re-sign:re-aligned end-to-end sequence modelling with deep recurrent CNN-HMMs[C]//CVPR 2017. Hawaii:IEEE Press, 2017:3416-3424.
[4] Cui Runpeng, Liu Hu, Zhang Changshui, et al. Recu-rrent convolutional neural networks for continuous sign language recognition by staged optimization[C]//CVPR 2017. Hawaii:IEEE Press, 2017:1610-1618.
[5] Yang Wenwen, Tao Jinxu, Ye Zhongfu, et al. Conti-nuous sign language recognition using level building based on fast hidden Markov model[J]. Pattern Recognition Letters, 2016, 78(15):28-35.
[6] Huang Jie, Zhou Wenggang, Zhang Qilin, et al. Video-based sign language recognition without temporal segmentation[C]//AAAI-18. Louisiana:AAAI Press, 2018:2257-2264.
[7] Guo Dan, Zhou Wengang, Wang Meng, et al. Hierarchical LSTM for sign language translation[C]//AAAI-18. Louisiana:AAAI Press, 2018:6845-6852.
[8] Zhang Jihai, Zhou Wengang, Li Houqiang, et al. A threshold-based HMM-DTW approach for continuous sign language recognition[C]//ICIMCS'14. New York:ACM Press, 2014:237-240.
[9] Li Kehuang, Zhou Zhengyu, Lee C H. Sign transition modeling and a scalable solution to continuous sign language recognition for real-world applications[J]. ACM Transactions on Accessible Computing, 2016, 8(2):1-23.
[10] Bukhari J, Rehman M, Malik S I, et al. American sign language translation through sensory glove; signspeak[J]. International Journal of u- and e- Service, Science and Technology, 2015, 8(1):131-142.
[11] Kamal S M, Chen Y, Li S, et al. Technical approaches to Chinese sign language processing:a review[J]. IEEE Access, 2019, 7:96926-96935.
[12] Gao Wen, Fang Gaolin, Zhao Debin, et al. A Chinese sign language recognition system based on SOFM/SRN/HMM[J]. Pattern Recognition, 2004, 37(12):2389-2402.
[13] Benalcazar M E, Jaramillo A G, Zea J A, et al. Hand gesture recognition using machine learning and the Myo armband[C]//25th European Signal Processing Confe-rence. Kos island:Institute of Electrical and Electronics Engineers Incorporated, 2017:1040-1044.
[14] Yang Xidong, Chen Xiang, Cao Xiang, et al. Chinese sign language recognition based on an optimized tree-structure framework[J]. IEEE Journal of Biomedical and Health Informatics, 2017, 21(4):994-1004.
[15] Kaya E, Kumbasar T. Hand gesture recognition systems with the wearable Myo armband[C]//CEIT 2018. Istanbul:IEEE Press, 2018:1-6.
[16] Zhuang Yongjie, Lv Bo, Sheng Xinjun, et al. Towards Chinese sign language recognition using surface electromyography and accelerometers[C]//201724th International Conference on Mechatronics and Machine Vision in Practice. Albany:IEEE Press, 2017:1-5.
[17] Yu Yi, Chen Xiang, Cao Shuai, et al. Exploration of Chinese sign language recognition using wearable sensors based on deep belief net[J]. IEEE Journal of Biome-dical and Health Informatics, 2020, 24(5):1310-1320.
[18] Mittal A, Kumar P, Roy P P, et al. A modified LSTM model for continuous sign language recognition using leap motion[J]. IEEE Sensors Journal, 2019, 19(16):7056-7063.
[19] Gupta R, Jha N. Real-time continuous sign language classification using ensemble of windows[C]//ICACCS 2020. Copenhagen:IEEE Press, 2020:73-78.
[20] Wahid F, Tafreshi R, Langari R, et al. A multi-window majority voting strategy to improve hand gesture recognition accuracies using electromyography signal[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(2):427-436.
[21] Hou Jiahui, Li Xiangyang, Zhu Peide, et al. Signspeaker:a real-time, high-precision smartwatch-based sign language translator[C]//MobiCom'19. New York:ACM Press, 2019:1-15.

A Split Sliding Window-Based Continuous Chinese Sign Language Recognition System

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	LI Zhaobin, ZHANG Lu, ZHAO Hong, WEI Zhanzhen. Certificateless Threshold-Based Conditional Proxy Re-Encryption Scheme [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(1): 44-49,83.
[2]	YE Kang, ZHANG Shujun, GUO Qi, LI Hui, CUI Xuehong. Continuous Sign Language Recognition Based on CM-Transformer [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(5): 49-53,78.
[3]	SHI Rui, FENG Hua-min, YANG Yang, YUAN Feng, LIU Biao. Mobile Crowdsensing Scheme with Strong Privacy-Preserving [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(5): 114-120.
[4]	LIU Ning-chun, GAO Shuai, HOU Xin-di, GUO Xing-chang. A Data Access Control Scheme in Information-Centric Mobile Ad Hoc Networks [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(2): 54-60.
[5]	LI Zeng-peng, WANG Jiu-ru, ZHANG Wen-yin, MA Chun-guang. A Threshold Public Key Encryption via Dual Regev Scheme [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(4): 83-87.
[6]	ZHANG Jin-ling, CUI Tong, LIU Wei-bin. Adaptive Time Window Function Optimization Based Electrocardiography Feature Waveform Recognition [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2019, 42(4): 82-88.
[7]	WU Xin-zhong, XIA Ling-xiang, ZHANG Xu, ZHOU Cheng. Voice Activity Detection Method Based on MFPH [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2019, 42(2): 83-89.
[8]	QUI Xue-song, ZHANG Xun, SONG Yan-bin, ZHAO Bing, XU Si-ya. Two-Stage Traffic Anomaly Detection Method Based on Entropy and Linear Relation [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2018, 41(4): 56-62.
[9]	ZHA Ya-xing, LUO Shou-shan, LI Wei, BIAN Jian-chao. Dynamic Group Public Auditing Scheme for Shared Data on Attribute-Based Threshold Signature [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(5): 43-49.
[10]	NI Shui-ping, CHANG Hui-gang. Double-Threshold Joint Spectrum Sensing Based on Energy and Wavelet Transform [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(4): 117-121.
[11]	ZHANG Jin-ling, ZHENG Xin. Stationary Wavelet Transform and New Threshold Function Used in ECG Signal Denoising [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(1): 28-31.
[12]	WANG Hai-rui. WCET Estimation for Real-Time Communication System by the Module of GPD [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2016, 39(s1): 6-9.
[13]	TIAN Li-qin, LI Jun-jian, WU Ze-nan. Trust Evaluation of Web User Behavior with Weight Optimal Balance [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2016, 39(6): 99-103,130.
[14]	ZHAO Sheng-mei, TANG Wen-juan, ZHENG Bao-yu. Generalized Semi-Iterative Hard Thresholding Pursuit Algorithm and Its Application in Ghost Imaging [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2016, 39(1): 117-121.
[15]	CHANG Li-wei, ZHENG Shi-hui, XING You-song, JIN Lei, YANG Yi-xian. Verifiable Multi-Use Dynamic Threshold Large Secret Sharing Scheme [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(5): 86-90.