Architecture and Algorithms of Intelligent Dialogue System

doi:10.13190/j.jbupt.2019-169

Journal of Beijing University of Posts and Telecommunications ›› 2019, Vol. 42 ›› Issue (6): 10-19.doi: 10.13190/j.jbupt.2019-169

• REVIEW • Previous Articles Next Articles

Architecture and Algorithms of Intelligent Dialogue System

HUANG Yi, FENG Jun-lan, HU Min, WU Xiao-ting, DU Xiao-yu

China Mobile Research Institute, Beijing 100053, China

Received:2019-07-31 Published:2019-11-15

Abstract

Abstract: Intelligent human-machine dialogue system comprehensively utilizes a number of core technologies in the field of artificial intelligence. In recent years, with the development of basic algorithms such as deep learning and reinforcement learning, the overall structure, algorithm system and application mode of human-machine dialogue system have been changed and improved greatly. In order to sort out and summarize this technology, the architecture of human-machine dialogue system, the core algorithm of human-machine dialogue system, challenges and technical directions in this field are reviewed.

Key words: dialogue system, deep learning, human-machine interaction

CLC Number:

TP391.1

HUANG Yi, FENG Jun-lan, HU Min, WU Xiao-ting, DU Xiao-yu. Architecture and Algorithms of Intelligent Dialogue System[J]. Journal of Beijing University of Posts and Telecommunications, 2019, 42(6): 10-19.

References

[1] Turing Alan M. Computing machinery and intelligence[J]. Mind, 1950, LIX(236):433-460.
[2] 陈晨, 朱晴晴, 严睿, 等. 基于深度学习的开放领域对话系统研究综述[J]. 计算机学报, 2019, 42(7):1439-1466. Chen Chen, Zhu Qingqing, Yan Rui, et al. Survey on deep learning based open domain dialogue system[J]. Chinese Journal of Computers, 2019, 42(7):1439-1466.
[3] López-Cózar Ramón, Zoraida Callejas, Griol David, et al. Review of spoken dialogue systems[J]. Loquens, 2015, 1(2):2386-2637.
[4] Chen Hongshen, Liu Xiaorui, Yin Dawei, et al. A survey on dialogue systems:recent advances and new frontiers[J]. SIGKDD Explorations, 2017, 19(2):25-35.
[5] Young Steve, Gašić Milica, Thomson Blaise, et al. POMDP-based statistical spoken dialogue systems:a review[J]. Proceedings of the IEEE, 2013, 101(5):1160-1179.
[6] Lee Sungjin, Zhu Qi, Takanobu Ryuichi, et al. ConvLab:multi-domain end-to-end dialog system platform[C]//Proceedings of the 57^th Annual Meeting of the Association for Computational Linguistics(ACL). Florence:[s.n.], 2019:64-69.
[7] Wen Tsung-Hsien, Vandyke David, Mrkši Nikola, et al. A network-based end-to-end trainable task-oriented dialogue system[C]//European Chapter of the Association for Computational Linguistics(EACL). Valencia:[s. n.], 2017:438-449.
[8] Huang Po-Sen, He Xiaodong, Gao Jianfeng, et al. Learning deep structured semantic models for web search using click through data[C]//Proceedings of the 22^nd ACM International Conference on Information and Knowledge Management. Amazon:[s.n.], 2013:2333-2338.
[9] Shen Yelong, He Xiaodong, Gao Jianfeng, et al. A latent semantic model with convolutional-pooling structure for information retrieval[C]//Proceedings of the 23^rd ACM International Conference on Information and Knowledge Management(CIKM). NewYork:ACM Press, 2014:101-110.
[10] Shen Yelong, He Xiaodong, Gao Jianfeng, et al. Learning semantic representations using convolutional neural networks for web search[C]//Proceedings of the 23^rd International Conference on WWW. Seoul:[s.n.], 2014:373-374.
[11] Hu Baotian, Lu Zhengdong, Li Hang, et al. Convolutional neural network architectures for matching natural language sentences[C]//Proceedings of the Advances in Neural Information Processing System. Montreal:[s.n.], 2014:2042-2050.
[12] Tai Kai Sheng, Socher Richard, Manning Christopher D. Improved semantic representations from tree-structured long short-term memory networks[C]//Proceedings of the 53^rd Annual Meeting of the Association for Computational(ACL). Beijing:[s.n.], 2015:15-26.
[13] Yin Wenpeng, Schutze Hinrich, Xiang Bing, et al. ABCNN:attention-based convolutional neural network for modeling sentence[J]. arXiv preprint arXiv:1512. 05193.
[14] Pang Liang, Lan Yanyan, Guo Jiafeng, et al. Text matching as image recognition[C]//Proceedings of the 30^th AAAI Conference on Artificial Intelligence. Phoenix:[s.n.], 2016:2793-2799.
[15] Chen Qian, Zhu Xiaodan, Ling Zhenhua, et al. Enhanced LSTM for natural language inference[C]//Proceedings of the 55^th Annual Meeting of the Association for Computational Linguistics(ACL). Vancouver:[s.n.], 2017:17-29.
[16] Tur Gokhan, Deng Li, Hakkani-Tur Dilek, et al. Towards deeper understanding:deep convex networks for semantic utterance classification[C]//2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Kyoto:[s.n.], 2012:5045-5048.
[17] Lee Ji Young, Dernoncourt Franck. Sequential short-text classification with recurrent and convolutional neural networks[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. San Diego California:[s.n.], 2016:515-520.
[18] Liu Bing, Ian Lane. Attention-based recurrent neural network models for joint intent detection and slot filling[C]//Interspeech 2016, 17^th Annual Conference of the International Speech Communication Association. San Francisco:[s.n.], 2016:685-689.
[19] Goo Chih-Wen, Gao Guang, Hsu Yun-Kai, et al. Slot-gated modeling for joint slot filling and intent prediction[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. New Orleans:[s.n.], 2018:753-757.
[20] Aaron Jaech, Heck Larry, Ostendorf Mari. Domain adaptation of recurrent neural networks for natural language understanding[C]//Interspeech 2016, 17^th Annual Conference of the International Speech Communication Association. San Francisco:[s.n.], 2016:690-694.
[21] Franck Dernoncourt, Lee Ji Young, Bui Trung H. Robust dialog state tracking for large ontologies[M]. Singapore:Springer, 2016:475-485.
[22] Williams Jason, Young Steve. Partially observable Markov decision processes for spoken dialog systems[J]. Computer Speech & Language, 2007, 21(2):393-422.
[23] Mrkšić Nikola, Séaghdha Diarmuid Ó, Wen Tsung Hsien, et al. Neural belief tracker:data-driven dialogue state tracking[C]//Proceedings of the 55^th Annual Meeting of the Association for Computational Linguistics. Vancouver:[s.n.], 2017:1777-1788.
[24] Xu Puyang, Hu Qi. An end-to-end approach for handling unknown slot values in dialogue state tracking[C]//Proceedings of the 56^th Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia:[s.n.], 2018:1448-1457.
[25] Zhong Victor, Xiong Caiming, Socher Richard. Global-locally self-attentive dialogue state tracker[EB/OL]. 2018(2018-09-01)[2019-12-15]. http://arxiv.org/abs/1805.09655.
[26] Ren Liliang, Xie Kaige, Chen Lu, et al. Towards universal dialogue state tracking[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium:[s.n.], 2018:2780-2786.
[27] Williams Jason. Multi-domain learning and generalization in dialog state tracking[C]//Proceedings of the SIGDIAL 2013 Conference, the 14^th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Metz, France:[s.n.], 2013:433-441.
[28] Mrkšić Nikola, Séaghdha Diarmuid Ó, Thomson Blaise, et al. Multi-domain dialog state tracking using recurrent neural networks[C]//Proceedings of the 53^rd Annual Meeting of the Association for Computational Linguistics and the 7^th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing. Beijing, China:[s.n.], 2015:794-799.
[29] Zhou Keyan, Zong Chengqing, Wu Hua, et al. Predicting and tagging dialog-act using MDP and SVM[C]//2008 6^th International Symposium on Chinese Spoken Language Processing. Kunming:IEEE, 2008:293-296.
[30] Li Yuxi. Deep reinforcement learning:an overview[EB/OL]. 2017(2017-01-01)[2019-12-10]. http://arxiv.org/abs/1701.07274.
[31] Mnih Volodymyr, Kavukcuoglu Koray, Silver David, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540):529-533.
[32] Cuayáhuitl Heriberto. SimpleDS:a simple deep reinforcement learning dialogue system[J]. Springer, 2017(427):109-118.
[33] Cuayáhuitl Heriberto, Yu Seunghak, Williamson Ashley, et al. Scaling up deep reinforcement learning for multi-domain dialogue systems[C]//2017 International Joint Conference on Neural Networks (IJCNN). Anchorage, AK, USA:[s.n.], 2017:3339-3346.
[34] Ultes Stefan, Rojas-Barahona Lina, Su Pei-Hao, et al. Pydial:a multi-domain statistical dialogue system toolkit[C]//Proceedings of the 55^th Annual Meeting of the Association for Computational Linguistics-System Demonstrations. Canada:Vancouver, 2017:73-78.
[35] Casanueva Iñigo, Budzianowski Paweł, Su Pei-Hao, et al. A benchmarking environment for reinforcement learning based task oriented dialogue management[EB/OL]. 2018(2018-04-03)[2019-12-09]. http://arxiv.org/abs/1711.11023.
[36] Weisz Gellert, Budzianowski Pawel, Su Pei-Hao, et al. Sample efficient deep reinforcement learning for dialogue systems with large action spaces[J]. IEEE/ACM Transactions on Audio Speech and Language Processing, 2018, 26(11):2083-2097.
[37] Li Xiujun, Chen Yun-Nung, Li Lihong, et al. End-to-end task-completion neural dialogue systems[C]//Proceedings of the Eighth International Joint Conference on Natural Language Processing(IJCNLP). Taipei:[s.n.], 2017:733-743.
[38] Paulus Romain, Xiong Caiming, Socher Richard. A deep reinforced model for abstractive summarization[C]//6^th International Conference on Learning Representations(ICLR). Vancouver, BC, Canada:[s.n.], 2018.
[39] Budzianowski Paweł, Wen Tsung-hsien, Tseng Bo-hsiang, et al. MultiWOZ-A large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels:[s.n.], 2018:5016-5026.
[40] Zhao Tiancheng, Xie Kaige, Eskenazi M. Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies(NAACL-HLT). Minneapolis, MN, USA:[s.n.], 2019:1208-1218.
[41] Galley Michel, Brockett Chris, Gao Xiang, et al. End-to-end conversation modeling:Dstc7 task 2 description[C]//Dialog System Technology Challenges. Honolulu:[s.n.], 2018.
[42] Keneshloo Yaser, Shi Tian, Ramakrishnan Naren, et al. Deep reinforcement learning for sequence-to-sequence models[EB/OL]. 2019(2019-04-01)[2019-12-15]. http://arxiv.org/abs/1805.09461.
[43] Peters Matthew E, Neumann Mark, Iyyer Mohit, et al. Deep contextualized word representations[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies(NAACL-HLT). New Orleans, Louisiana, USA:[s.n.], 2018:2227-2237.
[44] Radford Alec, Narasimhan Karthik, Salimans Tim, et al. Improving language understanding by generative pre-training[EB/OL]. 2018(2018-05-08)[2019-09-23]. https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.
[45] Devlin Jacob, Chang Ming-Wei, Lee Kenton, et al. BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies(NAACL-HLT). Minneapolis, MN, USA:[s.n.], 2019:4171-4186.

Architecture and Algorithms of Intelligent Dialogue System

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	LIU Yang, TENG Yinglei, NIU Tao, ZHI Jialin. Filter Pruning Algorithm Based on Deep Reinforcement Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 31-36.
[2]	. Multi-Mode Guided Point Cloud Completion Method in Complex Environment [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(3): 103-108.
[3]	RONG Zhenyu, LIU Jianyi. Retinal Blood Vessel Segmentation Based on Transformer and MLP [J]. Journal of Beijing University of Posts and Telecommunications, 2023, 46(1): 26-31.
[4]	HUANG Hongcheng, KONG Tiantian, HU Min, TAO Yang, KOU Lan. Knowledge Driven Management Strategy of Human-Machine Active Dialogue [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(5): 109-114.
[5]	LIU Jintong, YANG Guoxing, LIU Xiaohong, WANG Guangyu. Smoothing Attack Algorithm Based on Electrocardiogram Classification [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(4): 44-50.
[6]	HUANG Liuting, LIU Kexin, NIU Kai, CHANG Chun, HE Zhiqiang. Automatic Recognition of Mucus Impaction in CT Images of Asthmatic Patients Using Deep Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(4): 58-63.
[7]	LIU Changyuan, HU Haoyuan, BI Xiaojun. Driver Distraction Recognition Using Bilinear Fusion Networks [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(2): 79-84.
[8]	ZHAO Haiying, ZHU Hui, HOU Xiaogang. Traditional Custume Image Semantic Segmentation Based on Improved EMA Unit [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(1): 69-74.
[9]	ZHANG Binyu, ZHAO Yanyun, DU Yunhao, WAN Junfeng, TONG Zhihang. Character Detection Method for PCB Image Based on Deep Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(1): 108-114.
[10]	JIA Jun, FENG Chun-yan, XIA Hai-lun, ZHANG Tian-kui, LI Cheng-gang. Communication Networks Fault Prediction Method Based on Sample Equalization and Feature Interaction [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(6): 59-66.
[11]	PU Yun-wei, GUO Jiang, LIU Tao-tao, WU Hai-xiao. A Recognition Method for Radar Emitter Signals Based on Convolutional Neural Network with Multiple Learning Units [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(6): 74-82.
[12]	WANG Yi-fei, MO Shuang, WU Wen-rui, FAN Shao-hua, XIAO Ding. Internal-External Convolutional Networks for Network Intrusion Detection [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(5): 94-100.
[13]	GAO Hui, ZHANG Ji-wei, LAI Yang, WANG Wen-dong. Deep Learning Based Semi-Automatic Labeling System for Human Images [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(1): 104-109.
[14]	PU Yue-yi, WANG Wen-han, ZHU Qiang, CHEN Peng-peng. Urban Short-Term Traffic Flow Prediction Algorithm Based on CNN-ResNet-LSTM Model [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(5): 9-14.
[15]	MA Xiang-liang, LI Bing, YANG Dan, HUANG Ke-zhen, DUAN Xiao-yi. Reverse-Analysis of S-Box for SM4-Like Algorithms Based on Side Channel Technology [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(5): 118-124.