个性化图像检索和推荐的研究

doi:10.13190/j.jbupt.2017.03.002

北京邮电大学学报 ›› 2017, Vol. 40 ›› Issue (3): 19-30.doi: 10.13190/j.jbupt.2017.03.002

个性化图像检索和推荐的研究

冀振燕, 姚伟娜, 皮怀雨

北京交通大学软件学院, 北京 100044

收稿日期:2016-11-20 出版日期:2017-06-28 发布日期:2017-06-28
作者简介:冀振燕(1972-),女,副教授,Email:zhyji@bjtu.edu.cn.
基金资助:
国家自然科学基金项目（61272353）

Research on Personalized Image Retrieval and Recommendation

JI Zhen-yan, YAO Wei-na, PI Huai-yu

School of Software Engineering, Beijing Jiaotong University, Beijing 100044, China

Received:2016-11-20 Online:2017-06-28 Published:2017-06-28

摘要/Abstract

摘要： 为了解决信息过载的问题，个性化图像检索和推荐技术成为目前图像检索领域的新趋势，其不仅可提高检索的效率和准确率，还可满足用户的个性化需求.根据不同个性化信息的数据源，可将个性化图像检索和推荐分为基于内容的个性化图像检索和推荐与协同过滤个性化图像检索和推荐.对于基于内容的个性化图像检索和推荐，分析了用户兴趣获取、用户兴趣表示和个性化实现3个核心环节，并对所采用的关键技术进行了对比，指出了优缺点；对于协同过滤个性化图像检索和推荐，分析了基于用户、物品和模型的3种协同过滤方法.最后分析对比了基于内容和协同过滤2种个性化图像检索和推荐方法，并指出了未来的工作方向.

关键词: 个性化图像检索, 图像推荐, 基于内容, 协同过滤

Abstract: In order to solve the problem of information overload, personalized image retrieval and recommendation technology has become the new trend in image retrieval area. It can not only improve the efficiency and accuracy of retrieval, but also meet users' personalized requirements. Personalized image retrieval and recommendation can be divided into content-based personalization and collaborative-filtering-based personalization according to different sources of personalized information. Firstly the development of image retrieval are summarized. Then the key technologies of content-based personalized image retrieval and recommendation are analyzed from three aspects, user interest acquisition, user interest representation and personalization implementation. The key technologies are compared. Their advantages and disadvantages are pointed out. For collaborative-filtering-based personalized image retrieval and recommendation, user-based, item-based and model-based collaborative filtering methods are contrasted. At the end of the paper, the content-based methods and the collaborative filtering methods are discussed, and the future work is shaped.

Key words: personalized image retrieval, image recommendation, content-based, collaborative filtering

中图分类号:

TN929.53

冀振燕, 姚伟娜, 皮怀雨. 个性化图像检索和推荐的研究[J]. 北京邮电大学学报, 2017, 40(3): 19-30.

JI Zhen-yan, YAO Wei-na, PI Huai-yu. Research on Personalized Image Retrieval and Recommendation[J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(3): 19-30.

参考文献

[1] Datta R, Joshi D, Li Jia, et al. Image retrieval:ideas, influences, and trends of the new age[J]. ACM Computing Surveys, 2008, 40(2):1-60.
[2] Hiremath P, Pujari J. Content based image retrieval using color, texture and shape features[C]//International Conference on Advanced Computing and Communications. New York:IEEE, 2007:780-784.
[3] 陈慧婷, 覃团发, 唐振华, 等. 综合纹理统计模型与全局主颜色的图像检索方法[J]. 北京邮电大学学报, 2011, 34(S1):100-103. Chen Huiting, Qin Tuanfa, Tang Zhenhua, et al. A method of image retrievals based on texture probability statistics and global dominant color[J]. Journal of Beijing University of Posts and Telecommunications, 2011, 34(S1):100-103.
[4] 庄凌, 庄越挺, 吴江琴, 等. 一种基于稀疏典型性相关分析的图像检索方法[J]. 软件学报, 2012, 23(5):1295-1304. Zhuang Ling, Zhuang Yueting, Wu Jiangqin, et al. Image retrieval approach based on sparse canonical correlation analysis[J]. Journal of Software, 2012, 23(5):1295-1304.
[5] Sarwar S, Qayyum Z U, Majeed S. Ontology based image retrieval framework using qualitative semantic image descriptions[J]. Procedia Computer Science, 2013, 22:285-294.
[6] Liu Ying, Zhang Dengsheng, Lu Guojun, et al. A survey of content-based image retrieval with high-level semantics[J]. Pattern Recognition, 2007, 40(1):262-282.
[7] 朱道广, 郭志刚, 赵永威. 基于空间上下文加权词汇树的图像检索方法[J]. 模式识别与人工智能, 2013, 26(11):1050-1056. Zhu Daoguang, Guo Zhigang, Zhao Yongwei. Image retrieval with spatial context weighting based vocabulary tree[J]. Pattern Recognition and Artificial Intelligence, 2013, 26(11):1050-1056.
[8] Zhou Zhihua, Zhang Minling, Huang Shengjun, et al. Multi-instance multi-label learning[J]. Artificial Intelligence, 2008, 176(1):2291-2320.
[9] Wu Qingyao, Ng M K, Ye Yunming. Markov-miml:a markov chain-based multi-instance multi-label learning algorithm[J]. Knowledge and Information Systems, 2013, 37(1):83-104.
[10] 李志欣, 施智平, 陈宏朝, 等. 基于语义学习的图像多模态检索[J]. 计算机工程, 2013, 39(3):258-263. Li Zhixin, Shi Zhiping, Chen Hongchao, et al. Multi-model image retrieval based on semantic learning[J]. Computer Engineering, 2013, 39(3):258-263.
[11] Pereira J C, Coviello E, Doyle G, et al. On the role of correlation and abstraction in cross-modal multimedia retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(3):521-535.
[12] Feng Fangxiang, Li Ruifan, Wang Xiaojie. Deep correspondence restricted Boltzmann machine for cross-modal retrieval[J]. Neurocomputing, 2015, 154:50-60.
[13] Jiang Qingyuan, Li Wujun. Deep cross-modal hashing[EB/OL]. arXiv preprint, 2016[2016-10]. https://arxiv.org/abs/1602.02255.
[14] Cao Yue, Long Mingsheng, Wang Jianmin, et al. Deep visual-semantic Hashing for cross-modal retrieval[C]//The 22^nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco:ACM, 2016:1445-1454.
[15] Ghorab M R, Zhou Dong, O'connor A, et al. Personalised information retrieval:survey and classification[J]. User Modeling and User-Adapted Interaction, 2013, 23(4):381-443.
[16] Skowron M, Tkal M, Ferwerda B, et al. Fusing social media cues:personality prediction from Twitter and instagram[C]//The 25^th International Conference Companion on World Wide Web. Montréal:International World Wide Web Conferences Steering Committee, 2016:107-108.
[17] Sang Jitao, Xu Changsheng, Lu Dongyuan. Learn to personalized image search from the photo sharing websites[J]. IEEE Transactions on Multimedia, 2011, 14(4):963-974.
[18] Liu Dong, Hua Xiansheng, Wang Meng, et al. Boost search relevance for tag-based social image retrieval[C]//IEEE International Conference on Multimedia and Expo. New York:IEEE, 2009:1636-1639.
[19] Cheung M, She James. Bag-of-features tagging approach for a better recommendation with social big data[C]//The 4^th International Conference on Advances in Information Mining and Management. Berlin:Springer, 2014:83-88.
[20] 邱兆文, 张田文. 基于用户多媒体数据管理模型的个性化图像检索[J]. 电子学报, 2008, 36(9):1746-1749. Qiu Zhaowen, Zhang Tianwen. Individuation image retrieval based on user multimedia data management model[J]. Acta Electronica Sinica, 2008, 36(9):1746-1749.
[21] Fan Jianping, Keim D A, Gao Yuli, et al. JustClick:personalized image recommendation via exploratory search from large-scale flickr images[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2009, 19(2):273-288.
[22] Yu Jun, Tao Dacheng, Wang Meng, et al. Learning to rank using user clicks and visual features for image retrieval[J]. IEEE Transactions on Cybernetics, 2015, 45(4):767-779.
[23] 李志欣, 施志平, 李志清, 等. 图像检索中语义映射方法综述[J]. 计算机辅助设计与图形学学报, 2008, 20(8):1085-1096. Li Zhixin, Shi Zhiping, Li Zhiqing, et al. A survey of semantic mapping in image retrieval[J]. Journal of Computer-Aided Design & Computer Graphics, 2008, 20(8):1085-1096.
[24] Zhang Hanwang, Zha Zhengjun, Yang Yang, et al. Attribute-augmented semantic hierarchy:towards bridging semantic gap and intention gap in image retrieval[C]//The 21^st ACM International Conference on Multimedia. New York:ACM, 2013:33-42.
[25] Jayech K, Mahjoub M A. New approach using Bayesian network to improve content based image classification systems[J]. International Journal of Computer Science Issues, 2012, 7(6):53-62.
[26] Kurtz C, Depeursinge A, Napel S, et al. On combining image-based and ontological semantic dissimilarities for medical image retrieval applications[J]. Medical Image Analysis, 2014, 18(7):1082-1100.
[27] Lin Chuenhorng, Chen Chiehchun, Lee H L, et al. Fast k-means algorithm based on a level histogram for image retrieval[J]. Expert Systems with Applications, 2014, 41(7):3276-3283.
[28] Deng Jia, Dong Wei, Socher R, et al. Imagenet:a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami:IEEE Press, 2009:248-255.
[29] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. Nevada:[s.n.], 2012:1097-1105.
[30] Karpathy A, Li Feifei. Deep visual-semantic alignments for generating image descriptions[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE Press, 2015:3128-3137.
[31] Xia Rongkai, Pan Yan, Lai Hanjiang, et al. Supervised Hashing for image retrieval via image representation learning[C]//The 28^th AAAI Conference on Artificial Intelligence. Québec City:AAAI Press, 2014:1-2.
[32] Zhao Fang, Huang Yongzhen, Wang Liang, et al. Deep semantic ranking based Hashing for multi-label image retrieval[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE Press, 2015:1556-1564.
[33] Su Jahwung, Huang Weijyun, Yu Philip S, et al. Efficient relevance feedback for content-based image retrieval by mining user navigation patterns[J]. IEEE Transactions on Knowledge & Data Engineering, 2011, 23(3):360-372.
[34] Adomavicius G, Tuzhilin A. Toward the next generation of recommender systems:a survey of the state-of-the-art and possible extensions[J]. IEEE Transactions on Knowledge & Data Engineering, 2005, 17(6):734-749.
[35] Paik J H. A novel TF-IDF weighting scheme for effective ranking[C]//The 36^th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM, 2013:343-352.
[36] Whissell J S, Clarke C L. Improving document clustering using Okapi BM25 feature weighting[J]. Information Retrieval Journal, 2011, 14(5):466-487.
[37] Zhang Jing, Zhou Li, Shen Lansun, et al. A personalized image retrieval based on user interest model[J]. International Journal of Pattern Recognition & Artificial Intelligence, 2010, 24(3):401-419.
[38] Zhou Dong, Lawless S, Wade V. Improving search via personalized query expansion using social media[J]. Information Retrieval, 2012, 15(3-4):218-242.
[39] Zhang Yin, Jin Rong, Zhou Zhihua. Understanding bag-of-words model:a statistical framework[J]. International Journal of Machine Learning & Cybernetics, 2010, 1(1):43-52.
[40] Tu Nguyenanh, Dinh D L, Rasel M K, et al. Topic modeling and improvement of image representation for large-scale image retrieval[J]. Information Sciences, 2016, 366:99-120.
[41] Shekhar R, Jawahar C V. Word image retrieval using bag of visual words[C]//2012 IEEE 10^th IAPR International Workshop on Document Analysis Systems. New Jersey:IEEE Press, 2012:297-301.
[42] Yang Jun, Jiang Yugang, Hauptmann A G, et al. Evaluating bag-of-visual-words representations in scene classification[C]//The International Workshop on Multimedia Information Retrieval. New York:ACM, 2007:197-206.
[43] Liu Lei. Contextual topic model based image recommendation system[C]//IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology. New Jersey:IEEE Press, 2015:239-240.
[44] Deng Jia, Berg A C, Li Feifei. Hierarchical semantic indexing for large scale image retrieval[C]//IEEE Conference on Computer Vision and Pattern Recognition. New Jersey:IEEE Press, 2011:785-792.
[45] Jiang Xing, Tan An-hwee. Learning and inferencing in user ontology for personalized Semantic web search[J]. Information Sciences, 2009, 179(16):2794-2808.
[46] Wan Ji, Wang Dayong, Hoi S C H, et al. Deep learning for content-based image retrieval:a comprehensive study[C]//The 22^nd ACM International Conference on Multimedia. Orlando:ACM, 2014:157-166.
[47] Geng Xue, Zhang Hanwang, Bian Jingwen, et al. Learning image and user features for recommendation in social networks[C]//IEEE International Conference on Computer Vision. Kerkyra:IEEE Press, 2015:4274-4282.
[48] Lei Chenyi, Liu Dong, Li Weiping, et al. Comparative deep learning of hybrid representations for image recommendations[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE Press, 2016:2545-2553.
[49] Burdescu D D, Mihai C G, Stanescu L, et al. Automatic image annotation and semantic based image retrieval for medical domain[J]. Neurocomputing, 2013, 109(8):33-48.
[50] Kurtz C, Beaulieu C F, Napel S, et al. A hierarchical knowledge-based approach for retrieving similar medical images described with semantic annotations[J]. Journal of Biomedical Informatics, 2014, 49(C):227-244.
[51] Kovashka A, Grauman K. Attribute pivots for guiding relevance feedback in image search[C]//IEEE International Conference on Computer Vision. Las Vegas:IEEE Press, 2013:297-304.
[52] Tong Simon, Chang Edward. Support vector machine active learning for image retrieval[C]//The 9^th ACM International Conference on Multimedia. New York:ACM, 2001:107-118.
[53] Rajendran P, Madheswaran M. Hybrid medical image classification using association rule mining with decision tree algorithm[J]. Computer Science, 2010, 3(10):1173-1178.
[54] 杨恒宇, 李慧宗, 林耀进, 等. 协同过滤中有影响力近邻的选择[J]. 北京邮电大学学报, 2016, 39(1):29-34. Yang Hengyu, Li Huizong, Lin Yaojin, et al. Influential neighbor selection in collaborative filtering[J]. Journal of Beijing University of Post and Telecommunications, 2016, 39(1):29-34.
[55] Zhao Shiwan, Du Nan, Nauerz A, et al. Improved recommendation based on collaborative tagging behaviors[C]//The 13^th International Conference on Intelligent User Interfaces. New York:ACM, 2008:413-416.
[56] Sarwar B, Karypis G, Konstan J, et al. Item-based collaborative filtering recommendation algorithms[C]//The 10^th International Conference on World Wide Web. New York:ACM, 2001:285-295.
[57] Zhou Ke, Yang Shuanghong, Zha Hongyuan. Functional matrix factorizations for cold-start recommendation[C]//The 34^th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM, 2011:315-324.
[58] Thorat P B, Goudar R M, Barve S. Survey on collaborative filtering, content-based filtering and hybrid recommendation system[J]. International Journal of Computer Applications, 2015, 110(4):31-36.
[59] Ma Zhanyu, Leijon A. A model-based collaborative filtering method for bounded support data[C]//IEEE International Conference on Network Infrastructure and Digital Content. Beijing:IEEE Press, 2012:545-548.
[60] Fernández-Tobías I, Braunhofer M, Elahi M, et al. Alleviating the new user problem in collaborative filtering by exploiting personality information[J]. User Modeling and User-Adapted Interaction, 2016, 26(2-3):1-35.
[61] Yang Chunfeng, Zhou Yipeng, Chen Liang, et al. Social-group-based ranking algorithms for cold-start video recommendation[J]. International Journal of Data Science & Analytics, 2016, 1(3-4):165-175.
[62] Candillier L, Meyer F, Boull M. Comparing state-of-the-art collaborative filtering systems[C]//International Workshop on Machine Learning and Data Mining in Pattern Recognition. Berlin Heidelberg:Springer, 2007:548-562
[63] Sanchez F, Barrilero M, Uribe S, et al. Social and content hybrid image recommender system for mobile social networks[J]. Mobile Networks & Applications, 2012, 17(6):782-795.
[64] Lekakos G, Caravelas P. A hybrid approach for movie recommendation[J]. Multimedia Tools & Applications, 2008, 36(1-2):55-70.
[65] Widisinghe A, Ranasinghe D, Kulathilaka K, et al. PicSeek:collaborative filtering for context-based image recommendation[C]//International Conference on Information and Automation for Sustainability. Colombo:IEEE Press, 2010:225-232.
[66] Liu Xianming, Tsai M H, Huang Thomas. Analyzing user preference for social image recommendation[EB/OL]. arXiv preprint, 2016[2016-10-10]. https://arxiv.org/abs/1604.07044.

个性化图像检索和推荐的研究

Research on Personalized Image Retrieval and Recommendation

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	王永, 尹恩民, 冉珣. 基于BC聚类的差分隐私保护推荐算法[J]. 北京邮电大学学报, 2021, 44(2): 81-88.
[2]	游思晴, 周丽, 赵东杰, 薛菲. 基于粒子群优化算法的协同过滤推荐并行化研究[J]. 北京邮电大学学报, 2018, 41(6): 115-122.
[3]	王永, 邓江洲. 基于KL散度的用户相似性协同过滤算法[J]. 北京邮电大学学报, 2017, 40(2): 110-114.
[4]	王晓军, 付超. 利用模糊分块改进协同过滤的扩展性和准确性[J]. 北京邮电大学学报, 2017, 40(1): 74-78.
[5]	唐瑞春, 方丽华, 庞敏, 许晓伟, 丁香乾. QoS预测机制的个性化云媒体服务选择[J]. 北京邮电大学学报, 2016, 39(6): 42-46.
[6]	王晓军. 推荐系统中分布式混合协同过滤方法[J]. 北京邮电大学学报, 2016, 39(2): 25-29.
[7]	杨恒宇, 李慧宗, 林耀进, 张佳. 协同过滤中有影响力近邻的选择[J]. 北京邮电大学学报, 2016, 39(1): 29-34.
[8]	李钊, 卢苇, 邢薇薇, 孙占全, 王伟东, 魏云超. CNN视觉特征的图像检索[J]. 北京邮电大学学报, 2015, 38(s1): 103-106,120.
[9]	徐雅斌, 孙晓晨. 位置社交网络的个性化位置推荐[J]. 北京邮电大学学报, 2015, 38(5): 118-124.
[10]	马华, 胡志刚. 客户端上下文感知的Web服务QoS预测方法[J]. 北京邮电大学学报, 2015, 38(4): 91-98.
[11]	王晓军. 利用项目属性和偏好改进协同过滤推荐[J]. 北京邮电大学学报, 2014, 37(6): 68-71,76.
[12]	鄂海红, 宋美娜, 李川, 江周峰. 结合时间上下文挖掘学习兴趣的协同过滤推荐算法[J]. 北京邮电大学学报, 2014, 37(6): 49-53.
[13]	周超, 李博. 一种基于用户信任网络的推荐方法[J]. 北京邮电大学学报, 2014, 37(4): 98-102.
[14]	赵珊，汤永利，刘静. 基于DCT系数空间分布的JPEG图像检索算法[J]. 北京邮电大学学报, 2009, 32(5): 32-35.
[15]	赵珊1,2　　刘静1. DCT压缩域的图像检索技术[J]. 北京邮电大学学报, 2008, 31(5): 5-8.