Research on Personalized Image Retrieval and Recommendation

doi:10.13190/j.jbupt.2017.03.002

JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM ›› 2017, Vol. 40 ›› Issue (3): 19-30.doi: 10.13190/j.jbupt.2017.03.002

• Review • Previous Articles Next Articles

Research on Personalized Image Retrieval and Recommendation

JI Zhen-yan, YAO Wei-na, PI Huai-yu

School of Software Engineering, Beijing Jiaotong University, Beijing 100044, China

Received:2016-11-20 Online:2017-06-28 Published:2017-06-28

Abstract

Abstract: In order to solve the problem of information overload, personalized image retrieval and recommendation technology has become the new trend in image retrieval area. It can not only improve the efficiency and accuracy of retrieval, but also meet users' personalized requirements. Personalized image retrieval and recommendation can be divided into content-based personalization and collaborative-filtering-based personalization according to different sources of personalized information. Firstly the development of image retrieval are summarized. Then the key technologies of content-based personalized image retrieval and recommendation are analyzed from three aspects, user interest acquisition, user interest representation and personalization implementation. The key technologies are compared. Their advantages and disadvantages are pointed out. For collaborative-filtering-based personalized image retrieval and recommendation, user-based, item-based and model-based collaborative filtering methods are contrasted. At the end of the paper, the content-based methods and the collaborative filtering methods are discussed, and the future work is shaped.

Key words: personalized image retrieval, image recommendation, content-based, collaborative filtering

CLC Number:

TN929.53

JI Zhen-yan, YAO Wei-na, PI Huai-yu. Research on Personalized Image Retrieval and Recommendation[J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(3): 19-30.

References

[1] Datta R, Joshi D, Li Jia, et al. Image retrieval:ideas, influences, and trends of the new age[J]. ACM Computing Surveys, 2008, 40(2):1-60.
[2] Hiremath P, Pujari J. Content based image retrieval using color, texture and shape features[C]//International Conference on Advanced Computing and Communications. New York:IEEE, 2007:780-784.
[3] 陈慧婷, 覃团发, 唐振华, 等. 综合纹理统计模型与全局主颜色的图像检索方法[J]. 北京邮电大学学报, 2011, 34(S1):100-103. Chen Huiting, Qin Tuanfa, Tang Zhenhua, et al. A method of image retrievals based on texture probability statistics and global dominant color[J]. Journal of Beijing University of Posts and Telecommunications, 2011, 34(S1):100-103.
[4] 庄凌, 庄越挺, 吴江琴, 等. 一种基于稀疏典型性相关分析的图像检索方法[J]. 软件学报, 2012, 23(5):1295-1304. Zhuang Ling, Zhuang Yueting, Wu Jiangqin, et al. Image retrieval approach based on sparse canonical correlation analysis[J]. Journal of Software, 2012, 23(5):1295-1304.
[5] Sarwar S, Qayyum Z U, Majeed S. Ontology based image retrieval framework using qualitative semantic image descriptions[J]. Procedia Computer Science, 2013, 22:285-294.
[6] Liu Ying, Zhang Dengsheng, Lu Guojun, et al. A survey of content-based image retrieval with high-level semantics[J]. Pattern Recognition, 2007, 40(1):262-282.
[7] 朱道广, 郭志刚, 赵永威. 基于空间上下文加权词汇树的图像检索方法[J]. 模式识别与人工智能, 2013, 26(11):1050-1056. Zhu Daoguang, Guo Zhigang, Zhao Yongwei. Image retrieval with spatial context weighting based vocabulary tree[J]. Pattern Recognition and Artificial Intelligence, 2013, 26(11):1050-1056.
[8] Zhou Zhihua, Zhang Minling, Huang Shengjun, et al. Multi-instance multi-label learning[J]. Artificial Intelligence, 2008, 176(1):2291-2320.
[9] Wu Qingyao, Ng M K, Ye Yunming. Markov-miml:a markov chain-based multi-instance multi-label learning algorithm[J]. Knowledge and Information Systems, 2013, 37(1):83-104.
[10] 李志欣, 施智平, 陈宏朝, 等. 基于语义学习的图像多模态检索[J]. 计算机工程, 2013, 39(3):258-263. Li Zhixin, Shi Zhiping, Chen Hongchao, et al. Multi-model image retrieval based on semantic learning[J]. Computer Engineering, 2013, 39(3):258-263.
[11] Pereira J C, Coviello E, Doyle G, et al. On the role of correlation and abstraction in cross-modal multimedia retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(3):521-535.
[12] Feng Fangxiang, Li Ruifan, Wang Xiaojie. Deep correspondence restricted Boltzmann machine for cross-modal retrieval[J]. Neurocomputing, 2015, 154:50-60.
[13] Jiang Qingyuan, Li Wujun. Deep cross-modal hashing[EB/OL]. arXiv preprint, 2016[2016-10]. https://arxiv.org/abs/1602.02255.
[14] Cao Yue, Long Mingsheng, Wang Jianmin, et al. Deep visual-semantic Hashing for cross-modal retrieval[C]//The 22^nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco:ACM, 2016:1445-1454.
[15] Ghorab M R, Zhou Dong, O'connor A, et al. Personalised information retrieval:survey and classification[J]. User Modeling and User-Adapted Interaction, 2013, 23(4):381-443.
[16] Skowron M, Tkal M, Ferwerda B, et al. Fusing social media cues:personality prediction from Twitter and instagram[C]//The 25^th International Conference Companion on World Wide Web. Montréal:International World Wide Web Conferences Steering Committee, 2016:107-108.
[17] Sang Jitao, Xu Changsheng, Lu Dongyuan. Learn to personalized image search from the photo sharing websites[J]. IEEE Transactions on Multimedia, 2011, 14(4):963-974.
[18] Liu Dong, Hua Xiansheng, Wang Meng, et al. Boost search relevance for tag-based social image retrieval[C]//IEEE International Conference on Multimedia and Expo. New York:IEEE, 2009:1636-1639.
[19] Cheung M, She James. Bag-of-features tagging approach for a better recommendation with social big data[C]//The 4^th International Conference on Advances in Information Mining and Management. Berlin:Springer, 2014:83-88.
[20] 邱兆文, 张田文. 基于用户多媒体数据管理模型的个性化图像检索[J]. 电子学报, 2008, 36(9):1746-1749. Qiu Zhaowen, Zhang Tianwen. Individuation image retrieval based on user multimedia data management model[J]. Acta Electronica Sinica, 2008, 36(9):1746-1749.
[21] Fan Jianping, Keim D A, Gao Yuli, et al. JustClick:personalized image recommendation via exploratory search from large-scale flickr images[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2009, 19(2):273-288.
[22] Yu Jun, Tao Dacheng, Wang Meng, et al. Learning to rank using user clicks and visual features for image retrieval[J]. IEEE Transactions on Cybernetics, 2015, 45(4):767-779.
[23] 李志欣, 施志平, 李志清, 等. 图像检索中语义映射方法综述[J]. 计算机辅助设计与图形学学报, 2008, 20(8):1085-1096. Li Zhixin, Shi Zhiping, Li Zhiqing, et al. A survey of semantic mapping in image retrieval[J]. Journal of Computer-Aided Design & Computer Graphics, 2008, 20(8):1085-1096.
[24] Zhang Hanwang, Zha Zhengjun, Yang Yang, et al. Attribute-augmented semantic hierarchy:towards bridging semantic gap and intention gap in image retrieval[C]//The 21^st ACM International Conference on Multimedia. New York:ACM, 2013:33-42.
[25] Jayech K, Mahjoub M A. New approach using Bayesian network to improve content based image classification systems[J]. International Journal of Computer Science Issues, 2012, 7(6):53-62.
[26] Kurtz C, Depeursinge A, Napel S, et al. On combining image-based and ontological semantic dissimilarities for medical image retrieval applications[J]. Medical Image Analysis, 2014, 18(7):1082-1100.
[27] Lin Chuenhorng, Chen Chiehchun, Lee H L, et al. Fast k-means algorithm based on a level histogram for image retrieval[J]. Expert Systems with Applications, 2014, 41(7):3276-3283.
[28] Deng Jia, Dong Wei, Socher R, et al. Imagenet:a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami:IEEE Press, 2009:248-255.
[29] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. Nevada:[s.n.], 2012:1097-1105.
[30] Karpathy A, Li Feifei. Deep visual-semantic alignments for generating image descriptions[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE Press, 2015:3128-3137.
[31] Xia Rongkai, Pan Yan, Lai Hanjiang, et al. Supervised Hashing for image retrieval via image representation learning[C]//The 28^th AAAI Conference on Artificial Intelligence. Québec City:AAAI Press, 2014:1-2.
[32] Zhao Fang, Huang Yongzhen, Wang Liang, et al. Deep semantic ranking based Hashing for multi-label image retrieval[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE Press, 2015:1556-1564.
[33] Su Jahwung, Huang Weijyun, Yu Philip S, et al. Efficient relevance feedback for content-based image retrieval by mining user navigation patterns[J]. IEEE Transactions on Knowledge & Data Engineering, 2011, 23(3):360-372.
[34] Adomavicius G, Tuzhilin A. Toward the next generation of recommender systems:a survey of the state-of-the-art and possible extensions[J]. IEEE Transactions on Knowledge & Data Engineering, 2005, 17(6):734-749.
[35] Paik J H. A novel TF-IDF weighting scheme for effective ranking[C]//The 36^th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM, 2013:343-352.
[36] Whissell J S, Clarke C L. Improving document clustering using Okapi BM25 feature weighting[J]. Information Retrieval Journal, 2011, 14(5):466-487.
[37] Zhang Jing, Zhou Li, Shen Lansun, et al. A personalized image retrieval based on user interest model[J]. International Journal of Pattern Recognition & Artificial Intelligence, 2010, 24(3):401-419.
[38] Zhou Dong, Lawless S, Wade V. Improving search via personalized query expansion using social media[J]. Information Retrieval, 2012, 15(3-4):218-242.
[39] Zhang Yin, Jin Rong, Zhou Zhihua. Understanding bag-of-words model:a statistical framework[J]. International Journal of Machine Learning & Cybernetics, 2010, 1(1):43-52.
[40] Tu Nguyenanh, Dinh D L, Rasel M K, et al. Topic modeling and improvement of image representation for large-scale image retrieval[J]. Information Sciences, 2016, 366:99-120.
[41] Shekhar R, Jawahar C V. Word image retrieval using bag of visual words[C]//2012 IEEE 10^th IAPR International Workshop on Document Analysis Systems. New Jersey:IEEE Press, 2012:297-301.
[42] Yang Jun, Jiang Yugang, Hauptmann A G, et al. Evaluating bag-of-visual-words representations in scene classification[C]//The International Workshop on Multimedia Information Retrieval. New York:ACM, 2007:197-206.
[43] Liu Lei. Contextual topic model based image recommendation system[C]//IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology. New Jersey:IEEE Press, 2015:239-240.
[44] Deng Jia, Berg A C, Li Feifei. Hierarchical semantic indexing for large scale image retrieval[C]//IEEE Conference on Computer Vision and Pattern Recognition. New Jersey:IEEE Press, 2011:785-792.
[45] Jiang Xing, Tan An-hwee. Learning and inferencing in user ontology for personalized Semantic web search[J]. Information Sciences, 2009, 179(16):2794-2808.
[46] Wan Ji, Wang Dayong, Hoi S C H, et al. Deep learning for content-based image retrieval:a comprehensive study[C]//The 22^nd ACM International Conference on Multimedia. Orlando:ACM, 2014:157-166.
[47] Geng Xue, Zhang Hanwang, Bian Jingwen, et al. Learning image and user features for recommendation in social networks[C]//IEEE International Conference on Computer Vision. Kerkyra:IEEE Press, 2015:4274-4282.
[48] Lei Chenyi, Liu Dong, Li Weiping, et al. Comparative deep learning of hybrid representations for image recommendations[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE Press, 2016:2545-2553.
[49] Burdescu D D, Mihai C G, Stanescu L, et al. Automatic image annotation and semantic based image retrieval for medical domain[J]. Neurocomputing, 2013, 109(8):33-48.
[50] Kurtz C, Beaulieu C F, Napel S, et al. A hierarchical knowledge-based approach for retrieving similar medical images described with semantic annotations[J]. Journal of Biomedical Informatics, 2014, 49(C):227-244.
[51] Kovashka A, Grauman K. Attribute pivots for guiding relevance feedback in image search[C]//IEEE International Conference on Computer Vision. Las Vegas:IEEE Press, 2013:297-304.
[52] Tong Simon, Chang Edward. Support vector machine active learning for image retrieval[C]//The 9^th ACM International Conference on Multimedia. New York:ACM, 2001:107-118.
[53] Rajendran P, Madheswaran M. Hybrid medical image classification using association rule mining with decision tree algorithm[J]. Computer Science, 2010, 3(10):1173-1178.
[54] 杨恒宇, 李慧宗, 林耀进, 等. 协同过滤中有影响力近邻的选择[J]. 北京邮电大学学报, 2016, 39(1):29-34. Yang Hengyu, Li Huizong, Lin Yaojin, et al. Influential neighbor selection in collaborative filtering[J]. Journal of Beijing University of Post and Telecommunications, 2016, 39(1):29-34.
[55] Zhao Shiwan, Du Nan, Nauerz A, et al. Improved recommendation based on collaborative tagging behaviors[C]//The 13^th International Conference on Intelligent User Interfaces. New York:ACM, 2008:413-416.
[56] Sarwar B, Karypis G, Konstan J, et al. Item-based collaborative filtering recommendation algorithms[C]//The 10^th International Conference on World Wide Web. New York:ACM, 2001:285-295.
[57] Zhou Ke, Yang Shuanghong, Zha Hongyuan. Functional matrix factorizations for cold-start recommendation[C]//The 34^th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM, 2011:315-324.
[58] Thorat P B, Goudar R M, Barve S. Survey on collaborative filtering, content-based filtering and hybrid recommendation system[J]. International Journal of Computer Applications, 2015, 110(4):31-36.
[59] Ma Zhanyu, Leijon A. A model-based collaborative filtering method for bounded support data[C]//IEEE International Conference on Network Infrastructure and Digital Content. Beijing:IEEE Press, 2012:545-548.
[60] Fernández-Tobías I, Braunhofer M, Elahi M, et al. Alleviating the new user problem in collaborative filtering by exploiting personality information[J]. User Modeling and User-Adapted Interaction, 2016, 26(2-3):1-35.
[61] Yang Chunfeng, Zhou Yipeng, Chen Liang, et al. Social-group-based ranking algorithms for cold-start video recommendation[J]. International Journal of Data Science & Analytics, 2016, 1(3-4):165-175.
[62] Candillier L, Meyer F, Boull M. Comparing state-of-the-art collaborative filtering systems[C]//International Workshop on Machine Learning and Data Mining in Pattern Recognition. Berlin Heidelberg:Springer, 2007:548-562
[63] Sanchez F, Barrilero M, Uribe S, et al. Social and content hybrid image recommender system for mobile social networks[J]. Mobile Networks & Applications, 2012, 17(6):782-795.
[64] Lekakos G, Caravelas P. A hybrid approach for movie recommendation[J]. Multimedia Tools & Applications, 2008, 36(1-2):55-70.
[65] Widisinghe A, Ranasinghe D, Kulathilaka K, et al. PicSeek:collaborative filtering for context-based image recommendation[C]//International Conference on Information and Automation for Sustainability. Colombo:IEEE Press, 2010:225-232.
[66] Liu Xianming, Tsai M H, Huang Thomas. Analyzing user preference for social image recommendation[EB/OL]. arXiv preprint, 2016[2016-10-10]. https://arxiv.org/abs/1604.07044.

Research on Personalized Image Retrieval and Recommendation

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	WANG Yong, YIN En-min, RAN Xun. Differential Privacy-Preserving Recommendation Algorithm Based on Bhattacharyya Coefficient Clustering [J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(2): 81-88.
[2]	YOU Si-qing, ZHOU Li, ZHAO Dong-jie, XUE Fei. Research on Parallelization of Collaborative Filtering Recommendation Algorithm Based on Particle Swarm Optimization [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2018, 41(6): 115-122.
[3]	WANG Yong, DENG Jiang-zhou. User Similarity Collaborative Filtering Algorithm Based on KL Divergence [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(2): 110-114.
[4]	WANG Xiao-jun, FU Chao. Enhancing Scalability and Accuracy of Collaborative Filtering Using Fuzzy Blocking [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(1): 74-78.
[5]	TANG Rui-chun, FANG Li-hua, PANG Min, XU Xiao-wei, DING Xiang-qian. Personalized Cloud-media Services Selection by QoS Prediction [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2016, 39(6): 42-46.
[6]	WANG Xiao-jun. A Distributed Hybrid Collaborative Filtering Method in Recommender Systems [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2016, 39(2): 25-29.
[7]	YANG Heng-yu, LI Hui-zong, LIN Yao-jin, ZHANG Jia. Influential Neighbor Selection in Collaborative Filtering [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2016, 39(1): 29-34.
[8]	LI Zhao, LU Wei, XING Wei-wei, SUN Zhan-quan, WANG Wei-dong, WEI Yun-chao. Image Retrieval Based on CNN Visual Features [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(s1): 103-106,120.
[9]	XU Ya-bin, SUN Xiao-chen. Individual Location Recommendation for Location-Based Social Network [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(5): 118-124.
[10]	MA Hua, HU Zhi-gang. Client Context-Aware Prediction of QoS for Web Services [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(4): 91-98.
[11]	WANG Xiao-jun. Employing Item Attribute and Preference to Enhance the Collaborative Filtering Recommendation [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(6): 68-71,76.
[12]	E Hai-hong, SONG Mei-na, LI Chuan, JIANG Zhou-feng. A Collaborative Filtering Recommendation Algorithm with Time Context for Learning Interest Mining [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(6): 49-53.
[13]	ZHOU Chao, LI Bo. A Recommendation Method Based on User Trust Network [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(4): 98-102.
[14]	ZHANG Lei;CHEN Jun-liang;MENG Xiang-wu;SHEN Xiao-yan;DUAN Kun. BP Neural Networks-Based Collaborative Filtering Recommendation Algorithm [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2009, 32(6): 42-46.
[15]	ZHAO Shan,TANG Yongli,\ LIU Jing. JPEG Image Retrieval Based on the Spatial Distribution \=of the DCT Coefficients [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2009, 32(5): 32-35.