Resource Modeling and Performance Prediction of MapReduce in Cloud Computing Environment

doi:10.13190/j.jbupt.2014.s1.022

JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM ›› 2014, Vol. 37 ›› Issue (s1): 115-119.doi: 10.13190/j.jbupt.2014.s1.022

• Reports • Previous Articles Next Articles

Resource Modeling and Performance Prediction of MapReduce in Cloud Computing Environment

QIAO Yuan-yuan, LIU Fang, LING Yan, YIN Jin-song

School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2014-01-01 Online:2014-06-28 Published:2014-06-28
Supported by:

Abstract

Abstract:

The prediction of job running time and computing resource is very important in production environment. However, the performance of cloud computing is not easy to be predicted due to complicated computing resources in environment of cloud platform. A model is proposed based on resource consumption patterns for predicting the running time and central processing unit (CPU) resources consuming of MapReduce job in cloud computing environment. This model comes from polynomial regression modeling to predict the performance of MapReduce job. A variety of criteria is used to evaluate the model. Experiment shows that this model could predict the running time and CPU resources consuming of MapReduce job with high accuracy.

Key words: cloud computing, MapReduce, resource modeling, MapReduce job time modeling, performance prediction

CLC Number:

TN311

QIAO Yuan-yuan, LIU Fang, LING Yan, YIN Jin-song. Resource Modeling and Performance Prediction of MapReduce in Cloud Computing Environment[J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(s1): 115-119.

[1]	ZHANG Zhi-hua, WANG Meng-qing, MAO Wen-tao, LIU Chun-hong, CHENG Bo. Joint Prediction of Multi-Workload Sequences Based on Temporal Correlation in the Cloud [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(4): 68-75.
[2]	QIU Li-ke, LIU Jing, SUN Zhong-wei, ZHAO Yang-fan. A Fast Feature Selection Framework and Method [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2019, 42(3): 127-132.
[3]	YIN Xue-yuan, CHEN Xing-shu, LI Hui, CHEN Lin. A Method to Implement File Antivirus Based on Virtualization [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2018, 41(2): 50-55.
[4]	ZHU Xiao-ning, SUN Bin, ZHU Chun-ge. Reliability Evaluation Model of Large Scale Cloud Resources [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(s1): 53-57.
[5]	WANG Li-yu, SUN Bin, QIN Tong. Application of Improved DBSCAN Clustering Algorithm in Task Scheduling of Cloud Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(s1): 68-71.
[6]	XU Jiu-yun, GUAN Chao, JIANG Dan. A Subjectivity and Time-Varying Reputation-Oriented Trust Management Method in Cloud Services [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(5): 30-35.
[7]	LIU Jing, WU Hai-bo, WANG Shao-shuai, JIA Lei. A Colored Petri Nets Model Based Cloud Test Generation Approach and Implementation System [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2016, 39(5): 89-93,115.
[8]	XIAO Ding, JIA Ya-pu, SHI Chuan. Non-liner Energy Consumption Model for Cloud Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2016, 39(1): 107-111.
[9]	GUO Li-zheng, ZHANG Yi-fei, ZHAO Shu-guang. Optimization Method of Performance and Energy Consumption Aware under Data Center Environment [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(s1): 72-76.
[10]	YU Jun-yang, HU Zhi-gang, LIU Xiu-lei. Smallfiles on HDFS Merging based on the Energy Efficiency [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(6): 34-38.
[11]	WU Bin, LIN Xing, LI Wei-dong, LU Tian-liang, ZHANG Dong-mei. Smartphone Malware Detection Model Based on Artificial Immune System in Cloud Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(4): 34-38.
[12]	WANG Ting, XU Ke, WANG Na, SONG Jun-de. Optimized Scalable Scheme for Server Selections in Cloud Computing Environments [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(s1): 83-86.
[13]	WU Xu, XU Jin, LI Chun-wen, LIU Chuan-yi. Research on eID-Based Virtual Machine Trusted Attestation System in IaaS Cloud [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(5): 85-90.
[14]	LIU Xing, YUAN Chao-wei, YANG Zhen, HU Zhong-wei. User Handoff and Access Control Based on Mobile Agent in Mobile Cloud Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(2): 88-92.
[15]	SONG Jie, LI Tian-tian, YAN Zhen-xing, ZHU Zhi-liang. Load-Balanced Data Layout Approach in Data-Intensive Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2013, 36(4): 76-80.

Resource Modeling and Performance Prediction of MapReduce in Cloud Computing Environment

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments