A New Multi-Master Framework of MapReduce

doi:10.13190/jbupt.201204.89.jinj

JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM ›› 2012, Vol. 35 ›› Issue (4): 89-93.doi: 10.13190/jbupt.201204.89.jinj

• Reports • Previous Articles Next Articles

A New Multi-Master Framework of MapReduce

JIN Jing1,WANG Yan2,LI Xin1,CHEN Shan-zhi3

1State Laboratary of Switching and Networking Technology2Xinyang Power Supply Company of Henan Electronic Power Company 3China Academy of Telecommunication Technology

Received:2011-09-21 Revised:2011-11-01 Online:2012-08-28 Published:2012-07-08
Supported by:
;the Fundamental Research Funds for the Central Universities

Abstract

Abstract:

In the existing MapReduce framework, the single master is the performance bottleneck. As a result, the size of the cluster is limited. In order to break the bottleneck, a multimaster framework is proposed in this paper. The implementation is based on Apache Hadoop MapReduce. The main framework and signaling system are redesigned to sustain more than one master. Meanwhile a doublemaster hot backup scheme is designed to solve single master failure, so that the new framework is able to reduce the impact of master failure.

Key words: cloud computing, mapreduce framework, multi-master, performance bottleneck

CLC Number:

TP316.4

JIN Jing,WANG Yan,LI Xin,CHEN Shan-zhi. A New Multi-Master Framework of MapReduce[J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2012, 35(4): 89-93.

[1]	ZHANG Zhi-hua, WANG Meng-qing, MAO Wen-tao, LIU Chun-hong, CHENG Bo. Joint Prediction of Multi-Workload Sequences Based on Temporal Correlation in the Cloud [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(4): 68-75.
[2]	YIN Xue-yuan, CHEN Xing-shu, LI Hui, CHEN Lin. A Method to Implement File Antivirus Based on Virtualization [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2018, 41(2): 50-55.
[3]	ZHU Xiao-ning, SUN Bin, ZHU Chun-ge. Reliability Evaluation Model of Large Scale Cloud Resources [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(s1): 53-57.
[4]	WANG Li-yu, SUN Bin, QIN Tong. Application of Improved DBSCAN Clustering Algorithm in Task Scheduling of Cloud Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(s1): 68-71.
[5]	XU Jiu-yun, GUAN Chao, JIANG Dan. A Subjectivity and Time-Varying Reputation-Oriented Trust Management Method in Cloud Services [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(5): 30-35.
[6]	LIU Jing, WU Hai-bo, WANG Shao-shuai, JIA Lei. A Colored Petri Nets Model Based Cloud Test Generation Approach and Implementation System [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2016, 39(5): 89-93,115.
[7]	XIAO Ding, JIA Ya-pu, SHI Chuan. Non-liner Energy Consumption Model for Cloud Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2016, 39(1): 107-111.
[8]	GUO Li-zheng, ZHANG Yi-fei, ZHAO Shu-guang. Optimization Method of Performance and Energy Consumption Aware under Data Center Environment [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(s1): 72-76.
[9]	YU Jun-yang, HU Zhi-gang, LIU Xiu-lei. Smallfiles on HDFS Merging based on the Energy Efficiency [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(6): 34-38.
[10]	WU Bin, LIN Xing, LI Wei-dong, LU Tian-liang, ZHANG Dong-mei. Smartphone Malware Detection Model Based on Artificial Immune System in Cloud Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(4): 34-38.
[11]	WANG Ting, XU Ke, WANG Na, SONG Jun-de. Optimized Scalable Scheme for Server Selections in Cloud Computing Environments [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(s1): 83-86.
[12]	QIAO Yuan-yuan, LIU Fang, LING Yan, YIN Jin-song. Resource Modeling and Performance Prediction of MapReduce in Cloud Computing Environment [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(s1): 115-119.
[13]	WU Xu, XU Jin, LI Chun-wen, LIU Chuan-yi. Research on eID-Based Virtual Machine Trusted Attestation System in IaaS Cloud [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(5): 85-90.
[14]	LIU Xing, YUAN Chao-wei, YANG Zhen, HU Zhong-wei. User Handoff and Access Control Based on Mobile Agent in Mobile Cloud Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(2): 88-92.
[15]	SONG Jie, LI Tian-tian, YAN Zhen-xing, ZHU Zhi-liang. Load-Balanced Data Layout Approach in Data-Intensive Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2013, 36(4): 76-80.

A New Multi-Master Framework of MapReduce

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments