Load Balancing Strategy Based on CMM Model in HDFS

doi:10.13190/j.jbupt.2014.05.005

JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM ›› 2014, Vol. 37 ›› Issue (5): 20-25.doi: 10.13190/j.jbupt.2014.05.005

• Papers • Previous Articles Next Articles

Load Balancing Strategy Based on CMM Model in HDFS

LU Mei-lian¹, ZHU Liang-liang²

1. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China;
2. Science and Technology Development Center, Hua Xia Bank Limited Company, Beijing 100005, China

Received:2013-12-18 Online:2014-10-28 Published:2014-10-28

Abstract

Abstract:

In order to balance the load of Hadoop distributed file system with lower load migrating cost and data transmission cost, a certainty multi-stage and multi-object (CMM) decision model was proposed. The model is a directed acyclic graph built on decision nodes, which adopts the remaining load capacities of CPU, memory and disk as decision preconditions, and also adopts load balancing effect, load migrating cost and data transmission cost as decision targets. By CMM model, the best balancing plan is determined by selecting results of multiple decision stages and computing the plan usage. Simulations show that the CMM based strategy can achieve better load balancing effect, load migrating cost and data transmission cost.

Key words: distributed file system, load balancing, decision model, remaining load capacity

CLC Number:

TN391

LU Mei-lian, ZHU Liang-liang. Load Balancing Strategy Based on CMM Model in HDFS[J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(5): 20-25.

[1]	SU Yang, WEI Liansuo, GUO Yuan. Multi-Objective Fusion Potential Game Wireless Ad Hoc Network Topology Control Algorithm [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(4): 91-97.
[2]	. Multi-Objective Fusion Ordinal Potential Game Wireless Ad Hoc Network Topology Control Algorithm [J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(4): 105-111.
[3]	YAN Yi-fei, WANG Zhi-li, QIU Xue-song, WANG Jia-lu. A Shuffle Partition Optimization Scheme Based on Data Skew Model in Spark [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(2): 116-121.
[4]	LIU Bao-ju, YU Peng, FENG Lei, QIU Xue-song, JIANG Hao. Rerouting Algorithm for Load Balancing in SDN-Enabled Smart Grid Communication Network [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(2): 16-22.
[5]	MA Qing-liu, YU Peng, WU Jia-hui, XIONG Ao, YAN Yong. A Integrated Energy Service Channel Optimization Mechanism Based on Deep Reinforcement Learning [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(2): 87-93.
[6]	WANG Xuan, HOU Rong-hui, XU Wei-lin. Dynamic Path Switching Technology for LEO Satellite Networks [J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(2): 80-86,109.
[7]	ZHU Bin, SUN Bin. A Load Balancing Predication Algorithm of CART and KNN [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(s1): 93-97.
[8]	WANG Ying-jie, LIU Xiao-ning, LUO Huan-huan, ZHOU Gui-ping, WANG Yan-ru. Wavelength Allocation Scheme Based on User Traffic Prediction in TWDM-PON [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(6): 74-79.
[9]	YU Jun-yang, HU Zhi-gang, LIU Xiu-lei. Smallfiles on HDFS Merging based on the Energy Efficiency [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2015, 38(6): 34-38.
[10]	WANG Ying, XIONG Wen-cheng, LI Wen-jing. Random Virtual Network Embedding Algorithm Based on Maximum Independent Link Set [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(s1): 8-11.
[11]	WANG Ting, XU Ke, WANG Na, SONG Jun-de. Optimized Scalable Scheme for Server Selections in Cloud Computing Environments [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(s1): 83-86.
[12]	GAO Ya, QIU Zhi-liang, ZHANG Jian. A Cell Assignment Algorithm for Balancing Multicast Traffic with Small Fanout [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2014, 37(5): 91-95.
[13]	SONG Jie, LI Tian-tian, YAN Zhen-xing, ZHU Zhi-liang. Load-Balanced Data Layout Approach in Data-Intensive Computing [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2013, 36(4): 76-80.
[14]	GUO Chong-tao, SHENG Min, SHI Yan. A Novel Traffic Transferring Based Load Balancing Scheme in Cellular Networks [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2012, 35(6): 92-96.
[15]	. Adaptive Load Balancing Algorithm for Parallel Volume Rendering [J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2012, 35(1): 99-102.

Load Balancing Strategy Based on CMM Model in HDFS

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments