一种基于动态划分的MapReduce负载均衡方法.pdf

1000 计算机研究与发展 ISSN 1239／CN11-17771TP Journalof Researchand Computer Development 一种基于动态划分的MapReduce负载均衡方法 2 周家帅1’2 王 琦3 高 军1 1(北京大学信息科学技术学院北京100871) 2(高可信软件技术教育部重点实验室(北京大学)北京 100871) 3(国家计算机网络应急技术处理协调中心北京100029) (zjiash@pku．edu．cn) An forLoad in via Approach BalancingMapReduceDynamicPartitioning Zhou Gao Qi3，andJunl，2 Jiashuail”，Wang 1(SchoolElectronics and 100871) of EngineeringComputerScience，PekingUniversity，Beijing 2(KeyLaboratoryofHighConfidenceSoftwareTechnologies(PekingUniversity)，Ministryof 100871) 3(NationalNetwork TechnicalTeamCoordinationCenter 100029) Computer EmergencyResponse ofChina，Beijing AbstractAsadistributed in massive computingframework，MapReducegainspopularityhanding data．Theloadbalanceisthe tothe ofdistributed current key performance computing．The cannotusttheloadbalanceoverclusters tothedata MapReduceimplementationadj according distributionwithdefaultrandomhash function．Inthis makeextensions adaptively partition paper，we to framework．Basedonthe resultsinthe side，we determinethe MapReduce sample map dynamically numberofreducersandbuilda functiontoenableloadbalanceinthereduceside． partition of andthen theresultsto trackerwill the tracker．Job Specifically，wesampleoutputmap report job constructa function tothedata