超大规模并行计算容错恢复的实现和挑战.pdfVIP

超大规模并行计算容错恢复的实现和挑战 1+ 2 贺军 , 柴华 1(IBM 中国系统与科技研发中心,北京 100193) 2(IBM 中国系统与科技研发中心,北京 100193) The Implementation and Challenges of Fault-Tolerance/Restoration in Massive The Implementation and Challenges of Fault-Tolerance/Restoration in Massive TThhee IImmpplleemmeennttaattiioonn aanndd CChhaalllleennggeess ooff FFaauulltt--TToolleerraannccee//RReessttoorraattiioonn iinn MMaassssiivvee Parallel Computing Parallel Computing PPaarraalllleell CCoommppuuttiinngg 1+ 2 He Jun , Chai Hua 1(IBM CSTL, Beijing, 100193, China) 2(IBM CSTL, Beijing, 100193, China) + Corresponding author: Phn +86-10-8245-4966, Fax +86-10-8245-1146, E-mail: justinhe@, Received 2012-07-03; Accepted 2012-07-03 Abstract Abstract AAbbssttrraacctt: This paper is intended to make an introduction for the leading methods of fault-tolerant in ultra-large scale parallel computing, and tries to summarize their differentiation in brief, furthermore it emphasizes the challenges and issues for fault-tolerant are emerging for parallel computing of massive scale. Vitally, IBM checkpoint/restart in HPC software stack is represented in details, including the architecture, implementation and the distributed processing. Besides, IBM parallel environment, programming model and container virtualization are introduced for their relationship with checkpoint/restart in IBM HPC software total provisioning. Keywords Keywords KKeeyywwoorrddss: Massive ParallelComputing;Fault-Tolerance;ContainerVirtualization,Checkpoint;Restart 作者简介 贺军: (1975－),男,湖北江夏人,硕士,IBM中国系统与科技研发中心顾问软件工程师,主要研究领域为高 性能计算、并行计算环境和协议和分布式软件;柴华(1982－),男,河南漯河人，硕士,IBM中国系统与科技研发中心 高级软件工程师,主要研究领域为并行计算环境和协议。 贺军,柴华 超大规模并行计算容错恢复的实现和挑战 摘 要: 本文介绍和比较当前超大规模并行计算的一些容错与恢复的方法，描述 IBM 高性能并行计算的 Checkpoint/Restart 技术架构。同时探讨了并行计算的容错和恢复在超大规模集群架构下所面临的问题和挑战， 以及结合并行计算编程模型的实现，阐述其与容错恢复机制之间的关联。/ 关键词: 超大规模并行计算; 容错；容器虚拟化；检查点；恢复 中图法分类号: TP316.4 文献标识码:A 1 1 11 引言 自从2008 年IBM 名为“Roadrunner”（“走鹃”）的计算机系统拉开了