分布式存储系统中基于纠删码的故障节点恢复的实现.docxVIP

┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ 装 ┊ ┊ ┊ ┊ ┊ 订 ┊ ┊ ┊ ┊ ┊ 线 ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ 摘要 PAGE II 摘 要 随着互联网技术的快速发展，海量数据的存储系统受到了严重的挑战。先前在数据量很少的情况下可采用单个节点进行数据存储，但是随着大数据规模的快速增长，单点存储已不能适应海量数据的存储。现在分布式存储系统已经得到了广泛的应用，但是其庞大的存储节点必将导致节点失效情况不断发生。为了保证数据的可靠性，经常采用的容错技术有多副本技术和纠删码技术，与纠删码技术相比，多副本技术在相同数据可靠性的情况下将消耗更多的存储空间，所以纠删码技术越来越受到企业和学者的注意。 本文首先介绍了大数据和分布式存储系统的背景以及当前常见的容错技术和之间的比较；然后介绍了纠删码的相关概念、原理以及编码实现技术，目前常见的纠删码技术包括RS编码、分组码和再生码，并且从数据编码性能和数据修复性能两方面对各种纠删码进行比较，也对不同存储系统对纠删码的性能需求进行分析；重点介绍了RS编码的理论、原理；最后对RS编码算法进行c语言实现，分别对不同节点失效进行实验仿真。 关键词: 纠删码 分布式存储 编码 数据恢复 ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ 装 ┊ ┊ ┊ ┊ ┊ 订 ┊ ┊ ┊ ┊ ┊ 线 ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ┊ ABSTRACT ABSTRACT With the rapid development of Internet technology, massive data storage system has been a serious challenge. In the case of a small amount of data, a single node can be used for data storage, but with the rapid growth of large data size, single point of storage can not adapt to the storage of massive data. Distributed storage systems are now widely used, but their huge storage nodes will inevitably lead to node failures continue to occur. In order to ensure the reliability of the data, often with the fault-tolerant technology with multiple copies of technology and error detection code technology, compared with the error detection technology, multi-copy technology in the same data reliability will consume more storage space, so More and more by the enterprises and scholars attention. This paper first introduces the background of large data and distributed storage system and the comparison of common fault-tolerant technologies. Then, the related concepts, principles and coding techniques of error-correcting codes are introduced. The common error- Coding, block code and regenerative code, and compare the various error-checking codes from the aspects of data coding performance and data repair performance. The performance requirements of different storage systems are also analyzed. The theory of RS coding , The principle of the final implementation