- 1、本文档共36页,可阅读全部内容。
- 2、有哪些信誉好的足球投注网站(book118)网站文档一经付费(服务费),不意味着购买了该文档的版权,仅供个人/单位学习、研究之用,不得用于商业用途,未经授权,严禁复制、发行、汇编、翻译或者网络传播等,侵权必究。
- 3、本站所有内容均由合作方或网友上传,本站不对文档的完整性、权威性及其观点立场正确性做任何保证或承诺!文档内容仅供研究参考,付费前请自行鉴别。如您付费,意味着您自己接受本站规则且自行承担风险,本站不退款、不进行额外附加服务;查看《如何避免下载的几个坑》。如果您已付费下载过本站文档,您可以点击 这里二次下载。
- 4、如文档侵犯商业秘密、侵犯著作权、侵犯人身权等,请点击“版权申诉”(推荐),也可以打举报电话:400-050-0827(电话支持时间:9:00-18:30)。
查看更多
Hadoop File System UB Computer Science and EngineeringHadoop文件系统布法罗大学计算机科学与工程
Objectives Primary objective of HDFS is to store data reliably in the presence of failures. Three common failures are: Namenode failure, Datanode failure and network partition. * * DataNode failure and heartbeat A network partition can cause a subset of Datanodes to lose connectivity with the Namenode. Namenode detects this condition by the absence of a Heartbeat message. Namenode marks Datanodes without Hearbeat and does not send any IO requests to them. Any data registered to the failed Datanode is not available to the HDFS. Also the death of a Datanode may cause replication factor of some of the blocks to fall below their specified value. * * Re-replication The necessity for re-replication may arise due to: A Datanode may become unavailable, A replica may become corrupted, A hard disk on a Datanode may fail, or The replication factor on the block may be increased. * * Cluster Rebalancing HDFS architecture is compatible with data rebalancing schemes. A scheme might move data from one Datanode to another if the free space on a Datanode falls below a certain threshold. In the event of a sudden high demand for a particular file, a scheme might dynamically create additional replicas and rebalance other data in the cluster. These types of data rebalancing are not yet implemented: research issue. * * Data Integrity Consider a situation: a block of data fetched from Datanode arrives corrupted. This corruption may occur because of faults in a storage device, network faults, or buggy software. A HDFS client creates the checksum of every block of its file and stores it in hidden files in the HDFS namespace. When a clients retrieves the contents of file, it verifies that the corresponding checksums match. If does not match, the client can retrieve the block from a replica. * * Metadata Disk Failure FsImage and EditLog are central data structures of HDFS. A corruption of these files can cause a HDFS instance to be non-functional. For this reason, a Namenode can be config
您可能关注的文档
- CAD课程设计红外线遥感调速器设计.doc
- CaF2纳米晶体作为润滑脂添加剂摩擦学调查.doc
- cadcam课程设计支架建模与加工.doc
- CCB与脑卒中 从CCB抗动脉粥样硬化作用看 对脑血管保护.ppt
- CDMA2000系统分析.doc
- CDMA扩频通信中同步系统设计开题报告.doc
- CADCAM软件应用UG软件应用模块滑动手柄编程与加工.ppt
- CFG桩复合地基地基处理技术及其发展现状彭东林.doc
- Ch11数据高级处理.ppt
- ch03现代教学媒体软件开发和利用.ppt
- 2024至2030年中国羚羊角类饮片行业深度调查与前景预测分析报告.docx
- 重庆市面向中国农业大学定向选调2024届大学毕业生2024年国家公务员考试考试大纲历年真题14笔试历.docx
- 重庆市面向西北工业大学定向选调2024届大学毕业生00笔试历年典型考题及解题思路附答案详解.docx
- 中国不动杆菌感染治疗药行业市场现状分析及竞争格局与投资发展研究报告2024-2029版.docx
- 2024至2030年全球与中国ETL软件市场现状及未来发展趋势.docx
- 初中八年级(初二)生物下册期末考试1含答案解析.docx
- 干簧式继电器项目申请报告.docx
- 2024至2030年中国左氧氟沙星片行业深度调查与前景预测分析报告.docx
- 菜籽项目申请报告.docx
- 2024至2030年中国八角钢行业深度调查与前景预测分析报告.docx
文档评论(0)