Hadoop集群安装手册.docx

Hadoop完全分布式的安装和配置目录Hadoop完全分布式的安装和配置1配置hosts文件2创建hadoop账号3安装jdk4配置SSH加密5下载hadoop软件6安装hadoop7启动守护进程11网页上查看节点状态12安装ant13配置eclipse14Cygwin安装配置19配置hosts文件创建hadoop账号创建一个用户组hadoop创建用户主目录hadoop创建用户hadoop，并把该用户分配到hadoop组为hadoop账号创建密码安装jdk设置环境变量#java homeexport JAVA_HOME=/usr/java/jdk1.7.0_21#hadoop homeexport HADOOP_HOME=/home/hadoop/hadoop-2.6.0export PATH=$HADOOP_HOME/bin:$PATH使参数生效配置SSH加密共享秘钥在其他节点生成秘钥，然后将所有节点的公钥（id_rsa.pub）内容复制到一个authorized_keys中，再把authorized_keys复制到每个节点中。下载hadoop软件官网：//安装hadoop解压安装包将解压后的目录移动到hadoop主目录下修改组修改hadoop-env.sh文件export JAVA_HOME=/usr/java/jdk1.7.0_21export HADOOP_PREFIX=/home/hadoop/hadoop-2.6.0修改core-site.xml文件 property namefs.defaultFS/name valuehdfs://hadoop01:9000/value /property property namehadoop.tmp.dir/name value/home/hadoop/tmp/value/property参数参考：/docs/r2.6.0/hadoop-project-dist/hadoop-common/core-default.xml/docs/r2.6.0/hadoop-project-dist/hadoop-common/core-default.xml修改hdfs-site.xml文件 property name.dir/name valuefile:/home/hadoop/tmp/dfs/name/value /property property namedfs.datanode.data.dir/name valuefile:/usr/sap/hadoop_data/value /property property namedfs.datanode.ipc.address/name value:50020/value /property property namedfs.datanode.http.address/name value:50075/value /property property namedfs.replication/name value3/value /property property namedfs.webhdfs.enabled/name valuetrue/value/property参数参考：/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml修改mapred-site.xml文件 property name/name valueyarn/value/property参数参考：/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml修改yarn-site.xml文件 property nameyarn.nodemanager.aux-services/name valuemapreduce_shuffle/value /propertyyarn.resourcemanager.hostname 该项可防止nodemanager启动后消失参数参考：/docs/r2.6.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml/docs/r2.6.0/hadoop-yarn/hadoop-yarn-common/yarn-default.xml