Centos安装ApacheHadoop2.7.7

准备文件:
hadoop-2.7.7.tar.gz

一、安装JDK

安装Hadoop之前需要保证JDK8成功安装
java -version

二、上传Hadoop压缩包

上传成功后解压文件:tar -zxvf hadoop-2.7.7.tar.gz
假设解压后的Hadoop目录为: /home/hadoop/hadoop-2.7.7

注意配置HADOOP_HOME环境变量
export HADOOP_HOME=/home/hadoop/hadoop-2.7.7
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

三、配置服务器免密登录

$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ chmod 0600 ~/.ssh/authorized_keys

执行完成后: ssh master命令则能够跳过输入密码阶段直接连接

四、修改配置

==配置文件所在位置: $HADOOP_HOME/etc/hadoop==

  1. core-size.xml
    <configuration>
     <property>
         <name>fs.defaultFS</name>
         <value>hdfs://master:9000</value>
     </property>
    </configuration>
  2. hdfs-size.xml
    <configuration>
     <property>
         <name>dfs.replication</name>
         <value>1</value>
     </property>
    </configuration>
  3. mapred-site.xml
    <configuration>
     <property>
         <name>mapreduce.framework.name</name>
         <value>yarn</value>
     </property>
     <property>
         <name>mapreduce.application.classpath</name>
         <value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*</value>
     </property>
    </configuration>
  4. yarn-site.xml
    <configuration>
     <property>
         <name>yarn.nodemanager.aux-services</name>
         <value>mapreduce_shuffle</value>
     </property>
     <property>
         <name>yarn.nodemanager.env-whitelist</name>
         <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
     </property>
    </configuration>

    五、启动

  5. 初始化HDFS hdfs namenode -format
  6. 启动HDFS start-dfs.sh
  7. 启动YARN start-yarn.sh
  8. 启动成功后访问http://master:50070查看HDFS状态

六、相关命令

start-dfs.sh 启动HDFS(namenode/datanode/SecondaryNamenode)
stop-dfs.sh 停止HDFS
start-yarn.sh 启动Yarn
stop-yarn.sh 停止Yarn
start-all.sh 启动HDFS和Yarn
stop-all.sh 停止HDFS和Yarn