centos7 安装hadoop2.7.6（分布式）

本文只做简单介绍，具体步骤操作请参考centos6.5 安装hadoop1.2.1亲测版

本篇只简单介绍安装步骤

1.安装目录 /usr/local/hadoop （HADOOP_HOME）

2,创建group hadoop

groupadd hadoop

 

3.创建用户hadoop 

useradd -d /home/hadoop -g hadoop hadoop
chown -R hadoop:hadoop /usr/local/hadoop

 

4.每台机器安装jdk（安装在相同的目录）

5.master 到 slave ssh 免密码登录

6.关闭所有机器防火墙。selinux

1）centos 7 以下
防火墙
查看防火墙状态
service iptables status
临时关闭防火墙
service iptables stop
永久关闭防火墙
chkconfig iptables off
 selinux
setenforce 0 #临时关闭
永久关闭需要修改如下文件
vim /etc/selinux/config
将SELINUX=enforcing改为SELINUX=disabled
2)centos 7
firewall-cmd --reload #重启firewall
systemctl stop firewalld.service #停止firewall
systemctl disable firewalld.service #禁止firewall开机启动
firewall-cmd --state #查看默认防火墙状态（关闭后显示notrunning，开启后显示running）
selinux 同上

 

7.修改hadoop配置文件

core-site.xml

<configuration>
 <property>
        <name>fs.defaultFS</name>
        <value>hdfs://ns1:9000</value>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>4096</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/home/hadoop/tmp</value>
        <description>Abase for other temporary   directories.</description>
    </property>
</configuration>

hadoop-env.sh

添加
export JAVA_HOME=/usr/local/jdk

hdfs-site.xml

<configuration>
<property>
<name>dfs.namenode.secondary.http-address</name>
               <value>ns1:9001</value>
       </property>
     <property>
             <name>dfs.namenode.name.dir</name>
             <value>file:/home/hadoop/name</value>
       </property>
      <property>
              <name>dfs.datanode.data.dir</name>
              <value>file:/home/hadoop/data</value>
       </property>
       <property>
               <name>dfs.replication</name>
               <value>2</value>
        </property>
        <property>
                 <name>dfs.webhdfs.enabled</name>
                  <value>true</value>
         </property>
</configuration>

mapred-env.sh

添加
export JAVA_HOME=/usr/local/jdk

mapred-site.xml

<configuration>
<property>
<name>mapreduce.framework.name</name>
                <value>yarn</value>
           </property>
          <property>
                  <name>mapreduce.jobhistory.address</name>
                  <value>ns1:10020</value>
          </property>
          <property>
                <name>mapreduce.jobhistory.webapp.address</name>
                <value>ns1:19888</value>
       </property>
</configuration>

slaves

ns1
dn1

yarn-env.sh

添加
export JAVA_HOME=/usr/local/jdk

yarn-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
               <value>mapreduce_shuffle</value>
        </property>
        <property>                                                                
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
               <value>org.apache.hadoop.mapred.ShuffleHandler</value>
        </property>
        <property>
               <name>yarn.resourcemanager.address</name>
               <value>ns1:8032</value>
       </property>
       <property>
               <name>yarn.resourcemanager.scheduler.address</name>
               <value>ns1:8030</value>
       </property>
       <property>
            <name>yarn.resourcemanager.resource-tracker.address</name>
             <value>ns1:8031</value>
      </property>
      <property>
              <name>yarn.resourcemanager.admin.address</name>
               <value>ns1:8033</value>
       </property>
       <property>
               <name>yarn.resourcemanager.webapp.address</name>
               <value>ns1:8088</value>
       </property>
</configuration>

 

8.修改环境变量，添加HADOOP_HOME 和JAVA_HOME

9.启动hadoop

1）在master上format namenode
hadoop namenode -format
2)启动hadoop
sh /user/local/hadoop/sbin/start-all.sh
3)启动jobhistory
sh /user/local/hadoop/sbin/mr-jobhistory-daemon.sh start historyserver

 

10.访问web界面查看

http://ns1:8088/cluster