阿里云3台机器搭建Hadoop HA服务
1 Mac电脑配置 阿里云配置机器
选择配置 按量付费 选择三台机器 2核8G
镜像选择CentOS7.4
网络设置 默认
系统配置 勾选好 添加有序后缀
设置好释放时间
最后再次确认信息
第二部 使用Zoc7 建立连接
并保持到zoc7软件 菜单栏
第三部 filezilla上传软件 jdk zookeeper hadoop
jdk1.8.191 Zookeeper3.4.12 apache-hadoop2.8.4
4 配置jdk环境
（1）解压安装包 tar -zvxf jdk*.tar.gz
(2) 修改/etc/profile文件 vi /etc/profile
（3） source /etc/profile
(4) 验证是否安装成功
5 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
6 hosts文件配置 三台机器都得配置
vi /etc/hosts
7 配置ssh免密登陆 详见另一篇博客
添加Hadoop用户 并生成Hadoop用户的ssh key
su hadoop
ssh-keygen -t rsa
8 安装Zookeeper
(1)解压 tar -zvxf zookeeper*.tar.gz
（2）重命名文件 mv zookeeper-3.4.12 zookeeper
（3）赋予文件目录到root chown -R root:root zookeeper
（4）修改配置
cd /opt/software/zookeeper/conf
cp zoo_sample.cfg zoo.cfg
vi zoo.cfg
修改 dataDir=/opt/software/zookeeper/data
server.1=hadoop001:2888:3888
server.2=hadoop002:2888:3888
server.3=hadoop003:2888:3888
之后新建数据目录
cd ../
mkdir data
touch data/myid
echo 1 > data/myid
## hadoop002/003,也修改配置,就如下不同
[root@hadoop001 software]# scp -r zookeeper hadoop002:/opt/software/
[root@hadoop001 software]# scp -r zookeeper hadoop003:/opt/software/
[root@hadoop002 zookeeper]# echo 2 > data/myid
[root@hadoop003 zookeeper]# echo 3 > data/myid
9 安装hadoop
（1）tar -zvxf hadoop-2.8.4.tar.gz
mv hadoop-2.8.4 hadoop
cd hadoop/etc/hadoop/
vi hadoop-env.sh
(2).修改$HADOOP_HOME/etc/opt/software/hadoop-env.sh
# The java implementation to use.
export JAVA_HOME=/opt/software/jdk1.8.0_191/
export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true -Djava.library.path=$HADOOP_HOME/lib:$HADOOP_HOME/lib/native"
(3).修改$HADOOP_HOME/etc/hadoop/core-site.xml
(4).修改$HADOOP_HOME/etc/hadoop/hdfs-site.xml
(5).修改$HADOOP_HOME/etc/hadoop/yarn-env.sh
#Yarn Daemon Options
#export YARN_RESOURCEMANAGER_OPTS
#export YARN_NODEMANAGER_OPTS
#export YARN_PROXYSERVER_OPTS
#export HADOOP_JOB_HISTORYSERVER_OPTS
#Yarn Logs
export YARN_LOG_DIR="/opt/software/hadoop/logs”
(6).修改$HADOOP_HOEM/etc/hadoop/mapred-site.xml
[root@hadoop001 hadoop]# cp mapred-site.xml.template mapred-site.xml
(7).修改$HADOOP_HOME/etc/hadoop/yarn-site.xml
(8).修改 slaves
[root@hadoop001 hadoop]# vi slaves
hadoop001
hadoop002
hadoop003
(9).创建临时文件夹和分发文件夹
[root@hadoop001 hadoop]# mkdir -p /opt/software/hadoop/tmp
[root@hadoop001 hadoop]# chmod -R 777 /opt/software/hadoop/tmp
[root@hadoop001 hadoop]# chown -R root:root /opt/software/hadoop/tmp
[root@hadoop001 hadoop]# scp -r hadoop root@hadoop002:/opt/software
[root@hadoop001 hadoop]# scp -r hadoop root@hadoop003:/opt/software
10 启动集群
(1)启动Zookeeper
[root@hadoop001 zookeeper]# $ZOOKEEPER_HOME/bin/zkServer.sh start
[root@hadoop002 zookeeper]# $ZOOKEEPER_HOME/bin/zkServer.sh start
[root@hadoop003 zookeeper]# $ZOOKEEPER_HOME/bin/zkServer.sh start
(2)启动Hadoop（HDFS + YARN）
[root@hadoop001 sbin]# hadoop-daemon.sh start journalnode
[root@hadoop002 hadoop]# cd /opt/software/hadoop/sbin
[root@hadoop003 sbin]# hadoop-daemon.sh start journalnode
（2）.NameNode 格式化
[root@hadoop001 hadoop]# hadoop namenode -format
（3）.同步 NameNode 元数据
同步 hadoop001 元数据到 hadoop002 主要是:dfs.namenode.name.dir，dfs.namenode.edits.dir 还应该确保共享存储目录下 (dfs.namenode.shared.edits.dir ) 包含 NameNode 所有的元数据。
[root@hadoop001 hadoop]# pwd
/opt/software/hadoop
[root@hadoop001 hadoop]# scp -r data/ root@hadoop002:/opt/software/hadoop
（4）.初始化 ZFCK
hdfs zkfc -formatZK
（5）.启动 HDFS 分布式存储系统
[root@hadoop001 sbin]# start-dfs.sh
10.启动 YARN 框架
(1)hadoop001 启动 Yarn，命令所在目录:$HADOOP_HOME/sbin
[root@hadoop001 hadoop]# start-yarn.sh
(2) hadoop002 备机启动 RM
[root@hadoop002 ~]# yarn-daemon.sh start resourcemanager
最后 验证进程
