Apache Hadoop 2.9.2 的HDFS High Available模式部署
Apache Hadoop 2.9.2 的HDFS High Available 模式部署
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
我们知道，当NameNode进程挂掉后，可以通过SecondName进行补救，我们可以快速进行恢复操作。但是其效率极低，可能等你启动集群成功后，半小时就过去了，会严重影响业务！这个时候我们就需要对NameNode做高可用。我们可以通过双NameNode消除单点故障。
一.HA概述
1>.所谓High Available，简称HA，即高可用（7*24小时不中断服务）。
2>.实现高可用最关键对策略是消除单点故障。HA严格来说应该分成各个组建对HA机制：HDFS对HA和YARN的HA。
3>.Hadoop2.0版本之前，在HDFS集群中NameNode存在单点故障（SPOF）
4>.NameNode主要在以下两个方面影响HDFS集群：
第一：NameNode机器发生意外，如宕机，集群将无法使用，知道管理员重启。
第二：NameNode机器需要升级，包括软件，硬件升级，此时集群也将无法使用。
HDFS HA功能通过配置Active/Standby两个NameNodes实现集群中对NameNode的热备来解决上述问题。如出现故障，如机器崩溃或机器需要升级维护，这时通过此种方式将NameNode很快的切换到另外一台机器。
二.HDFS HA工作机制
1>.HDFS HA工作要点
一.元数据管理方式需要改变： 1>.内存中各自保存一份元数据； 2>.Edits日志只有Active状态的namenode节点可以做写操作； 3>.两个namenode都可以读取edits； 4>.共享的edits放在一个共享存储中管理（qjournal和NFS两个主流实现）；
二.需要一个状态管理功能模块 实现了一个zkfailover，常驻在每一个namenode所在的节点，每一个zkfailover负责监控自己所在namenode节点，利用zk进行状态标识，当需要进行状态切换时，由zkfailover来负责切换，切换时需要防止brain split现象的发生。
三.必须保证两个NameNode之间能够ssh无密码登录。
四.隔离（Fence）
即同一时刻仅仅有一个NameNode对外提供服务
2>.HDFS HA 自动故障转移工作机制
我们知道使用命令hdfs haadmin -failover手动进行故障转移，在该模式下，即使 Active NameNode已经失效，系统也不会自动从Active NameNode转移到 Standby NameNode，下面学习如何配置部署HA自动进行故障转移。自动故障转移为HDFS部署增加了两个新组件：ZooKeeper和ZKFailoverController（ZKFC）进程。
ZooKeeper是维护少量协调数据，通知客户端这些数据的改变和监视客户端故障的高可用服务。HA的自动故障转移依赖于ZooKeeper的以下功能：
故障检测：
集群中的每个NameNode在ZooKeeper中维护了一个持久会话，如果机器崩溃，ZooKeeper中的会话将终止，ZooKeeper通知另一个NameNode需要触发故障转移。
Active NameNode选择：
ZooKeeper提供了一个简单的机制用于唯一的选择一个节点为active状态。如果目前 Active NameNode崩溃，另一个节点（Standby NameNode）可能从ZooKeeper获得特殊的排外锁以表明它应该成为 Active NameNode。
ZKFC是自动故障转移中的另一个新组件，是ZooKeeper的客户端，也监视和管理NameNode的状态。每个运行NameNode的主机也运行了一个ZKFC进程，ZKFC负责：
健康监测：
ZKFC使用一个健康检查命令定期地ping与之在相同主机的NameNode，只要该NameNode及时地回复健康状态，ZKFC认为该节点是健康的。如果该节点崩溃，冻结或进入不健康状态，健康监测器标识该节点为非健康的。
ZooKeeper会话管理：
当本地NameNode是健康的，ZKFC保持一个在ZooKeeper中打开的会话。如果本地NameNode处于active状态，ZKFC也保持一个特殊的znode锁，该锁使用了ZooKeeper对短暂节点的支持，如果会话终止，锁节点将自动删除。
基于ZooKeeper的选择：
如果本地NameNode是健康的，且ZKFC发现没有其它的节点当前持有znode锁，它将为自己获取该锁。如果成功，则它已经赢得了选择，并负责运行故障转移进程以使它的本地NameNode为active。故障转移进程与前面描述的手动故障转移相似，首先将之前Active NameNode进行降级操作，然后本地NameNode转换为active状态。这样做的好处就是防止脑裂，换句话说，尽管之前的Actice NameNode进程启动成功，它也依旧无法成为Active，而是乖乖的做Standby状态，当另一个Activ节点挂掉时它才会发生状态变化。
三.搭建zookeeper集群
1>.下载zookeeper软件（下载地址：https://www.apache.org/dyn/closer.cgi/zookeeper/）
[root@node101.yinzhengjie.org.cn ~]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz --2019-04-12 08:48:40-- https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz Resolving mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)... 101.6.8.193, 2402:f000:1:408:8100::1 Connecting to mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)|101.6.8.193|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 37676320 (36M) [application/x-gzip] Saving to: ‘zookeeper-3.4.14.tar.gz’ 100%[============================================================================================================================================>] 37,676,320 7.57MB/s in 4.5s 2019-04-12 08:48:45 (7.94 MB/s) - ‘zookeeper-3.4.14.tar.gz’ saved [37676320/37676320] [root@node101.yinzhengjie.org.cn ~]#
2>.加压zookeeper并配置环境变量
[root@node101.yinzhengjie.org.cn ~]# tar -zxf zookeeper-3.4.14.tar.gz -C /yinzhengjie/softwares/ [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/zookeeper-3.4.14/ total 1672 drwxr-xr-x 2 2002 2002 202 Mar 6 09:09 bin -rw-rw-r-- 1 2002 2002 97426 Mar 6 08:50 build.xml drwxr-xr-x 2 2002 2002 77 Mar 6 09:09 conf drwxr-xr-x 2 2002 2002 4096 Mar 6 09:10 dist-maven -rw-rw-r-- 1 2002 2002 1709 Mar 6 08:50 ivysettings.xml -rw-rw-r-- 1 2002 2002 10742 Mar 6 08:50 ivy.xml drwxr-xr-x 4 2002 2002 308 Mar 6 09:09 lib -rw-rw-r-- 1 2002 2002 11970 Mar 6 08:50 LICENSE.txt -rw-rw-r-- 1 2002 2002 3132 Mar 6 08:50 NOTICE.txt -rw-rw-r-- 1 2002 2002 31622 Mar 6 08:50 pom.xml -rw-rw-r-- 1 2002 2002 1765 Mar 6 08:50 README.md -rw-rw-r-- 1 2002 2002 1770 Mar 6 08:50 README_packaging.txt drwxr-xr-x 3 2002 2002 22 Mar 6 08:50 src -rw-rw-r-- 1 2002 2002 1515359 Mar 6 08:50 zookeeper-3.4.14.jar -rw-rw-r-- 1 2002 2002 836 Mar 6 09:10 zookeeper-3.4.14.jar.asc -rw-rw-r-- 1 2002 2002 33 Mar 6 08:50 zookeeper-3.4.14.jar.md5 -rw-rw-r-- 1 2002 2002 41 Mar 6 08:50 zookeeper-3.4.14.jar.sha1 drwxr-xr-x 3 2002 2002 47 Mar 6 09:09 zookeeper-client drwxr-xr-x 12 2002 2002 4096 Mar 6 09:09 zookeeper-contrib drwxr-xr-x 7 2002 2002 4096 Mar 6 09:09 zookeeper-docs drwxr-xr-x 3 2002 2002 35 Mar 6 09:09 zookeeper-it drwxr-xr-x 4 2002 2002 46 Mar 6 09:09 zookeeper-jute drwxr-xr-x 5 2002 2002 176 Mar 6 09:09 zookeeper-recipes drwxr-xr-x 3 2002 2002 32 Mar 6 09:09 zookeeper-server [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# tail -3 /etc/profile #ADD ZOOKEEPER_HOME PATH ZOOKEEPER_HOME=/yinzhengjie/softwares/zookeeper-3.4.14 PATH=$PATH:$ZOOKEEPER_HOME/bin [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# source /etc/profile [root@node101.yinzhengjie.org.cn ~]#
3>.配置zookeeper的配置文件
[root@node101.yinzhengjie.org.cn ~]# cp /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo_sample.cfg /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo.cfg [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/zookeeper-3.4.14/conf/ total 16 -rw-rw-r-- 1 2002 2002 535 Mar 6 08:50 configuration.xsl -rw-rw-r-- 1 2002 2002 2161 Mar 6 08:50 log4j.properties -rw-r--r-- 1 root root 922 Apr 12 08:55 zoo.cfg -rw-rw-r-- 1 2002 2002 922 Mar 6 08:50 zoo_sample.cfg [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo.cfg # 滴答，计时的基本单位，默认是2000毫秒，即2秒。它是zookeeper最小的时间单位，用于丈量心跳时间和超时时间等，通常设置成默认2秒即可。 tickTime=2000 # 初始化限制是10滴答，默认是10个滴答，即默认是20秒。指定follower节点初始化是链接leader节点的最大tick次数。 initLimit=5 # 数据同步的时间限制，默认是5个滴答，即默认时间是10秒。设定了follower节点与leader节点进行同步的最大时间。与initLimit类似，它也是以tickTime为单位进行指定的。 syncLimit=2 # 指定zookeeper的工作目录，这是一个非常重要的参数，zookeeper会在内存中在内存只能中保存系统快照，并定期写入该路径指定的文件夹中。生产环境中需要注意该文件夹的磁盘占用情况。 dataDir=/home/yinzhengjie/zookeeper # 监听zookeeper的默认端口。zookeeper监听客户端链接的端口，一般设置成默认2181即可。 clientPort=2181 # 这个操作将限制连接到 ZooKeeper 的客户端的数量，限制并发连接的数量，它通过 IP 来区分不同的客户端。此配置选项可以用来阻止某些类别的 Dos 攻击。将它设置为 0 或者忽略而不进行设置将会取消对并发连接的限制。 #maxClientCnxns=60 # 在上文中已经提到，3.4.0及之后版本，ZK提供了自动清理事务日志和快照文件的功能，这个参数指定了清理频率，单位是小时，需要配置一个1或更大的整数，默认是0，表示不开启自动清理功能。 #autopurge.purgeInterval=1 # 这个参数和上面的参数搭配使用，这个参数指定了需要保留的文件数目。默认是保留3个。 #autopurge.snapRetainCount=3 #server.x=[hostname]:nnnnn[:nnnnn]，这里的x是一个数字，与myid文件中的id是一致的。右边可以配置两个端口，第一个端口用于F和L之间的数据同步和其它通信，第二个端口用于Leader选举过程中投票通信。 server.101=node101.yinzhengjie.org.cn:2888:3888 server.102=node102.yinzhengjie.org.cn:2888:3888 server.103=node103.yinzhengjie.org.cn:2888:3888 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/zookeeper-3.4.14/conf/java.env #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com #指定JDK的安装路径 export JAVA_HOME=/yinzhengjie/softwares/jdk #指定zookeeper的heap内存大小 export JVMFLAGS="-Xms256m -Xmx256m $JVMFLAGS" [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
4>.编写zookeeper的启动脚本
[root@node101.yinzhengjie.org.cn ~]# vi /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# cat /usr/local/bin/xzk.sh #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com #判断用户是否传参 if [ $# -ne 1 ];then echo "无效参数，用法为: $0 {start|stop|restart|status}" exit fi #获取用户输入的命令 cmd=$1 #定义函数功能 function zookeeperManger(){ case $cmd in start) echo "启动服务" remoteExecution start ;; stop) echo "停止服务" remoteExecution stop ;; restart) echo "重启服务" remoteExecution restart ;; status) echo "查看状态" remoteExecution status ;; *) echo "无效参数，用法为: $0 {start|stop|restart|status}" ;; esac } #定义执行的命令 function remoteExecution(){ for (( i=101 ; i<=103 ; i++ )) ; do tput setaf 2 echo ========== node${i}.yinzhengjie.org.cn zkServer.sh $1 ================ tput setaf 9 ssh node${i}.yinzhengjie.org.cn "source /etc/profile ; zkServer.sh $1" done } #调用函数 zookeeperManger [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# chmod +x /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]# ll /usr/local/bin/xzk.sh -rwxr-xr-x 1 root root 1125 Apr 18 15:54 /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]#
5>.创建myid文件并写入服务器编号
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'mkdir /home/yinzhengjie/zookeeper/' [WARNING]: Consider using file module with state=directory rather than running mkdir node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# for (( i=101;i<=103;i++ )) do ssh node${i}.yinzhengjie.org.cn "echo -n $i > /home/yinzhengjie/zookeeper/myid" ;done [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'cat /home/yinzhengjie/zookeeper/myid' node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> 102 node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> 103 node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> 101 [root@node101.yinzhengjie.org.cn ~]#
6>.分发配置文件
[root@node101.yinzhengjie.org.cn ~]# scp /etc/profile node102.yinzhengjie.org.cn:/etc/profile 100% 2127 1.2MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp /etc/profile node103.yinzhengjie.org.cn:/etc/profile 100% 2127 1.4MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/zookeeper-3.4.14/ node102.yinzhengjie.org.cn:/yinzhengjie/softwares/ ........ log.42 100% 184 128.3KB/s 00:00 log.63b 100% 48KB 22.7MB/s 00:00 snapshot.0 100% 296 277.6KB/s 00:00 snapshot.272 100% 55KB 26.5MB/s 00:00 snapshot.273 100% 55KB 24.3MB/s 00:00 snapshot.639 100% 140KB 27.2MB/s 00:00 snapshot.83f 100% 4824 3.5MB/s 00:00 minikdc-krb5.conf 100% 1103 1.3MB/s 00:00 minikdc.ldiff 100% 1630 2.2MB/s 00:00 log.100000001 100% 2401KB 50.2MB/s 00:00 log.100001bf0 100% 1001KB 59.6MB/s 00:00 snapshot.100000000 100% 73 17.1KB/s 00:00 snapshot.100001bec 100% 2507KB 61.6MB/s 00:00 check_compatibility.py 100% 7212 4.7MB/s 00:00 checkstyle-noframes-sorted.xsl 100% 5489 6.5MB/s 00:00 checkstyle.xml 100% 7489 7.3MB/s 00:00 findbugsExcludeFile.xml 100% 5027 6.6MB/s 00:00 log4j.properties 100% 2712 1.9MB/s 00:00 test-patch.properties 100% 855 930.7KB/s 00:00 test-github-pr.sh 100% 24KB 9.7MB/s 00:00 test-patch.sh 100% 24KB 19.4MB/s 00:00 pom.xml 100% 10KB 7.6MB/s 00:00 LICENSE.txt 100% 12KB 5.9MB/s 00:00 NOTICE.txt 100% 3132 1.2MB/s 00:00 README.md 100% 1765 1.4MB/s 00:00 README_packaging.txt 100% 1770 2.5MB/s 00:00 build.xml 100% 95KB 39.5MB/s 00:00 ivy.xml 100% 10KB 8.2MB/s 00:00 ivysettings.xml 100% 1709 1.3MB/s 00:00 pom.xml 100% 31KB 8.9MB/s 00:00 zookeeper-3.4.14.jar 100% 1480KB 41.5MB/s 00:00 zookeeper-3.4.14.jar.md5 100% 33 25.6KB/s 00:00 zookeeper-3.4.14.jar.sha1 100% 41 60.7KB/s 00:00 zookeeper-3.4.14.jar.asc 100% 836 1.2MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/zookeeper-3.4.14/ node103.yinzhengjie.org.cn:/yinzhengjie/softwares/ ........ log.42 100% 184 128.3KB/s 00:00 log.63b 100% 48KB 22.7MB/s 00:00 snapshot.0 100% 296 277.6KB/s 00:00 snapshot.272 100% 55KB 26.5MB/s 00:00 snapshot.273 100% 55KB 24.3MB/s 00:00 snapshot.639 100% 140KB 27.2MB/s 00:00 snapshot.83f 100% 4824 3.5MB/s 00:00 minikdc-krb5.conf 100% 1103 1.3MB/s 00:00 minikdc.ldiff 100% 1630 2.2MB/s 00:00 log.100000001 100% 2401KB 50.2MB/s 00:00 log.100001bf0 100% 1001KB 59.6MB/s 00:00 snapshot.100000000 100% 73 17.1KB/s 00:00 snapshot.100001bec 100% 2507KB 61.6MB/s 00:00 check_compatibility.py 100% 7212 4.7MB/s 00:00 checkstyle-noframes-sorted.xsl 100% 5489 6.5MB/s 00:00 checkstyle.xml 100% 7489 7.3MB/s 00:00 findbugsExcludeFile.xml 100% 5027 6.6MB/s 00:00 log4j.properties 100% 2712 1.9MB/s 00:00 test-patch.properties 100% 855 930.7KB/s 00:00 test-github-pr.sh 100% 24KB 9.7MB/s 00:00 test-patch.sh 100% 24KB 19.4MB/s 00:00 pom.xml 100% 10KB 7.6MB/s 00:00 LICENSE.txt 100% 12KB 5.9MB/s 00:00 NOTICE.txt 100% 3132 1.2MB/s 00:00 README.md 100% 1765 1.4MB/s 00:00 README_packaging.txt 100% 1770 2.5MB/s 00:00 build.xml 100% 95KB 39.5MB/s 00:00 ivy.xml 100% 10KB 8.2MB/s 00:00 ivysettings.xml 100% 1709 1.3MB/s 00:00 pom.xml 100% 31KB 8.9MB/s 00:00 zookeeper-3.4.14.jar 100% 1480KB 41.5MB/s 00:00 zookeeper-3.4.14.jar.md5 100% 33 25.6KB/s 00:00 zookeeper-3.4.14.jar.sha1 100% 41 60.7KB/s 00:00 zookeeper-3.4.14.jar.asc 100% 836 1.2MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
7>.启动zookeeper集群
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps' node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> Jps node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> Jps node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> Jps [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# xzk.sh start 启动服务 ========== node101.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED ========== node102.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED ========== node103.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# xzk.sh status 查看状态 ========== node101.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: follower ========== node102.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: leader ========== node103.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: follower [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps' node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> QuorumPeerMain Jps node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> QuorumPeerMain Jps node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> QuorumPeerMain Jps [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh Connecting to localhost:2181 2019-04-12 09:23:04,270 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-04-12 09:23:04,272 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-12 09:23:04,273 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf: 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root 2019-04-12 09:23:04,277 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89 Welcome to ZooKeeper! 2019-04-12 09:23:04,303 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-12 09:23:04,365 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2019-04-12 09:23:04,395 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x6500007f14d00000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] ls / [zookeeper] [zk: localhost:2181(CONNECTED) 1] ls /zookeeper [quota] [zk: localhost:2181(CONNECTED) 2] ls /zookeeper/quota [] [zk: localhost:2181(CONNECTED) 3] quit Quitting... 2019-04-12 09:23:13,101 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x6500007f14d00000 closed 2019-04-12 09:23:13,102 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x6500007f14d00000 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
四.配置HDFS HA集群
2>.将集群的完全分布式配置备份一份，再行修改操作
[root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/hadoop-2.9.2/etc/ total 4 drwxr-xr-x. 2 501 dialout 4096 Apr 18 16:03 hadoop [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# cp -r /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/ /yinzhengjie/softwares/hadoop-2.9.2/etc/full [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/hadoop-2.9.2/etc/ total 8 drwxr-xr-x 2 root root 4096 Apr 18 16:26 full drwxr-xr-x. 2 501 dialout 4096 Apr 18 16:03 hadoop [root@node101.yinzhengjie.org.cn ~]#
3>.配置hadoop-env-sh配置文件
[root@node101.yinzhengjie.org.cn ~]# which java #由于我们实验环境按照的是OpenJDK，默认的按照后的命令被放在了/usr下，因此我们应该在Hadoop的环境变啦配置文件中指定本季的Java命令的路径位置！ /usr/bin/java [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# grep JAVA_HOME /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/hadoop-env.sh | grep -v ^# export JAVA_HOME=/usr [root@node101.yinzhengjie.org.cn ~]#
4>.配置core-site.xml配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/core-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://yinzhengjie-hdfs</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/data/hadoop/ha</value> </property> </configuration> <!-- core-site.xml配置文件的作用： 用于定义系统级别的参数，如HDFS URL、Hadoop的临时目录以及用于rack-aware集群中的配置文件的配置等，此中的参数定义会覆盖core-default.xml文件中的默认配置。 fs.defaultFS 参数的作用： #声明namenode的地址，相当于声明hdfs文件系统。 hadoop.tmp.dir 参数的作用： #声明hadoop工作目录的地址。 --> [root@node101.yinzhengjie.org.cn ~]#
5>配置hdfs-site.xml配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/hdfs-site.xml <configuration> <!-- 完全分布式集群名称,注意这个名称需要和core-site.xml中定义的fs.defaultFS要一致哟！ --> <property> <name>dfs.nameservices</name> <value>yinzhengjie-hdfs</value> </property> <!-- 集群中NameNode节点都有哪些 --> <property> <name>dfs.ha.namenodes.yinzhengjie-hdfs</name> <value>namenode1,namenode2</value> </property> <!-- namenode1的RPC通信地址 --> <property> <name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode1</name> <value>node101.yinzhengjie.org.cn:9000</value> </property> <!-- namenode2的RPC通信地址 --> <property> <name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode2</name> <value>node103.yinzhengjie.org.cn:9000</value> </property> <!-- namenode1的http通信地址 --> <property> <name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode1</name> <value>node101.yinzhengjie.org.cn:50070</value> </property> <!-- namenode2的http通信地址 --> <property> <name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode2</name> <value>node103.yinzhengjie.org.cn:50070</value> </property> <!-- 指定NameNode元数据在JournalNode上的存放位置 --> <property> <name>dfs.namenode.shared.edits.dir</name> <value>qjournal://node103.yinzhengjie.org.cn:8485;node102.yinzhengjie.org.cn:8485;node101.yinzhengjie.org.cn:8485/yinzhengjie-hdfs</value> </property> <!-- 配置隔离机制，即同一时刻只能有一台服务器对外响应 --> <property> <name>dfs.ha.fencing.methods</name> <value>sshfence</value> </property> <!-- 使用隔离机制时需要ssh无秘钥登录--> <property> <name>dfs.ha.fencing.ssh.private-key-files</name> <value>/root/.ssh/id_rsa</value> </property> <!-- 声明journalnode服务器存储目录--> <property> <name>dfs.journalnode.edits.dir</name> <value>/data/hadoop/jn</value> </property> <!-- 关闭权限检查--> <property> <name>dfs.permissions.enable</name> <value>false</value> </property> <property> <name>dfs.ha.automatic-failover.enabled</name> <value>true</value> </property> <property> <name>ha.zookeeper.quorum</name> <value>node101.yinzhengjie.org.cn:2181,node102.yinzhengjie.org.cn:2181,node103.yinzhengjie.org.cn:2181</value> </property> <!-- 访问代理类：client，mycluster，active配置失败自动切换实现方式--> <property> <name>dfs.client.failover.proxy.provider.yinzhengjie-hdfs</name> <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> </property> </configuration> [root@node101.yinzhengjie.org.cn ~]#
6>.拷贝配置好的Hadoop环境到其他节点
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/ node102.yinzhengjie.org.cn:/yinzhengjie/softwares/hadoop-2.9.2/etc/ mapred-site.xml.template 100% 758 423.1KB/s 00:00 hadoop-metrics2.properties 100% 2598 2.0MB/s 00:00 ssl-server.xml.example 100% 2697 4.0MB/s 00:00 mapred-site.xml 100% 769 822.6KB/s 00:00 slaves 100% 81 104.5KB/s 00:00 yarn-site.xml 100% 693 1.3MB/s 00:00 hadoop-env.sh-2019-04-11 100% 4969 5.5MB/s 00:00 yarn-env.cmd 100% 2250 4.3MB/s 00:00 ssl-client.xml.example 100% 2316 4.1MB/s 00:00 httpfs-env.sh 100% 2230 4.7MB/s 00:00 kms-acls.xml 100% 3518 6.2MB/s 00:00 httpfs-signature.secret 100% 21 41.3KB/s 00:00 capacity-scheduler.xml 100% 7861 12.6MB/s 00:00 yarn-env.sh 100% 4876 10.3MB/s 00:00 mapred-env.sh 100% 1507 3.0MB/s 00:00 httpfs-log4j.properties 100% 1657 2.7MB/s 00:00 kms-env.sh 100% 3139 4.2MB/s 00:00 kms-site.xml 100% 5939 7.1MB/s 00:00 hadoop-env.cmd 100% 4133 7.8MB/s 00:00 container-executor.cfg 100% 1211 2.3MB/s 00:00 mapred-queues.xml.template 100% 4113 7.1MB/s 00:00 kms-log4j.properties 100% 1788 3.3MB/s 00:00 hadoop-policy.xml 100% 10KB 12.1MB/s 00:00 hdfs-site.xml 100% 2283 3.9MB/s 00:00 httpfs-site.xml 100% 620 1.2MB/s 00:00 core-site.xml 100% 761 1.4MB/s 00:00 hadoop-metrics.properties 100% 2490 3.5MB/s 00:00 hadoop-env.sh 100% 4961 8.7MB/s 00:00 mapred-env.cmd 100% 1076 1.9MB/s 00:00 configuration.xsl 100% 1335 2.4MB/s 00:00 log4j.properties 100% 14KB 17.0MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop/ node103.yinzhengjie.org.cn:/yinzhengjie/softwares/hadoop-2.9.2/etc/ mapred-site.xml.template 100% 758 639.0KB/s 00:00 hadoop-metrics2.properties 100% 2598 2.9MB/s 00:00 ssl-server.xml.example 100% 2697 4.1MB/s 00:00 mapred-site.xml 100% 769 2.4MB/s 00:00 slaves 100% 81 119.6KB/s 00:00 yarn-site.xml 100% 693 1.7MB/s 00:00 hadoop-env.sh-2019-04-11 100% 4969 4.5MB/s 00:00 yarn-env.cmd 100% 2250 2.9MB/s 00:00 ssl-client.xml.example 100% 2316 3.1MB/s 00:00 httpfs-env.sh 100% 2230 4.2MB/s 00:00 kms-acls.xml 100% 3518 6.5MB/s 00:00 httpfs-signature.secret 100% 21 42.6KB/s 00:00 capacity-scheduler.xml 100% 7861 9.3MB/s 00:00 yarn-env.sh 100% 4876 9.1MB/s 00:00 mapred-env.sh 100% 1507 3.3MB/s 00:00 httpfs-log4j.properties 100% 1657 3.4MB/s 00:00 kms-env.sh 100% 3139 4.7MB/s 00:00 kms-site.xml 100% 5939 10.5MB/s 00:00 hadoop-env.cmd 100% 4133 6.5MB/s 00:00 container-executor.cfg 100% 1211 2.4MB/s 00:00 mapred-queues.xml.template 100% 4113 8.7MB/s 00:00 kms-log4j.properties 100% 1788 2.8MB/s 00:00 hadoop-policy.xml 100% 10KB 13.1MB/s 00:00 hdfs-site.xml 100% 2283 4.8MB/s 00:00 httpfs-site.xml 100% 620 1.4MB/s 00:00 core-site.xml 100% 761 2.1MB/s 00:00 hadoop-metrics.properties 100% 2490 5.2MB/s 00:00 hadoop-env.sh 100% 4961 11.7MB/s 00:00 mapred-env.cmd 100% 1076 2.2MB/s 00:00 configuration.xsl 100% 1335 2.8MB/s 00:00 log4j.properties 100% 14KB 17.6MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
7>.在各个JournalNode节点上，启动Journalnode服务
[root@node101.yinzhengjie.org.cn ~]# hadoop-daemon.sh start journalnode starting journalnode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-journalnode-node101.yinzhengjie.org.cn.out [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps 15153 Jps 14983 JournalNode 7629 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# hadoop-daemon.sh start journalnode starting journalnode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-journalnode-node102.yinzhengjie.org.cn.out [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]# jps 11949 QuorumPeerMain 19166 JournalNode 19247 Jps [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# hadoop-daemon.sh start journalnode starting journalnode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-journalnode-node103.yinzhengjie.org.cn.out [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# jps 5040 JournalNode 30132 QuorumPeerMain 5146 Jps [root@node103.yinzhengjie.org.cn ~]#
8>.在任意一个NameNode节点上进行格式化并启动
[root@node101.yinzhengjie.org.cn ~]# hdfs namenode -format
[root@node101.yinzhengjie.org.cn ~]# hadoop-daemon.sh start namenode starting namenode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-namenode-node101.yinzhengjie.org.cn.out [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps 17559 JournalNode 18649 NameNode 18747 Jps 7629 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]#
9>.在另外一个NameNode节点上同步上一步NameNode的元数据信息并启动
[root@node103.yinzhengjie.org.cn ~]# hdfs namenode -bootstrapStandby 19/04/18 17:11:23 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = node103.yinzhengjie.org.cn/172.30.1.103 STARTUP_MSG: args = [-bootstrapStandby] STARTUP_MSG: version = 2.9.2 STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/activation-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jersey-json-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/mockito-all-1.8.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/zookeeper-3.4.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jets3t-0.9.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jetty-sslengine-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-digester-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-compress-1.4.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/stax-api-1.0-2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-net-3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jetty-util-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-codec-1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/xz-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/curator-client-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/servlet-api-2.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/json-smart-1.3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jetty-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/hadoop-annotations-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/curator-framework-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-configuration-1.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/xmlenc-0.52.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/hadoop-nfs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/hadoop-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/hadoop-common-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-native-client-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-native-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-nfs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-rbf-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-rbf-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-client-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/apacheds-i18n-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-client-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/curator-recipes-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/api-asn1-api-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-beanutils-1.7.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/api-util-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/activation-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-json-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-beanutils-core-1.8.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jets3t-0.9.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jetty-sslengine-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-digester-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-net-3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-codec-1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/xz-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/metrics-core-3.0.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/curator-client-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/servlet-api-2.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/guice-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/java-xmlbuilder-0.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/json-smart-1.3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jetty-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/curator-framework-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-configuration-1.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/xmlenc-0.52.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-registry-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-tests-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-router-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-api-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/xz-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/guice-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/hadoop-annotations-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.9.2.jar:/contrib/capacity-scheduler/*.jar STARTUP_MSG: build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r 826afbeae31ca687bc2f8471dc841b66ed2c6704; compiled by 'ajisaka' on 2018-11-13T12:42Z STARTUP_MSG: java = 1.8.0_201 ************************************************************/ 19/04/18 17:11:23 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] 19/04/18 17:11:23 INFO namenode.NameNode: createNameNode [-bootstrapStandby] ===================================================== About to bootstrap Standby ID namenode2 from: Nameservice ID: yinzhengjie-hdfs Other Namenode ID: namenode1 Other NN's HTTP address: http://node101.yinzhengjie.org.cn:50070 Other NN's IPC address: node101.yinzhengjie.org.cn/172.30.1.101:9000 Namespace ID: 688234594 Block pool ID: BP-1646329568-172.30.1.101-1555578389072 Cluster ID: CID-eb844a86-4ab9-4a0b-94f8-e8627f99d958 Layout version: -63 isUpgradeFinalized: true ===================================================== 19/04/18 17:11:24 INFO common.Storage: Storage directory /data/hadoop/ha/dfs/name has been successfully formatted. 19/04/18 17:11:24 INFO namenode.FSEditLog: Edit logging is async:true 19/04/18 17:11:24 INFO namenode.TransferFsImage: Opening connection to http://node101.yinzhengjie.org.cn:50070/imagetransfer?getimage=1&txid=0&storageInfo=-63:688234594:1555578389072:CID-eb844a86-4ab9-4a0b-94f8-e8627f99d958&bootstrapstandby=true 19/04/18 17:11:24 INFO namenode.TransferFsImage: Image Transfer timeout configured to 60000 milliseconds 19/04/18 17:11:24 INFO namenode.TransferFsImage: Combined time for fsimage download and fsync to all disks took 0.00s. The fsimage download took 0.00s at 0.00 KB/s. Synchronous (fsync) write to disk of /data/hadoop/ha/dfs/name/current/fsimage.ckpt_0000000000000000000 took 0.00s. 19/04/18 17:11:24 INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000000 size 323 bytes. 19/04/18 17:11:24 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at node103.yinzhengjie.org.cn/172.30.1.103 ************************************************************/ [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# hadoop-daemon.sh start namenode starting namenode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-namenode-node103.yinzhengjie.org.cn.out [root@node103.yinzhengjie.org.cn ~]# jps 7506 JournalNode 30132 QuorumPeerMain 9544 Jps 9452 NameNode [root@node103.yinzhengjie.org.cn ~]#
10>.启动所有的datanode
[root@node101.yinzhengjie.org.cn ~]# jps 17559 JournalNode 18649 NameNode 26219 Jps 7629 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hadoop-daemon.sh start datanode starting datanode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-datanode-node101.yinzhengjie.org.cn.out [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps 17559 JournalNode 18649 NameNode 26393 Jps 7629 QuorumPeerMain 26286 DataNode [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# jps 11949 QuorumPeerMain 29631 Jps 21471 JournalNode [root@node102.yinzhengjie.org.cn ~]# hadoop-daemon.sh start datanode starting datanode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-datanode-node102.yinzhengjie.org.cn.out [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]# jps 11949 QuorumPeerMain 29677 DataNode 29791 Jps 21471 JournalNode [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# jps 7506 JournalNode 30132 QuorumPeerMain 16183 Jps 9452 NameNode [root@node103.yinzhengjie.org.cn ~]# hadoop-daemon.sh start datanode starting datanode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-datanode-node103.yinzhengjie.org.cn.out [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# jps 16224 DataNode 7506 JournalNode 30132 QuorumPeerMain 9452 NameNode 16332 Jps [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]#
11>.手动将node101.yinzhengjie.org.cn节点切换为Active状态
[root@node101.yinzhengjie.org.cn ~]# hdfs haadmin -getServiceState namenode1 standby [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs haadmin -transitionToActive namenode1 --forcemanual #我们可以手动强制切换！但会他会弹出提示信息，说是手动切换可能会导致脑裂的发生！ You have specified the --forcemanual flag. This flag is dangerous, as it can induce a split-brain scenario that WILL CORRUPT your HDFS namespace, possibly irrecoverably. It is recommended not to use this flag, but instead to shut down the cluster and disable automatic failover if you prefer to manually manage your HA state. You may abort safely by answering 'n' or hitting ^C now. Are you sure you want to continue? (Y or N) y #问我们是否迹象，我们输入y即可！ 19/04/18 17:50:29 WARN ha.HAAdmin: Proceeding with manual HA state management even though automatic failover is enabled for NameNode at node103.yinzhengjie.org.cn/172.30.1.103:9000 19/04/18 17:50:29 WARN ha.HAAdmin: Proceeding with manual HA state management even though automatic failover is enabled for NameNode at node101.yinzhengjie.org.cn/172.30.1.101:9000 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs haadmin -getServiceState namenode1 active [root@node101.yinzhengjie.org.cn ~]#
12>.配置HDFS HA自动故障转移
[root@node101.yinzhengjie.org.cn ~]# stop-dfs.sh Stopping namenodes on [node101.yinzhengjie.org.cn node103.yinzhengjie.org.cn] node101.yinzhengjie.org.cn: stopping namenode node103.yinzhengjie.org.cn: stopping namenode node103.yinzhengjie.org.cn: stopping datanode node101.yinzhengjie.org.cn: stopping datanode node102.yinzhengjie.org.cn: stopping datanode Stopping journal nodes [node103.yinzhengjie.org.cn node102.yinzhengjie.org.cn node101.yinzhengjie.org.cn] node103.yinzhengjie.org.cn: stopping journalnode node102.yinzhengjie.org.cn: stopping journalnode node101.yinzhengjie.org.cn: stopping journalnode Stopping ZK Failover Controllers on NN hosts [node101.yinzhengjie.org.cn node103.yinzhengjie.org.cn] node103.yinzhengjie.org.cn: no zkfc to stop node101.yinzhengjie.org.cn: no zkfc to stop [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# xzk.sh start 启动服务 ========== node101.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED ========== node102.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED ========== node103.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# xzk.sh status 查看状态 ========== node101.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: leader ========== node102.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: follower ========== node103.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: follower [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs zkfc -formatZK 19/04/18 18:01:30 INFO tools.DFSZKFailoverController: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting DFSZKFailoverController STARTUP_MSG: host = node101.yinzhengjie.org.cn/172.30.1.101 STARTUP_MSG: args = [-formatZK] STARTUP_MSG: version = 2.9.2 STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/zookeeper-3.4.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/xz-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-configuration-1.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/hadoop-annotations-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/xmlenc-0.52.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-compress-1.4.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/curator-framework-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-codec-1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jetty-sslengine-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/json-smart-1.3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/mockito-all-1.8.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-net-3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/stax-api-1.0-2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/servlet-api-2.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jetty-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jets3t-0.9.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-digester-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/curator-client-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jersey-json-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/activation-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jetty-util-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/hadoop-nfs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/hadoop-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/hadoop-common-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-native-client-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-nfs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-client-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-native-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-rbf-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-rbf-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/guice-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/api-util-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/xz-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-configuration-1.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-beanutils-1.7.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/curator-recipes-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/xmlenc-0.52.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/java-xmlbuilder-0.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/curator-framework-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-codec-1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jetty-sslengine-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/json-smart-1.3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/apacheds-i18n-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/metrics-core-3.0.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-net-3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/api-asn1-api-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/servlet-api-2.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jetty-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jets3t-0.9.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-digester-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-client-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/curator-client-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-json-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-beanutils-core-1.8.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/activation-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-api-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-router-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-tests-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-registry-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/guice-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/xz-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/hadoop-annotations-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.2.jar:/contrib/capacity-scheduler/*.jar STARTUP_MSG: build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r 826afbeae31ca687bc2f8471dc841b66ed2c6704; compiled by 'ajisaka' on 2018-11-13T12:42Z STARTUP_MSG: java = 1.8.0_201 ************************************************************/ 19/04/18 18:01:30 INFO tools.DFSZKFailoverController: registered UNIX signal handlers for [TERM, HUP, INT] 19/04/18 18:01:30 INFO tools.DFSZKFailoverController: Failover controller configured for NameNode NameNode at node101.yinzhengjie.org.cn/172.30.1.101:9000 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:host.name=node101.yinzhengjie.org.cn 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_201 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64/jre 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/yinzhengjie/softwares/hadoop-2.9.2/etc/hadoop:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/zookeeper-3.4.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/xz-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-configuration-1.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/hadoop-annotations-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/xmlenc-0.52.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-compress-1.4.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/curator-framework-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-codec-1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jetty-sslengine-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/json-smart-1.3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/mockito-all-1.8.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-net-3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/stax-api-1.0-2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/servlet-api-2.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jetty-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jets3t-0.9.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-digester-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/curator-client-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jersey-json-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/activation-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/jetty-util-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/hadoop-nfs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/hadoop-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/common/hadoop-common-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-native-client-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-nfs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-client-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-native-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-rbf-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-rbf-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/hdfs/hadoop-hdfs-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/guice-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/api-util-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/xz-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-configuration-1.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-beanutils-1.7.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/curator-recipes-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/xmlenc-0.52.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/java-xmlbuilder-0.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/curator-framework-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-codec-1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jetty-sslengine-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/json-smart-1.3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/apacheds-i18n-2.0.0-M15.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/metrics-core-3.0.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-net-3.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/api-asn1-api-1.0.0-M20.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/servlet-api-2.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jetty-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jets3t-0.9.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-digester-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-client-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/curator-client-2.7.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jersey-json-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-beanutils-core-1.8.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/activation-1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-api-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-router-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-tests-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-client-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-registry-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/guice-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/asm-3.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/xz-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/hadoop-annotations-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.9.2-tests.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.9.2.jar:/yinzhengjie/softwares/hadoop-2.9.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.2.jar:/contrib/capacity-scheduler/*.jar 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/yinzhengjie/softwares/hadoop-2.9.2/lib/native 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.0-957.el7.x86_64 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:user.name=root 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:user.home=/root 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Client environment:user.dir=/root 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=node101.yinzhengjie.org.cn:2181,node102.yinzhengjie.org.cn:2181,node103.yinzhengjie.org.cn:2181 sessionTimeout=10000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@654f0d9c 19/04/18 18:01:30 INFO zookeeper.ClientCnxn: Opening socket connection to server node102.yinzhengjie.org.cn/172.30.1.102:2181. Will not attempt to authenticate using SASL (unknown error) 19/04/18 18:01:30 INFO zookeeper.ClientCnxn: Socket connection established to node102.yinzhengjie.org.cn/172.30.1.102:2181, initiating session 19/04/18 18:01:30 INFO zookeeper.ClientCnxn: Session establishment complete on server node102.yinzhengjie.org.cn/172.30.1.102:2181, sessionid = 0x6600007830b30000, negotiated timeout = 10000 19/04/18 18:01:30 INFO ha.ActiveStandbyElector: Session connected. 19/04/18 18:01:30 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/yinzhengjie-hdfs in ZK. 19/04/18 18:01:30 INFO zookeeper.ZooKeeper: Session: 0x6600007830b30000 closed 19/04/18 18:01:30 INFO zookeeper.ClientCnxn: EventThread shut down 19/04/18 18:01:30 INFO tools.DFSZKFailoverController: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DFSZKFailoverController at node101.yinzhengjie.org.cn/172.30.1.101 ************************************************************/ [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# jps 31104 Jps 29947 QuorumPeerMain 30718 ZooKeeperMain [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# start-dfs.sh Starting namenodes on [node101.yinzhengjie.org.cn node103.yinzhengjie.org.cn] node103.yinzhengjie.org.cn: starting namenode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-namenode-node103.yinzhengjie.org.cn.out node101.yinzhengjie.org.cn: starting namenode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-namenode-node101.yinzhengjie.org.cn.out node103.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-datanode-node103.yinzhengjie.org.cn.out node102.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-datanode-node102.yinzhengjie.org.cn.out node101.yinzhengjie.org.cn: starting datanode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-datanode-node101.yinzhengjie.org.cn.out Starting journal nodes [node103.yinzhengjie.org.cn node102.yinzhengjie.org.cn node101.yinzhengjie.org.cn] node103.yinzhengjie.org.cn: starting journalnode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-journalnode-node103.yinzhengjie.org.cn.out node101.yinzhengjie.org.cn: starting journalnode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-journalnode-node101.yinzhengjie.org.cn.out node102.yinzhengjie.org.cn: starting journalnode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-journalnode-node102.yinzhengjie.org.cn.out Starting ZK Failover Controllers on NN hosts [node101.yinzhengjie.org.cn node103.yinzhengjie.org.cn] node103.yinzhengjie.org.cn: starting zkfc, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-zkfc-node103.yinzhengjie.org.cn.out node101.yinzhengjie.org.cn: starting zkfc, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-zkfc-node101.yinzhengjie.org.cn.out [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps 31376 DataNode 31600 JournalNode 29947 QuorumPeerMain 31246 NameNode 30718 ZooKeeperMain 31902 Jps 31807 DFSZKFailoverController [root@node101.yinzhengjie.org.cn ~]#
五.验证HDFS HA的可用性（注意，如果你执行下面的操作之前，请确认你两个NameNode之间是否配置了免密码登陆！否则可能看不到我博客中展示的效果哟！）
1>.将Active NameNode进程杀死，观察Standby NameNode是否会自动切换
[root@node101.yinzhengjie.org.cn ~]# hdfs haadmin -getServiceState namenode1 #查看当前节点的状态 active [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# kill -9 `jps | grep NameNode | awk '{print $1}' ` #直接杀掉NameNode进程 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs haadmin -getServiceState namenode1 19/04/18 18:15:24 INFO ipc.Client: Retrying connect to server: node101.yinzhengjie.org.cn/172.30.1.101:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=1, sleepTime=1000 MILLISECONDS) Operation failed: Call From node101.yinzhengjie.org.cn/172.30.1.101 to node101.yinzhengjie.org.cn:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused [root@node101.yinzhengjie.org.cn ~]#
[zk: localhost:2181(CONNECTED) 0] ls /hadoop-ha/yinzhengjie-hdfs [ActiveBreadCrumb, ActiveStandbyElectorLock] [zk: localhost:2181(CONNECTED) 1]
[zk: localhost:2181(CONNECTED) 1] get /hadoop-ha/yinzhengjie-hdfs/ActiveStandbyElectorLock yinzhengjie-hdfs namenode2node103.yinzhengjie.org.cn �F(�> cZxid = 0x2000001fc ctime = Thu Apr 18 18:34:24 CST 2019 mZxid = 0x2000001fc mtime = Thu Apr 18 18:34:24 CST 2019 pZxid = 0x2000001fc cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x65001122c5ce0036 dataLength = 63 numChildren = 0 [zk: localhost:2181(CONNECTED) 2]
[zk: localhost:2181(CONNECTED) 2] get /hadoop-ha/yinzhengjie-hdfs/ActiveBreadCrumb yinzhengjie-hdfs namenode2node103.yinzhengjie.org.cn �F(�> cZxid = 0x200000009 ctime = Thu Apr 18 18:02:56 CST 2019 mZxid = 0x2000001fd mtime = Thu Apr 18 18:34:25 CST 2019 pZxid = 0x200000009 cversion = 0 dataVersion = 3 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 63 numChildren = 0 [zk: localhost:2181(CONNECTED) 3]
2>.将Active NameNode机器网络断开，观察Standby NameNode是否会自动切换
[root@node101.yinzhengjie.org.cn ~]# hadoop-daemon.sh start namenode #首先我们把刚刚杀死的NamenNode进程启动起来。 starting namenode, logging to /yinzhengjie/softwares/hadoop-2.9.2/logs/hadoop-root-namenode-node101.yinzhengjie.org.cn.out [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs haadmin -getServiceState namenode1 #启动后，发现其状态为standby。 standby [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ssh node103.yinzhengjie.org.cn #远程到active的服务器上，我们要搞点事情。 Last login: Thu Apr 18 16:52:18 2019 from 172.30.1.2 [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# hdfs haadmin -getServiceState namenode2 active [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# reboot #我们把服务器重启一下，模拟Active节点的主机故障！ Connection to node103.yinzhengjie.org.cn closed by remote host. Connection to node103.yinzhengjie.org.cn closed. [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs haadmin -getServiceState namenode1 #再次查看，发现刚刚的Standby状态转换成Active啦！说明切换成功！ active [root@node101.yinzhengjie.org.cn ~]#
[zk: localhost:2181(CONNECTED) 11] get /hadoop-ha/yinzhengjie-hdfs/ActiveStandbyElectorLock yinzhengjie-hdfs namenode1node101.yinzhengjie.org.cn �F(�> cZxid = 0x2000001f7 ctime = Thu Apr 18 18:23:13 CST 2019 mZxid = 0x2000001f7 mtime = Thu Apr 18 18:23:13 CST 2019 pZxid = 0x2000001f7 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x6700112119110038 dataLength = 63 numChildren = 0 [zk: localhost:2181(CONNECTED) 12]
[zk: localhost:2181(CONNECTED) 12] get /hadoop-ha/yinzhengjie-hdfs/ActiveBreadCrumb yinzhengjie-hdfs namenode1node101.yinzhengjie.org.cn �F(�> cZxid = 0x200000009 ctime = Thu Apr 18 18:02:56 CST 2019 mZxid = 0x2000001f8 mtime = Thu Apr 18 18:23:30 CST 2019 pZxid = 0x200000009 cversion = 0 dataVersion = 2 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 63 numChildren = 0 [zk: localhost:2181(CONNECTED) 13]
当你的才华还撑不起你的野心的时候，你就应该静下心来学习。当你的能力还驾驭不了你的目标的时候，你就应该沉下心来历练。问问自己，想要怎样的人生。
欢迎加入基础架构自动化运维：598432640，大数据SRE进阶之路：959042252，DevOps进阶之路：526991186
