5.1.1 搭建Hadoop+HBase+ZooKeeper分布式集群环境

在4.1.1 Hadoop2.5.2 HA高可靠性集群搭建(Hadoop+Zookeeper)章节中创建了Hadoop+zookeeper HA,本章节在此集成上部署Hbase。

一、环境配置

 略。。（参考4.1.1 Hadoop2.5.2 HA高可靠性集群搭建(Hadoop+Zookeeper)章节）

二、Hadoop的安装和配置

  略。。（参考4.1.1 Hadoop2.5.2 HA高可靠性集群搭建(Hadoop+Zookeeper)章节）

三、ZooKeeper集群安装

  略。。（参考4.1.1 Hadoop2.5.2 HA高可靠性集群搭建(Hadoop+Zookeeper)章节）

 

四、HBase集群的安装和配置


1.    解压缩hbase-1.3.1-bin.tar.gz并重命名为hbase, 编辑/hbase/conf/hbase-env.sh


 

# The java implementation to use.  Java 1.7+ required.
export JAVA_HOME=/usr/jdk1.8

# Extra Java CLASSPATH elements.  Optional.
export HBASE_CLASSPATH=/usr/hadoop/etc/hadoop

# The maximum amount of heap to use. Default is left to JVM default.
# export HBASE_HEAPSIZE=1G

# Uncomment below if you intend to use off heap cache. For example, to allocate 8G of 
# offheap, set the value to "8G".
# export HBASE_OFFHEAPSIZE=1G

# Extra Java runtime options.
# Below are what we set by default.  May only work with SUN JVM.
# For more on why as well as other possible settings,
# see http://wiki.apache.org/hadoop/PerformanceTuning
export HBASE_OPTS="-XX:+UseConcMarkSweepGC"

# Configure PermSize. Only needed in JDK7. You can safely remove it for JDK8+
export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m"
export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m"
export HBASE_MANAGES_ZK=false
export HBASE_PID_DIR=/home/hadoopdir/pids 

 

 

 

2.    编辑hbase-site.xml

 

<configuration>
<property>
        <name>hbase.rootdir</name>
        <value>hdfs://ns/hbase</value>
    </property>
    <property>
        <name>hbase.master</name>
        <value>60000</value>
    </property>
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
    </property>
    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2281</value>
    </property>
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>master,slave1,slave2</value>
    </property>
    <property>
        <name>zookeeper.session.timeout</name>
        <value>60000000</value>
    </property>
    <property>
        <name>dfs.support.append</name>
        <value>true</value>
    </property>
</configuration>

 

3.   修改regionservers文件，增加:

slave1
slave2

4.    分发hbase到其他两个节点

 

 

五、启动hbase

[hadoop@Master ~]$ start-hbase.sh
starting master, logging to /usr/hadoop/hbase-1.3.1/logs/hbase-hadoop-master-Master.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
slave2: starting regionserver, logging to /usr/hadoop/hbase-1.3.1/bin/../logs/hbase-hadoop-regionserver-Slave2.out
slave1: starting regionserver, logging to /usr/hadoop/hbase-1.3.1/bin/../logs/hbase-hadoop-regionserver-Slave1.out
slave2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
slave2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
slave1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
slave1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0

查询各个节点的集群进程情况

[hadoop@Master ~]$ jps
QuorumPeerMain
HMaster
Jps
ResourceManager
DFSZKFailoverController
NodeManager
NameNode
JournalNode
DataNode


[hadoop@Slave1 ~]$ jps
HRegionServer
NameNode
Jps
QuorumPeerMain
DataNode
DFSZKFailoverController
NodeManager
JournalNode

[hadoop@Slave2 conf]$ jps
HRegionServer
Jps
JournalNode
DataNode
NodeManager
QuorumPeerMain

 

进程都已经开启，进入habse shell环境,


 

$ hbase/bin/hbase shell                

 

[hadoop@Master ~]$ hbase shell
2017-07-02 13:18:54,748 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hadoop/hbase-1.3.1/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.3.1, r930b9a55528fe45d8edce7af42fef2d35e77677a, Thu Apr 6 19:36:54 PDT 2017

hbase(main):001:0> status
1 active master, 0 backup masters, 2 servers, 0 dead, 1.0000 average load

hbase(main):002:0>

 

hbase 状态也可以通过网页进行查看：