hadoop 1.1.2和 hive 0.10 和hbase-0.94.10-security整合

1.复制jar包,拷贝hbase-0.94.10-security.jar，hbase-0.94.10-security-tests.jar，zookeeper-3.4.5.jar，protobuf-java-2.4.0a.jar到hive/lib下，删掉lib下面旧版的jar包。　

没有做网上资料所说的向hadoop拷贝jar和配置文件的动作，不过只有hbase有了一个下面的错误在60000上

�������|�������)org.apache.hadoop.ipc.RPC$VersionMismatch���>Server IPC version 3 cannot communicate with client version 47

但是在60010端口的监控仍然正常，不明白为什么？希望有同学能给个答案撒

在读了hbase0.94和hadoop1.1.2的IPC相关部分的源代码明白了为什么会有这个错误了，这个 错误会影响RPC的调用，但是不影响编程使用，只需要zookeeper能正常工作即可。

hbase中控制IPC的是HbaseServer.java这个类，关于版本检验的代码如下

private void setupBadVersionResponse(int clientVersion) throws IOException {
      String errMsg = "Server IPC version " + CURRENT_VERSION +
      " cannot communicate with client version " + clientVersion;
      ByteArrayOutputStream buffer = new ByteArrayOutputStream();

      if (clientVersion >= 3) {
        // We used to return an id of -1 which caused server to close the
        // connection without telling the client what the problem was.  Now
        // we return 0 which will keep the socket up -- bad clients, unless
        // they switch to suit the running server -- will fail later doing
        // getProtocolVersion.
        Call fakeCall =  new Call(0, null, this, responder, 0);
        // Versions 3 and greater can interpret this exception
        // response in the same manner
        setupResponse(buffer, fakeCall, Status.FATAL,
            null, VersionMismatch.class.getName(), errMsg);

        responder.doRespond(fakeCall);
      }
    }

版本定义部分的代码

public abstract class HBaseServer implements RpcServer {

  /**
   * The first four bytes of Hadoop RPC connections
   */
  public static final ByteBuffer HEADER = ByteBuffer.wrap("hrpc".getBytes());
  public static final byte CURRENT_VERSION = 3;

然而在hadoop1.1.2中的相关源代码

public abstract class Server {
  private final boolean authorize;
  private boolean isSecurityEnabled;
  
  /**
   * The first four bytes of Hadoop RPC connections
   */
  public static final ByteBuffer HEADER = ByteBuffer.wrap("hrpc".getBytes());
  
  // 1 : Introduce ping and server does not throw away RPCs
  // 3 : Introduce the protocol into the RPC connection header
  // 4 : Introduced SASL security layer
  public static final byte CURRENT_VERSION = 4;

RPC的版本不一致的说，而且hbase中的RPC通信是不是通过hadoop-core-x.jar包来实现的，所以，把jar拷贝到hbase下仍然会产生这个错误，虽然hbase可以正常使用，但是60000端口的RPC服务是不能用的

2.修改hive-site.xml配置文件，添加以下内容

<property>     
   <name>hive.querylog.location</name>     
   <value>/usr/hive/logs</value>     
    </property>
    <property>  
   <name>hive.aux.jars.path</name>   
   <value>file:///usr/local/hive/lib/hive-hbase-handler-0.10.0.jar,file:///usr/local/hive/lib/hbase-0.94.10-security.jar,
file:///usr/local/hive/lib/zookeeper-3.4.5.jar,file:///usr/local/hive/lib/protobuf-java-2.4.0a.jar</value>  
</property>    

3.启动hive，hive -hiveconf hbase.zookeeper.quorum=zookeepermaster(zookeeper的master)

4.开始测试，建一个表试验。

CREATE TABLE hbase_table1(key int, value1 string,value2 int) 
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.columns.mapping" = ":key,c1:value1,c2:value2"
)TBLPROPERTIES("hbase.table.name" = "hbase_table1");　
TBLPROPERTIES参数是可选的，如果不写的话，就默认是hive和hbase中的表名称一致

5.打开hbase，使用describe  'hbase_table1'结果如下图

官网：https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration