使用JNI获取hadoop 2.6版本显示Unknown

问题:

c++通过JNI调用hive-jdbc连接hive-server,当hadoop版本为2.6时,总是报错

 

Illegal Hadoop Version: Unknown (expected A.B.* format)

 

原因:

hadoop 2.6 获取版本源码如下(hadoop 3.x 版本修改了此代码,所以不会遇到此问题):

  protected VersionInfo(String component) {
    info = new Properties();
    String versionInfoFile = component + "-version-info.properties";
    InputStream is = null;
    try {
      is = Thread.currentThread().getContextClassLoader()
        .getResourceAsStream(versionInfoFile);
      if (is == null) {
        throw new IOException("Resource not found");
      }
      info.load(is);
    } catch (IOException ex) {
      LogFactory.getLog(getClass()).warn("Could not read '" +
          versionInfoFile + "', " + ex.toString(), ex);
    } finally {
      IOUtils.closeStream(is);
    }
  }

 

调用了函数 getContextClassLoader 获取classloader来加载hadoop版本信息文件,这个函数默认获取到的是system class loader,但是在JNI中使用了函数AttachCurrentThread,函数getContextClassLoader 获取到的将是 bootstrap class loader,加载文件错误,所以会抛出获取hadoop版本异常 

Java Native Interface Specification: 5 - The Invocation API (oracle.com)

 

AttachCurrentThread

jint AttachCurrentThread(JavaVM *vm, void **p_env, void *thr_args);

Attaches the current thread to a Java VM. Returns a JNI interface pointer in the JNIEnv argument.

Trying to attach a thread that is already attached is a no-op.

A native thread cannot be attached simultaneously to two Java VMs.

When a thread is attached to the VM, the context class loader is the bootstrap loader.

 

解决方法:

连接hive之前先调用函数 setContextClassLoader 重新设置 context loader

java.lang.Thread.currentThread().setContextClassLoader(
    java.lang.ClassLoader.getSystemClassLoader()
);

  

posted @ 2021-06-10 15:31  听书人  阅读(162)  评论(0)    收藏  举报