hadoop 之 nodemanager自动关闭（Secure IO is not possible without native code extensions）

 

场景

安装好hadoop之后，nodemanager自动关闭。查看日志如下：

java.lang.ExceptionInInitializerError
        at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogValue.secureOpenFile(AggregatedLogFormat.java:278)
        at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogValue.write(AggregatedLogFormat.java:230)
        at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogWriter.append(AggregatedLogFormat.java:450)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl$ContainerLogAggregator.doContainerLogAggregation(AppLogAggregatorImpl.java:570)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.uploadLogsForContainers(AppLogAggregatorImpl.java:269)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.doAppLogAggregation(AppLogAggregatorImpl.java:429)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.run(AppLogAggregatorImpl.java:388)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService$2.run(LogAggregationService.java:399)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Secure IO is not possible without native code extensions.
        at org.apache.hadoop.io.SecureIOUtils.<clinit>(SecureIOUtils.java:74)

原因

网上搜了很久，各种改配置文件，都没解决到。就在要放弃的时候，发现glibc的版本和native版本不兼容。

系统环境是glibc-2.12。最后升级到glibc-2.14之后，解决！

升级

因为集群不具备编译glibc的工具。在其他机器，编译好之后拷贝到集群，然后修改链接。

可编译机器编译glibc

tar -zxvf glibc-2.14.tar.gz
cd glibc-2.14
mkdir build
cd build
../configure --prefix=/build
make
make install

集群机器修改链接

su - root
cd /lib64
ln -sf libc.so.6 -> /xxx/glibc-2.14/lib/libc-2.14.so