hbase 报regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.net.SocketException: Invalid argument处理方法
查看日志报错
017-09-21 16:59:34,161 WARN [regionserver/hbase-4/127.0.1.1:16020] regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying.
2017-09-21 16:59:37,162 INFO [regionserver/hbase-4/127.0.1.1:16020] regionserver.HRegionServer: reportForDuty to master=hbase-1,16000,1505979999780 with port=16020, start
code=15059830602232017-09-21 16:59:37,163 WARN [regionserver/hbase-4/127.0.1.1:16020] regionserver.HRegionServer: error telling master we are up
com.google.protobuf.ServiceException: java.net.SocketException: Invalid argument
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:240)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:898
2) at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2335)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:929)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Invalid argument
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Net.java:454)
at sun.nio.ch.Net.connect(Net.java:446)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupConnection(RpcClientImpl.java:416)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:722)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:909)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1244)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
... 5 more
2017-09-21 16:59:37,163 WARN [regionserver/hbase-4/127.0.1.1:16020] regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying.
报错信息为regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.net.SocketException: Invalid argument
提示SocketException socket异常
查询一下端口信息
[root@hbase-4 logs]# ss -tnl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 127.0.0.1:37781 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 128 *:50010 *:*
LISTEN 0 128 *:50075 *:*
LISTEN 0 128 *:50020 *:*
LISTEN 0 128 :::8040 :::*
LISTEN 0 128 :::8042 :::*
LISTEN 0 50 :::36430 :::*
LISTEN 0 50 ::ffff:127.0.1.1:3888 :::*
LISTEN 0 128 ::ffff:127.0.1.1:16020 :::*
LISTEN 0 128 :::22 :::*
LISTEN 0 128 :::13562 :::*
LISTEN 0 128 :::16030 :::*
LISTEN 0 128 :::42527 :::*
LISTEN 0 50 :::2181 :::*
可以看到127.0.1.1:16020，127.0.1.1:3888，127.0.0.1:37781 是回环IP
查看hosts表
$ cat /etc/hosts
10.39.10.34 hbase-1
10.39.10.35 hbase-2
10.39.21.19 hbase-3
10.39.10.40 hbase-4
127.0.1.1 hbase-4
删除掉127.0.1.1这条记录，重启所有相关服务
$ zkServer.sh stop
$ /opt/hadoop-2.8.1/sbin/hadoop-daemon.sh stop datanode
$ /opt/hadoop-2.8.1/sbin/yarn-daemon.sh stop nodemanager
$ hbase-daemon.sh stop regionserver
$ zkServer.sh start
$ /opt/hadoop-2.8.1/sbin/hadoop-daemon.sh start datanode
$ /opt/hadoop-2.8.1/sbin/yarn-daemon.sh start nodemanager
$ hbase-daemon.sh start regionserver
再次查看端口信息
[root@hbase-4 ~]# ss -tnl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 128 *:50010 *:*
LISTEN 0 128 *:50075 *:*
LISTEN 0 128 *:50020 *:*
LISTEN 0 128 127.0.0.1:43237 *:*
LISTEN 0 128 :::8040 :::*
LISTEN 0 128 :::8042 :::*
LISTEN 0 50 :::37803 :::*
LISTEN 0 50 ::ffff:10.39.10.40:3888 :::*
LISTEN 0 128 ::ffff:10.39.10.40:16020 :::*
LISTEN 0 128 :::22 :::*
LISTEN 0 128 :::13562 :::*
LISTEN 0 128 :::16030 :::*
LISTEN 0 50 :::2181 :::*
LISTEN 0 128 :::34086 :::*
浙公网安备 33010602011771号