hadoop安装
1、tar -zxvf hadoop-2.8.0.tar.gz
[root@localhost java-1.8.0]# useradd hadoop
[root@localhost java-1.8.0]# passwd hadoop
[root@localhost local]# chown hadoop:hadoop hadoop-2.8.0 -R
[root@localhost hadoop-2.8.0]# mkdir tmp
[hadoop@localhost hadoop-2.8.0]$ vim etc/hadoop/hadoop-env.sh
export JAVA_HOME=/usr //由于rpm装好 java已经分布在/usr/bin下面
vim etc/hadoop/core-site.xml
<configuration> <property> <name>fs.defaultFS</name> <value>hdfs://hadoop.master</value> </property> <property> <name>fs.default.name</name> <value>hdfs://hadoop.master:8020</value> </property> </configuration>
[hadoop@localhost hadoop-2.8.0]$ vim etc/hadoop/hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
./bin/hadoop namenode -format
./sbin/start-dfs.sh
./sbin/start-yarn.sh
网页浏览
http://127.0.0.1:50070/dfshealth.html#tab-overview
如果网页打不开,说明执行不成功,可能哪里配置错误了,start-dfs.sh 的时候看下输出log,进行排查错误
jps检查datanode 是否启动
[root@hadoop hadoop-2.8.0]# yum list | grep jdk-devel
[root@hadoop hadoop-2.8.0]# yum install java-1.8.0-openjdk-devel.x86_64
想在宿主机浏览器访问hadoop web页面,开启某个端口
[root@hadoop hadoop]# service iptables start Redirecting to /bin/systemctl start iptables.service Failed to start iptables.service: Unit not found. [root@hadoop hadoop]# firewall-cmd --zone=public --add-port=50070/tcp --permanent success [root@hadoop hadoop]# service iptables restart Redirecting to /bin/systemctl restart iptables.service Failed to restart iptables.service: Unit not found. [root@hadoop hadoop]# [root@hadoop hadoop]# firewall-cmd --reload success
在CentOS 7或RHEL 7或Fedora中防火墙由firewalld来管理,
如果要添加范围例外端口 如 1000-2000
语法命令如下:启用区域端口和协议组合
firewall-cmd [--zone=] --add-port=[-]/ [--timeout=]
此举将启用端口和协议的组合。端口可以是一个单独的端口 或者是一个端口范围 - 。协议可以是 tcp 或 udp。
实际命令如下:
添加
firewall-cmd --zone=public --add-port=80/tcp --permanent (--permanent永久生效,没有此参数重启后失效)
firewall-cmd --zone=public --add-port=1000-2000/tcp --permanent
重新载入
firewall-cmd --reload
查看
firewall-cmd --zone= public --query-port=80/tcp
删除
firewall-cmd --zone= public --remove-port=80/tcp --permanent
重大bug,datanode服务连接不到namenode,查看namenode 8030端口是否有在监听
<property> <name>dfs.namenode.servicerpc-bind-host</name> <value>0.0.0.0</value> <description> The actual address the service RPC server will bind to. If this optional address is set, it overrides only the hostname portion of dfs.namenode.servicerpc-address. It can also be specified per name node or name service for HA/Federation. This is useful for making the name node listen on all interfaces by setting it to 0.0.0.0. </description> </property> <property> <name>dfs.namenode.http-bind-host</name> <value>0.0.0.0</value> <description> The actual adress the HTTP server will bind to. If this optional address is set, it overrides only the hostname portion of dfs.namenode.http-address. It can also be specified per name node or name service for HA/Federation. This is useful for making the name node HTTP server listen on all interfaces by setting it to 0.0.0.0. </description> </property> <property> <name>dfs.namenode.https-bind-host</name> <value>0.0.0.0</value> <description> The actual adress the HTTPS server will bind to. If this optional address is set, it overrides only the hostname portion of dfs.namenode.https-address. It can also be specified per name node or name service for HA/Federation. This is useful for making the name node HTTPS server listen on all interfaces by setting it to 0.0.0.0. </description> </property>
Datanode denied communication with namenode
dfs-site.xml - Need to add following line.
<property> <name>dfs.namenode.datanode.registration.ip-hostname-check</name> <value>false</value></property>