hadoop集群搭建

软件安装：Linux（阿里云）安装Hadoop(详细教程+避坑)_linux 下载并配置hadoop环境-CSDN博客

网络配置、软件配置：Hadoop集群安装和搭建（全面超详细的过程）_hadoop集群搭建-CSDN博客

yarn配置：Hadoop3.1.3Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster_hadoop 3.1.3 exception from container launch-CSDN博客

yum换源：https://www.8a.hk/news/content/8645.html

 

Hadoop异常--There are 3 datanode(s) running and 3 node(s) are excluded in this operation.-CSDN博客

 

注意每一次修改配置文件之后一定要注意用户或者用户组有没用系统更新成root，被更新了需要重新chown，否则hadoop可能读取不到配置

 

 

网址：

hadoop:hadoop01:9870

yarn:hadoop01:8088

log:hadoop02:50090

 

hive 启动命令（hive建库一点要让hive自己建立，不要手动建立）

hadoop1:

nohup /opt/hive/bin/hive --service metastore >> /opt/hive/logs/metastore.log 2>&1 &

nohup /opt/hive/bin/hive --service hiveserver2 >> /opt/hive/logs/hiveserver2.log 2>&1 &

 

如果你是复制的想重新部署：

sudo chown -R yangyuqi:yangyuqi /opt

sudo chmod +x /opt/hadoop/bin/hdfs

sudo chmod +x /opt/hadoop/bin/yarn

 sudo chmod +x /opt/hadoop/bin/hadoop

sudo chmod +x /opt/hadoop/sbin/start-dfs.sh

sudo chmod +x /opt/hadoop/sbin/stop-dfs.sh

sudo chmod +x /opt/hadoop/sbin/start-yarn.sh

sudo chmod +x /opt/hadoop/sbin/stop-yarn.sh

sudo chmod +x /opt/hive/bin/hive

 

/opt/hive/bin/hive --service metastore --hiveconf hive.root.logger=TRACE,console