hadoop集群搭建

安装

masters：

1台 NameNode

1台 ResourceManager

workers：

剩下的机器既作为 DataNode 也作为 NodeManager

以 Non-Secure Mode 模式安装

需要为 hadoop daemon 配置运维环境和运行参数。

HDFS daemons are NameNode, SecondaryNameNode, and DataNode. YARN daemons are ResourceManager, NodeManager, and WebAppProxy. If MapReduce is to be used, then the MapReduce Job History Server will also be running. For large installations, these are generally running on separate hosts.

配置运行环境

需要配置etc/hadoop/hadoop-env.sh、etc/hadoop/mapred-env.sh、etc/hadoop/yarn-env.sh，还要配置JAVA_HOME

mon	Environment Variable
NameNode	HDFS_NAMENODE_OPTS
DataNode	HDFS_DATANODE_OPTS
Secondary NameNode	HDFS_SECONDARYNAMENODE_OPTS
ResourceManager	YARN_RESOURCEMANAGER_OPTS
NodeManager	YARN_NODEMANAGER_OPTS
WebAppProxy	YARN_PROXYSERVER_OPTS
Map Reduce Job History Server	MAPRED_HISTORYSERVER_OPTS

 

 

 

 

 

 

 

配置运行参数