hadoop环境搭建
http://wgkgood.blog.51cto.com/1192594/1332340
仿照如上文章进行了环境搭建
内容大致总结为
1. 配置集群内ssh免密码互相登录。请参照http://www.cnblogs.com/vigarbuaa/p/3579291.html
2. 安装hadoop. 配置如下文件
core-site.xml/mapred-site.xml/hdfs-site.xml/hadoop-env.sh/masters/slaves
3. 格式化namenode
root@kali:/data/hadoop/bin# hadoop namenode -format DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs command for it. 14/03/03 21:56:52 INFO namenode.NameNode: STARTUP_MSG: /************************************************************
4. 启动服务
root@kali:/data/hadoop/sbin# ./start-all.sh This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh Starting namenodes on [192.168.0.100] 192.168.0.100: ssh: connect to host 192.168.0.100 port 22: Connection refused The authenticity of host '192.168.0.101 (192.168.0.101)' can't be established. ECDSA key fingerprint is 64:9a:7d:2c:61:40:dc:66:8b:00:12:0e:98:67:2d:22. Are you sure you want to continue connecting (yes/no)? yes 192.168.0.101: Warning: Permanently added '192.168.0.101' (ECDSA) to the list of known hosts. 192.168.0.101: starting datanode, logging to /data/hadoop/logs/hadoop-root-datanode-kali.out Starting secondary namenodes [0.0.0.0] The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established. ECDSA key fingerprint is 64:9a:7d:2c:61:40:dc:66:8b:00:12:0e:98:67:2d:22. Are you sure you want to continue connecting (yes/no)? yes 0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts. 0.0.0.0: starting secondarynamenode, logging to /data/hadoop/logs/hadoop-root-secondarynamenode-kali.out starting yarn daemons resourcemanager running as process 5507. Stop it first. 192.168.0.101: starting nodemanager, logging to /data/hadoop/logs/yarn-root-nodemanager-kali.out
5. 验证服务是否启动
root@kali:/data/hadoop/sbin# jps -l 5507 org.apache.hadoop.yarn.server.resourcemanager.ResourceManager 5874 org.apache.hadoop.hdfs.server.datanode.DataNode 6196 org.apache.hadoop.yarn.server.nodemanager.NodeManager 6291 sun.tools.jps.Jps 6005 org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode
