linux mint安装hadoop

一、安装

安装ssh

openssh-server

 

配置jdk环境变量~/.bashrc参考

export JAVA_HOME=/opt/jdk1.7.0_55/

export JRE_HOME=${JAVA_HOME}/jre

export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib

export PATH=${JAVA_HOME}/bin:$PATH

 

export HADOOP_HOME=/opt/Servers/hadoop-2.7.7

export PATH=${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:$PATH

 

/etc/profile 可不设置

#Hadoop

export JAVA_HOME=/opt/jdk1.7.0_55

export HADOOP_HOME=/usr/local/SoftWare/hadoop-2.5.2

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

export HADOOP_OPTS=”-Djava.library.path=$HADOOP_HOME/lib”

export PATH=$PATH:$JAVA_HOME/bin:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin

#hive

export HIVE_HOME=/usr/local/SoftWare/hive

export PATH=$PATH:$HIVE_HOME/bin

#scala

export SCALA_HOME=/usr/local/SoftWare/scala-2.11.8

export PATH=$PATH:$SCALA_HOME/bin

 

http://hadoop.apache.org/releases.html

2.7.7 版本 binary

1、安装Hadoop

tar -zxvf hadoop-2.7.2.tar.gz -C /opt/Servers/

2、查看Hadoop是32 or 64 位

cd /opt//Servers/hadoop-2.7.2/lib/native

file libhadoop.so.1.0.0

3、配置/etc/hosts

sudo gedit /etc/hosts

hosts新增

127.0.0.1 localhost localhost.mainlocal

127.0.0.1 localhost

127.0.0.1 wangshida

127.0.0.1 hadoop

4、修改hadoop2.7.2/etc/hadoop/hadoop-env.sh指定JAVA_HOME

export hs=/opt/jdk/jdk1.8.0_161

 

5、修改hadoop2.7.2/etc/hadoop/core-site.xml 如下：

fs.defaultFS

hdfs://hadoop:9000

 

hadoop.tmp.dir

/opt/Servers/Cache/hadoop

 

6、修改hadoop2.7.2/etc/hadoop/hdfs-site.xml 如下：

dfs.replication 1

 

 

 

7、配置ssh免密登录

ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

chmod 0600 ~/.ssh/authorized_keys

 

8、创建目录用作缓存存放，并设置读写权限，创建以下目录并授权

mkdir /opt/Servers/Cache

chmod 777 /opt/Servers/Cache/ -R

 

9、第一次启动先格式化

格式化hdfs文件

hadoop namenode -format 会显示SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1

cd /opt/Servers/hadoop-2.7.7 或

./bin/hdfs namenode –format

 

10、启动hdfs

/opt/Servers/hadoop-2.7.7/bin/start-all.sh