[原创]CentOS 6 Ambari Hadoop平台的安装(本地库)

1. 安装前的准备

1.1 硬件要求

无特殊需求(根据Hadoop各个组件需求及实际情况,一般建议单个集群的机器数量在3台及以上:1台Master结点,2台机器作为Slave结点)。

1.2 软件要求

1.2.1 操作系统

仅支持如下操作系统:

  •   Red Hat Enterprise Linux (RHEL) v5.x or 6.x (64-bit)
  •   CentOS v5.x or 6.x (64-bit)
  •   SUSE Linux Enterprise Server (SLES) 11, SP1 (64-bit)

1.2.2 浏览器

Ambari Web接口运行的浏览器需求如下:

  Windows (Vista, 7)

  •    Internet Explorer 9.0 and higher
  •    Firefox latest stable release
  •    Safari latest stable release
  •    Google Chrome latest stable release

  Mac OS X (10.6 or later)

  • Firefox latest stable release
  • Safari latest stable release
  • Google Chrome latest stable release

  Linux (RHEL, CentOS, SLES)

  • Firefox latest stable release
  • Google Chrome latest stable release

1.2.3 数据库

Hadoop的一些组件需要用到传统关系型数据库。如MySQL,Derby,PostgreSQL。

1.3 安装环境准备

1.3.1 地址映射

配置详情

# vi /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.31.3    <主机的全名,如john.smith>

 

1.3.2 SSH无密码登录

确保ssh版本最新,sshd服务开启。

配置详情

1)        确认是否已经安装ssh

# rpm -qa | grep ssh

# yum install ssh //上条命令无输出时

2)        修改配置文件

# vi /etc/ssh/sshd_config

RSAAuthentication yes         # 启用 RSA 认证

PubkeyAuthentication yes       # 启用公钥私钥配对认证方式

AuthorizedKeysFile .ssh/authorized_keys      # 公钥文件路径

# service sshd restart //重启sshd服务

3)        生成秘钥

$ ssh-keygen -t rsa

Tips:敲完命令,一直回车,直到命令结束生成如下类似的图案

 

 

图1-1 生成秘钥

 

 

图1-2 生成的秘钥

4)        添加公钥

$ cat .ssh/id_rsa >> .ssh/authorized_keys

Tips.ssh 700, authorized_keys 644

 

 

图1-3 配置成功

1.3.3 关闭SELinux

命令:临时关闭

# setenforce 0

命令:永久关闭

#vi /etc/selinux/config

config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing - SELinux security policy is enforced.
#       permissive - SELinux prints warnings instead of enforcing.
#       disabled - SELinux is fully disabled.
SELINUX=disabled 
# SELINUXTYPE= type of policy in use. Possible values are:
#       targeted - Only targeted network daemons are protected.
#       strict - Full SELinux protection.
SELINUXTYPE=targeted

1.3.4 关闭iptables

命令

#chkconfig iptables off

#/etc/init.d/iptables stop

1.3.5 关闭Packagekit

命令

#vi /etc/yum/pluginconf.d/refresh-packagekit.conf

enabled=0

1.3.6 NTP

针对集群,需要一台时间同步服务器,确保集群时间同步,若差距太大,很多组件无法正常运行。

命令

# rpm -qa | grep ntp //查看是否已经安装NTP

# yum install ntp //安装NTP

# vi /etc/ntp.conf

// restrict default kod nomodify notrap nopeer noquery 改为

restrict default nomodify

// 添加/修改上一层时间同步服务器

server 0.cn.pool.ntp.org

server 0.asia.pool.ntp.org

server 2.asia.pool.ntp.org

# chkconfig ntpd on

# service ntpd start

1.3.7 卸载OpenJDK安装Oracle JDK

命令

#rpm -qa | grep java

#rpm -e --nodeps xxx

#rpm -ivh xxx.rpm

#vi /etc/profiles

//配置 JAVA_HOME

export JAVA_HOME=/xxx

export PATH=$JAVA_HOME/bin:$PATH

export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

#source /etc/profiles

1.3.8 安装、配置PostgreSQL或MySQL

PostgreSQL安装命令

# wget https://download.postgresql.org/pub/repos/yum/9.2/redhat/rhel-6-x86_64/pgdg-centos92-9.2-8.noarch.rpm

# rpm –ivh xxxx.rpm

# vi /etc/yum.repos.d/ CentOS-Base.repo

[base]

exclude=postgresql*

[updates]

exclude=postgresql*

# yum upgrade postgresql-libs 

# yum install postgresql92-server

# service postgresql-9.2 initdb

# service postgresql-9.2 start

# psql -U postgres

//修改配置文件/var/lib/pgsql/9.1/data/pg_hba.conf 将peer改为trust,将METHOD由默认的ident改为md5,再重启服务

MySQL安装、部署命令

# rpm -qa | grep mysql

# rpm -e --nodeps mysql  //卸载原有的MySQL

# yum install -y mysql-server mysql mysql-devel

# service mysqld start

# chkconfig mysqld on  //开机启动mysqld服务

# mysqladmin -u root password 'root' //初始化数据库root账户

1.3.9 安装MySQL依赖

命令

# yum install mysql-connector-java

# chmod 644 /usr/share/java/<mysql-connector-java.jar>

1.3.10 UMASK

命令

#echo umask 0022 >> /etc/profile

1.3.11 搭建本地库

(1)      基本依赖组件安装

命令

# yum install createrepo

# yum install openssl

# yum install nc

# yum install redhat-lsb

# yum install gcc

# yum install python-devel

# yum install python-libs

 

(2)      下载相关文件

ITEMS

URLS

Ambari 2.2.2

http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.2.2.0/

ambari-2.2.2.0-centos6.tar.gz

HDP 2.4

http://public-repo-1.hortonworks.com/HDP/centos6/2.x/

updates/2.4.2.0/HDP-2.4.2.0-centos6-rpm.tar.gz

HDP-UTILS

http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/

repos/centos6/HDP-UTILS-1.1.0.20-centos6.tar.gz

(3)      安装Apache httpd

命令

# yum install httpd

# chkconfig --levels 235 httpd on

# /etc/init.d/httpd start

# mkdir -p /var/www/html/ambari

# mkdir -p /var/www/html/hdp

# mkdir -p /var/www/html/hdp-utils

//打开浏览器,键入http://localhost会出现如下类似的页面,表面安装并开启成功。

 

 

(4)      解压、配置repo文件

命令

# tar -zxvf ambari-2.2.2.0-centos6.tar.gz -C  /var/www/html/ambari

# tar -zxvf HDP-2.4.2.0-centos6-rpm.tar.gz -C  /var/www/html/hdp

# tar -zxvf HDP-UTILS-1.1.0.20-centos6.tar.gz -C  /var/www/html/hdp-utils

 

# vi ambari.repo

[Updates-ambari-2.2.2.0]

name=ambari-2.2.2.0-Updates

baseurl=http://192.168.31.3/ambari/AMBARI-2.2.2.0/centos6/2.2.2.0-460/

gpgcheck=1

gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

 

# vi hdp.repo

[HDP-2.4.2.0]

name=HDPVersion-HDP-2.4.2.0

baseurl=http://192.168.31.3/hdp/HDP/centos6/2.x/updates/2.4.2.0/

gpgcheck=1

gpgkey=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.4.2.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

 

# vi hdp-utils.repo

[HDP-UTILS-1.1.0.20]

name=HDPUtilsVersion-HDP-UTILS-1.1.0.20

baseurl=http://192.168.31.3/hdp-utils/HDP-UTILS-1.1.0.20/repos/centos6/

gpgcheck=1

gpgkey=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.4.2.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

 

# cp ambari.repo /etc/yum.repos.d/

# cp hdp.repo /etc/yum.repos.d/

# cp hdp-utils.repo /etc/yum.repos.d/

 

//打开编辑的链接,可以看到解压出的文件已经挂在httpd上了。

 

 

 

 

 

 

 

 

2. Ambari组件安装

2.1 Ambari server

命令

# yum install ambari-server

# ambari-server start //启动服务

# ambari-server status //状态查看

 

//初始化数据库(以MySQL为例)

# mysql -u root -p

mysql> CREATE USER '<AMBARIUSER>'@'%' IDENTIFIED BY '<AMBARIPASSWORD>';

mysql> GRANT ALL PRIVILEGES ON *.* TO '<AMBARIUSER>'@'%';

mysql> CREATE USER '<AMBARIUSER>'@'localhost' IDENTIFIED BY '<AMBARIPASSWORD>';

mysql> GRANT ALL PRIVILEGES ON *.* TO '<AMBARIUSER>'@'localhost';

mysql> CREATE USER '<AMBARIUSER>'@'<AMBARISERVERFQDN>' IDENTIFIED BY '<AMBARIPASSWORD>';

mysql> GRANT ALL PRIVILEGES ON *.* TO '<AMBARIUSER>'@'<AMBARISERVERFQDN>';

mysql> FLUSH PRIVILEGES;

 

# ambari-server setup //根据提示进行

//若以上安装过程未能成功初始化ambari数据库,则切换用户,手动导入 Ambari的元数据

#mysql -u <AMBARIUSER> -p

mysql> CREATE DATABASE <AMBARIDATABASE>;

mysql> USE <AMBARIDATABASE>;

mysql> SOURCE Ambari-DDL-MySQL-CREATE.sql;

 

注:<AMBARIUSER><AMBARIPASSWORD><AMBARISERVERFQDN>等在实际安装的过程中应根据具体的情况替换。

2.2 Ambari Agent

Ambari agent 按安装完Server之后,可以通过其WebUI进行安装,但在WebUI上安装成功率不高,可以通过手动安装。

 

命令

# yum install ambari-agent -y

# ambari-agent start

# ambari-agent status //状态查看

3. 大数据组件安装

(1) 确保ambari-server 启动,默认通过http://localhost:8080 登录Ambari WebUI,默认的登录密码是admin/admin。界面如下图所示:

 

 

(2) 登录成功,准备安装集群。

 

 【声明】转载请注明出处

posted @ 2017-05-24 14:42 A.Hawkin 阅读(...) 评论(...) 编辑 收藏