在先电IAAS平台中，搭建先电bigdata平台
以两台节点为例来组件Hadoop分布式集群，这里采用的系统版本为Centos7，如下表所示：
|
主机名 |
内存 |
硬盘 |
IP地址 |
角色 |
|
master |
8192MB |
100G |
192.168.200.131 |
Ambari-Server Ambari-Agent |
|
slave1 |
4096MB |
100G |
192.168.200.133 |
Ambari-Agent |
1.1 配置主机名
# master
# hostnamectl set-hostname master
# hostname
master
# slave1
# hostnamectl set-hostname slave1
# hostname
slave1
1.2 修改hosts文件
# master
# vi /etc/hosts
192.168.200.131 master.hadoop master
192.168.200.133 slave1.hadoop
# slave1
# vi /etc/hosts
192.168.200.131 master.hadoop
192.168.200.133 slave1.hadoop slave1
注意：主机名映射采用FQDN格式。
1.3 修改yum源
# master
将XianDian-BigData-v2.2.iso挂在到/mnt目录下，将其中的ambari解压到/opt目录下，并在master节点配置ftp服务。
注意：
因为安装大数据相关软件包时，可能会用到相关依赖软件包，所以需要配置Centos7 Yum源，这里可以采用IAAS中的Centos7 Yum源。
# master & slave1
# cd /etc/yum.repos.d/
# rm -vf *
配置Yum源
# vi ambari.repo
[centos7]
baseurl=ftp://192.168.2.10/centos7/
（ 注：具体的yum源根据真实环境配置，本次为实验室测试环境 )
gpgcheck=0
enabled=1
name=centos
[ambari]
name=ambari
baseurl=ftp://10.0.3.61/ambari
（ 注：具体的yum源根据真实环境配置，本次为实验室测试环境 )
enabled=1
gpgcheck=0
# master
# yum -y install httpd
将/mnt/文件夹中HDP-2.6.1.0和HDP-UTILS-1.1.0.21两个文件夹拷贝到/var/www/html/目录下。启动httpd服务。
# systemctl enable httpd.service
# systemctl status httpd.service
1.4 配置ntp
# master
# yum -y install ntp
# vi /etc/ntp.conf
注释或者删除以下四行
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst
添加以下两行
server 127.127.1.0
fudge 127.127.1.0 stratum 10
#systemctl enable ntpd
#systemctl start ntpd
# slave1
# yum -y install ntpdate
# ntpdate master
# systemctl enable ntpdate
1.5 配置SSH
# master & slave1
检查2个节点是否可以通过无密钥相互访问，如果未配置，则进行SSH无密码公钥认证配置。如下：
# yum install openssh-clients
# ssh-keygen
# ssh-copy-id master.hadoop
# ssh-copy-id slave1.hadoop
ssh登录远程主机查看是否成功
# ssh master.hadoop
# exit
# ssh slave1.hadoop
# exit
1.6 禁用Transparent Huge Pages
操作系统后台有一个叫做khugepaged的进程，它会一直扫描所有进程占用的内存，在可能的情况下会把4kpage交换为Huge Pages，在这个过程中，对于操作的内存的各种分配活动都需要各种内存锁，直接影响程序的内存访问性能，并且，这个过程对于应用是透明的，在应用层面不可控制,对于专门为4k page优化的程序来说，可能会造成随机的性能下降现象。
# master & slave1
# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
# echo never > /sys/kernel/mm/transparent_hugepage/enabled
# echo never > /sys/kernel/mm/transparent_hugepage/defrag
# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
重启后失效，需要再次执行
1.7 安装配置JDK
# master
# mkdir /usr/jdk64/
# mount -o loop XianDian-BigData-v2.2.iso /mnt/
# cd /mnt/
# tar -zxvf jdk-8u77-linux-x64.tar.gz -C /usr/jdk64/
# vi /etc/profile
export JAVA_HOME=/usr/jdk64/jdk1.8.0_77
export PATH=$JAVA_HOME/bin:$PATH
# source /etc/profile
#java –version
java version "1.8.0_77"
Java(TM) SE Runtime Environment (build 1.8.0_77-b03)
Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)
# slave1
#mkdir /usr/jdk64/
#scp 172.24.14.106://mnt/jdk-8u77-linux-x64.tar.gz .
# tar -zxvf jdk-8u77-linux-x64.tar.gz -C /usr/jdk64/
# vi /etc/profile
export JAVA_HOME=/usr/jdk64/jdk1.8.0_77
export PATH=$JAVA_HOME/bin:$PATH
# source /etc/profile
#java –version
java version "1.8.0_77"
Java(TM) SE Runtime Environment (build 1.8.0_77-b03)
Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)
2 配置ambari-server
# master
# yum -y install ambari-server
2.1 安装MariaDB数据库
# master
# yum install mariadb mariadb-server mysql-connector-java
启动服务
# systemctl enable mariadb
# systemctl start mariadb
配置MySQL
#mysql_secure_installation
按enter确认后设置数据库root密码，我们这里设置为“bigdata”
Remove anonymous users? [Y/n] y
Disallow root login remotely? [Y/n] n
Remove test database and access to it? [Y/n] y
Reload privilege tables now? [Y/n] y
创建ambari数据库
# mysql -uroot -pbigdata
MariaDB [(none)]> create database ambari;
MariaDB [(none)]> grant all privileges on ambari.* to 'ambari'@'localhost' identified by 'bigdata';
MariaDB [(none)]> grant all privileges on ambari.* to 'ambari'@'%' identified by 'bigdata';
MariaDB [(none)]> use ambari;
MariaDB [ambari]> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
MariaDB [ambari]> quit
2.2 安装配置ambari-server
# master
vi /etc/profile
export buildNumber=2.6.0.0
# ambari-server setup
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)?
Customize user account for ambari-server daemon [y/n] (n)? n
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
Path to JAVA_HOME: /usr/jdk64/jdk1.8.0_77
Validating JDK on Ambari Server...done.
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
====================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
====================================================================
Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata):
Proceed with configuring remote database connection properties [y/n] (y)?
Ambari Server 'setup' completed successfully.
# ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar
启动ambari-server服务
# ambari-server start
登陆界面http://192.168.200.131:8080/
登录用户名密码为admin：admin
3 配置ambari-agent
# master & slave1
# yum -y install ambari-agent
# vi /etc/ambari-agent/conf/ambari-agent.ini
[server]
hostname= master.hadoop
# ambari-agent restart
# tail -f /var/log/ambari-agent/ambari-agent.log
INFO 2017-01-12 09:44:20,919 Controller.py:265 - Heartbeat response received (id = 1340)
INFO 2017-01-12 09:44:30,820 Heartbeat.py:78 - Building Heartbeat: {responseId = 1340, timestamp = 1484214270820, commandsInProgress = False, componentsMapped = True}
