精品-大数据CDH集群安装部署

一、系统环境

操作系统：CentOS 7.2 x64
Clouder Manager：5.14.4
CDH：5.14.4

二、机器规划

准备4个节点，2个master节点，3个datanode节点, 3个zk节点。
master：8核16G ;10.16.30.212; bigdataop-test-30-212
datanode：8核16G; 10.16.30.212-215; bigdataop-test-30-212/215

三、集群基础准备工作(非常重要*****)

    1、配置hosts
       1）hostname建议小写，如果开启kerberos大写认证报错       2）hosts配置为集群所有主机映射
       [root@localhost ~]# echo "hostname" > /etc/hostname
    2、免秘钥登录（略...）
       在规划的namenode节点中，使用ops账号建立免秘钥，且有权限执行sudo -i切换到root账户
       ssh-keygen             #建立公钥和私钥
       ssh-copy-id + IP         #将公钥copy到集群其他节点 
  
    3、ntp时钟同步（略...）    *** hadoop强依赖

       [root@localhost ~]# cat /etc/ntp.conf
       driftfile /var/lib/ntp/drift
       restrict default kod nomodify notrap nopeer noquery
       restrict -6 default kod nomodify notrap nopeer noquery
       restrict 127.0.0.1 
       restrict -6 ::1
       server 192.168.10.100
       includefile /etc/ntp/crypto/pw
       keys /etc/ntp/keys

       [root@localhost ~]# ntpdate 192.168.10.100
       [root@localhost ~]# systemctl enable ntpd
       [root@localhost ~]# systemctl restart ntpd

    4、安装系统依赖库
       [root@localhost ~]# yum -y install bind-utils libxslt cyrus-sasl-plain cyrus-sasl-gssapi portmap fuse-libs /lib/lsb/init-functions httpd mod_ssl openssl-devel python-psycopg2 MySQL-python fuse
    5、关闭防火墙
       [root@localhost ~]# systemctl stop firewalld
       [root@localhost ~]# systemctl disabled firewalld
    6、关闭selinux（略...）
    7、安装jdk8（略...）
    8、关闭swap
       (1) 第一步 关闭swap分区:
           [root@localhost ~]# swapoff -a
       (2) 第二步修改配置文件 - /etc/fstab
           删除swap相关行 /mnt/swap swap swap defaults 0 0 这一行或者注释掉这一行
       (3) 第三步确认swap已经关闭
           [root@localhost ~]# free -m
       (4) 第四步调整 swappiness 参数
           [root@localhost ~]# echo 0 > /proc/sys/vm/swappiness # 临时生效
           [root@localhost ~]# vim /etc/sysctl.conf # 永久生效
           #修改 vm.swappiness 的修改为 0
           vm.swappiness=0
           [root@localhost ~]# sysctl -p # 使配置生效
    9、透明大叶修改
       [root@localhost ~]# echo 'never' >/sys/kernel/mm/transparent_hugepage/enabled
       [root@localhost ~]# echo 'never' >/sys/kernel/mm/transparent_hugepage/defrag
       [root@localhost ~]# echo "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled" >> /etc/rc.d/rc.local
       [root@localhost ~]# echo "echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag" >> /etc/rc.d/rc.local
    10、磁盘格式化（可选项，根据服务器操作）
       #!/bin/bash
       i=1
       for  disk in `ls -l /dev/sd* |awk '{print substr($10,0,8)}'|grep -v sda|uniq`
       do
                parted ${disk}  << EXIT
                mklabel gpt
                mkpart primary 0 -1
                ignore
                quit
       EXIT
                mkfs.ext4 ${disk}1
       echo "****************${disk}1  Fdisked!Waithing For 10 second***********"
       sleep 1s
       mkdir -p /mnt/hdfs/$i

       echo "${disk}1   /data/${i}      ext4       defaults,noatime       0 0 ">>/etc/fstab
       i=$(($i + 1))
       done
       cat /etc/fstab
       mount -a

四、安装说明

      注意：离线安装，要先下载对应的CM和对应的CDH
      cm和cdh的下载地址：
      http://archive.cloudera.com      ## url后面根据cm/cdh

五、构建本地yum源：
   5.1 安装web服务
     略搭建本地http或nginx服务...
   5.2 下载安装包
     Cloudera Manager安装包地址：
     http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.14.0/RPMS/x86_64/cloudera-manager-agent-5.14.0-1.cm5140.p0.25.el7.x86_64.rpm
     http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.14.0/RPMS/x86_64/cloudera-manager-daemons-5.14.0-1.cm5140.p0.25.el7.x86_64.rpm
     http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.14.0/RPMS/x86_64/cloudera-manager-server-5.14.0-1.cm5140.p0.25.el7.x86_64.rpm
     CDH安装包地址：
     http://archive.cloudera.com/cdh5/parcels/5.14.4/CDH-5.14.4-1.cdh5.14.4.p0.3-el7.parcel
     http://archive.cloudera.com/cdh5/parcels/5.14.4/CDH-5.14.4-1.cdh5.14.4.p0.3-el7.parcel.sha1
     http://archive.cloudera.com/cdh5/parcels/5.14.4/manifest.json
    
   5.3 配置repo
     yum install -y yum-utils createrepo
     mkdir /var/www/html/cdh/
     mv cloudera*  /var/www/html/cm/    &&    mv CDH*  /var/www/html/cdh/        ## 移动rpm文件夹到web server下
     [root@localhost packages]# cd /var/www/html/cdh/5 
     [root@localhost cdh]# createrepo .                  #创建repo
     [root@localhost cm]# createrepo .
     [root@localhost ~]# systemctl start nginx
      server {
            listen          80;
            server_name     localhost;
            access_log      /var/log/nginx_access.log main;
            error_log       /var/log/nginx_error.log warn;
            root	/var/www/html;
            location / {
            autoindex on;
            }
      }

   5.4 所有节点添加yum源的配置  ***
     [root@localhost ~]# vim /etc/yum.repos.d/cm.repo
     [cloudera-manager]
     name = Cloudera Manager, Version 5.14.0
     baseurl = http://10.193.xx.xx/cdh/
     gpgcheck = 0
   5.5 验证源：yum list|grep cloudera
     如果验证失败更新一下yum源
     1）删除缓存rm -rf /var/cache/yum
     2) yum makecache
     3) yum clean all

六、集群安装部署

   6.1 管理节点上安装cloudera-manager-server、cloudera-manager-daemons、cloudera-manager-agent
     [root@localhost ~]# yum -y install cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent
     （1）安装完后再/opt/cloudera/parcel-repo目录下下载如下三个文件并授权cloudera-scm属主属组：
     [root@localhost ~]# cd /opt/cloudera/parcel-repo
     CDH-5.14.4-1.cdh5.14.4.p0.3-el7.parcel
     CDH-5.14.4-1.cdh5.14.4.p0.3-el7.parcel.sha
     manifest.json
     （2）配置server
    　[root@localhost ~]# vim /etc/cloudera-scm-server/db.properties
      com.cloudera.cmf.db.type=mysql
      com.cloudera.cmf.db.host=xx.xx.xx.xx #数据库ＩＰ地址
      com.cloudera.cmf.db.name=scm
      com.cloudera.cmf.db.user=scm
      com.cloudera.cmf.db.password=123456
      com.cloudera.cmf.db.setupType=EXTERNAL

     （3）启动server
          systemctl start cloudera-scm-server
   6.2 安装MySQL服务（用来保存集群信息很重要，后期也会保存hive,hue等等）
     安装部署略...
     创建目录 mkdir /usr/share/java/
     下载mysql驱动器并放入到/usr/share/java/mysql-connector-java.jar
   6.3 Agent节点上安装cloudera-manager-daemons、cloudera-manager-agent
     [root@localhost ~]# yum -y install cloudera-manager-daemons cloudera-manager-agent
     （1）配置agent
          vim /etc/cloudera-scm-agent/config.ini
          server_host=10.193.xx.xx    ## 指向管理节点
     （2）启动agent
          systemctl start cloudera-scm-agent

   6.4 访问验证
       管理节点ip:7108