CentOS 7 CDH Hadoop安装过程

1. 安装jdk1.8

   tar -xzvf jdk-8u121-linux-x64.tar.gz

    

   配置jdk环境变量

   export JAVA_HOME=/usr/lib/java/jdk1.8.0_121

   export JRE_HOME=$HOME/jre

   export CLASS_PATH=.:$JAVA_HOME/lib:$JRE_HOME/lib

   export PATH=$JAVA_HOME/bin:$PATH:$HOME/bin

    

   禁用系统自带的OpenJDK

   update-alternatives –config java

2. 配置ssh无密码登录

   去掉/etc/ssh/sshd_config其中2行的注释，每台服务器都要设置，
   #RSAAuthentication yes
   #PubkeyAuthentication yes

    

   ssh-keygen -t rsa

   cat id_rsa.pub>> authorized_keys
   ssh root@192.168.0.183 cat ~/.ssh/id_rsa.pub>> authorized_keys

   scp authorized_keys root@192.168.0.183:/root/.ssh

   scp know_hosts root@192.168.0.183:/root/.ssh

    

3. 关闭防火墙和selinux

   

4. 安装mysql5.6

   groupadd mysql

   useradd -r -g mysql mysql

   //这句话报缺少模块，需要联网执行yum install perl-Module-Install

   ./mysql_install_db --user=mysql --basedir=/usr/local/mysql/mysql5.6 --datadir=/usr/local/mysql/mysql5.6/data

   cp my.cnf /etc/my.cnf

   cp mysql.server /etc/init.d/mysql

   修改cp mysql.server /etc/init.d/mysql中的basedir

   chkconfig --add mysql

   chkconfig mysql on

   service mysql start

5. Python2.6或者python2.7安装
6. 在Mysql中初始化数据库

   --hive数据库,为什么使用latin1，后面安装异常会做说明

   create database hive DEFAULT CHARSET latin1;

   --集群监控数据库

   create database amon DEFAULT CHARSET utf8;

   --hue数据库

   create database hue DEFAULT CHARSET utf8;

   --oozie数据库

   create database oozie default charset utf8;

   --授权

   grant all privileges on hive.* to 'scm'@'%' identified by 'cdh' with grant option;

   grant all privileges on amon.* to 'scm'@'%' identified by 'cdh' with grant option;

   grant all privileges on hue.* to 'scm'@'%' identified by 'cdh' with grant option;

   grant all privileges on oozie.* to 'scm'@'%' identified by 'cdh' with grant option;

   flush privileges;

7. 每个节点上添加用户

   useradd --system --home=/opt/cm-5.8.0/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

8. 部署Mysql Connector

   mv Downloads/mysql-connector-java-5.1.41-bin.jar /opt/cm-5.8.0/share/cmf/lib/

9. 修改cm-5.8.0 的config文件中的host属性
10. 部署至slave中

    scp –r /opt/cm-5.8.0 slave2:/opt/

11. 初始化manager数据库

    /opt/cm-5.8.0/share/cmf/schema/scm_prepare_database.sh mysql -hmaster.cdh -uroot -p1 --scm-host master.cdh scm scm2 scm3

12. CM的主节点配置本地源

    mv CDH-5.8.0-1.cdh5.8.0.p0.42-el7.parcel /opt/cloudera/parcel-repo

    mv CDH-5.8.0-1.cdh5.8.0.p0.42-el7.parcel.sha /opt/cloudera/parcel-repo

    mv manifest.json /opt/cloudera/parcel-repo

13. CM主节点启动server

    /opt/cm-5.8.0/etc/init.d/cloudera-scm-server start

    看启动日志：tail -f /opt/cm-5.8.0/log/cloudera-scm-server/cloudera-scm-server.log

    这时多了一个main的接收进程

14. CM子节点创建agent所需的文件夹 

    mkdir -p /opt/cm-5.8.0/run

    mkdir -p /opt/cm-5.8.0/run/cloudera-scm-agent

15. CM子节点启动agent 

    /opt/cm-5.8.0/etc/init.d/cloudera-scm-agent start

16. 调整交换区大小

    sysctl vm.swappiness=10

17. 调整透明大页面

    echo never > /sys/kernel/mm/transparent_hugepage/defrag

 

 

 

 

 

 

 

 

http://archive.cloudera.com/md5/

http://archive.cloudera.com/cdh5/