大数据平台CDH搭建笔记
0.主机参数
| 主机名 | IP | 资源(CPU/内存/硬盘/服务器厂家) |
|---|---|---|
| master | 121.89.199.2 | 08 Core/32GB/100GB/阿里云 |
| worker001 | 124.222.153.245 | 04 Core/8GB/100GB/腾讯云 |
| worker002 | 106.52.58.81 | 04 Core/4GB/100GB/腾讯云 |
| worker003 | 124.222.152.251 | 04 Core/4GB/100GB/腾讯云 |
成功版：血的教训。一定要用内网！！！！！！！！
| 主机名 | IP | 资源(CPU/内存/硬盘/服务器厂家) |
|---|---|---|
| master | 172.19.86.200 | 08 Core/32GB/100GB/阿里云 |
| worker001 | 172.19.86.201 | 02 Core/8GB/50GB/阿里云 |
| worker002 | 172.19.86.202 | 02 Core/4GB/50GB/阿里云 |
1.所需软件
2.环境初始化
2.1 修改主机hostname(所有机器)
hostnamectl set-hostname master
hostnamectl set-hostname worker01
hostnamectl set-hostname worker02
hostnamectl set-hostname worker03
以下添加到 /etc/hosts：
121.89.199.2 master
124.222.153.245 worker01
106.52.58.81 worker02
124.222.152.251 worker03
私网版(最好是加上别名)：
172.19.86.200 master master
172.19.86.201 worker01 worer01
172.19.86.202 worker02 worer02
2.2 关闭集群防火墙(所有机器)
systemctl stop firewalld && systemctl disable firewalld && iptables -F && systemctl enable firewalld
2.3 关闭集群selinux(所有机器)
vi /etc/selinux/config
SELINUX=disabled
## 需要重启才能生效
2.4 设置集群时钟同步(所有机器)(待完善)
timedatectl set-timezone Asia/Shanghai
停止chrony并关闭chrony自启动
chkconfig chronyd off
systemctl stop chronyd
安装
ntpdate/autogen-libopts/ntp
yum install -y ntpdate
yum install -y autogen-libopts
yum install -y ntp
上面依赖若有冲突则删除非isorepo源的依赖
yum remove ntpdate-4.2.6p5-29.el7.x86_64 -y
主节点配置
vim /etc/ntp.conf
restrict 124.222.153.245 mask 255.255.255.0 nomodify notrap
2.5 设置免密(所有机器)
1) 生成密钥
ssh localhost
cd ~/.ssh
ssh-keygen -t rsa ##生成 id_rsa和id_rsa.pub
cat id_rsa.pub >> authorized_keys
---
快捷命令
ssh-copy-id user@server
ssh-copy-id -i ~/.ssh/id_rsa.pub user@server
---
2）将集群密钥放入同一个authorized_keys并同步
## 然后将所有机器id_rsa.pub都复制到authorized_keys，分发到每台机器上
cp id_rsa.pub id_rsa.pub.worker02
scp id_rsa.pub.worker02 root@192.168.0.104:/home/
## master节点上
cat /home/id_rsa.pub.worker02 >> ~/.ssh/authorized_keys
3）修改ssh配置
# 如果报错can't be established
vim /etc/ssh/ssh_config
# 添加
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
2.6 部署JDK(所有机器)
mkdir /usr/java
tar -zxf /opt/cdh/jdk-8u201-linux-x64.tar.gz -C /usr/java/
chown -R root:root /usr/java/jdk1.8.0_201
echo "export JAVA_HOME=/usr/java/jdk1.8.0_201" >> /etc/profile
echo "export PATH=/usr/java/jdk1.8.0_201/bin:${PATH}" >> /etc/profile
source /etc/profile
which java
java -version
2.7 部署MYSQL(主机)
1) 解压
tar xzvf /opt/cdh/mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz -C /usr/local
cd /usr/local
mv mysql-5.7.11-linux-glibc2.5-x86_64 mysql
mkdir mysql/arch mysql/data mysql/tmp
2) 修改mysql配置文件
rm -rf /etc/my.cnf
vim /etc/my.cnf
# 删除后添加(注意不要被#注释掉了)：
[client]
port = 3306
socket = /usr/local/mysql/data/mysql.sock
default-character-set=utf8mb4
[mysqld]
port = 3306
socket = /usr/local/mysql/data/mysql.sock
skip-slave-start
skip-external-locking
key_buffer_size = 256M
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 4M
query_cache_size= 32M
max_allowed_packet = 16M
myisam_sort_buffer_size=128M
tmp_table_size=32M
table_open_cache = 512
thread_cache_size = 8
wait_timeout = 86400
interactive_timeout = 86400
max_connections = 600
# Try number of CPU's*2 for thread_concurrency
#thread_concurrency = 32
#isolation level and default engine
default-storage-engine = INNODB
transaction-isolation = READ-COMMITTED
server-id = 1739
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
pid-file = /usr/local/mysql/data/hostname.pid
#open performance schema
log-warnings
sysdate-is-now
binlog_format = ROW
log_bin_trust_function_creators=1
log-error = /usr/local/mysql/data/hostname.err
log-bin = /usr/local/mysql/arch/mysql-bin
expire_logs_days = 7
innodb_write_io_threads=16
relay-log = /usr/local/mysql/relay_log/relay-log
relay-log-index = /usr/local/mysql/relay_log/relay-log.index
relay_log_info_file= /usr/local/mysql/relay_log/relay-log.info
log_slave_updates=1
gtid_mode=OFF
enforce_gtid_consistency=OFF
# slave
slave-parallel-type=LOGICAL_CLOCK
slave-parallel-workers=4
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log_recovery=ON
#other logs
#general_log =1
#general_log_file = /usr/local/mysql/data/general_log.err
#slow_query_log=1
#slow_query_log_file=/usr/local/mysql/data/slow_log.err
#for replication slave
sync_binlog = 500
#for innodb options
innodb_data_home_dir = /usr/local/mysql/data/
innodb_data_file_path = ibdata1:1G;ibdata2:1G:autoextend
innodb_log_group_home_dir = /usr/local/mysql/arch
innodb_log_files_in_group = 4
innodb_log_file_size = 1G
innodb_log_buffer_size = 200M
#根据生产需要，调整pool size
innodb_buffer_pool_size = 2G
#innodb_additional_mem_pool_size = 50M #deprecated in 5.6
tmpdir = /usr/local/mysql/tmp
innodb_lock_wait_timeout = 1000
#innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit = 2
innodb_locks_unsafe_for_binlog=1
#innodb io features: add for mysql5.5.8
performance_schema
innodb_read_io_threads=4
innodb-write-io-threads=4
innodb-io-capacity=200
#purge threads change default(0) to 1 for purge
innodb_purge_threads=1
innodb_use_native_aio=on
#case-sensitive file names and separate tablespace
innodb_file_per_table = 1
lower_case_table_names=1
[mysqldump]
quick
max_allowed_packet = 128M
[mysql]
no-auto-rehash
default-character-set=utf8mb4
[mysqlhotcopy]
interactive-timeout
[myisamchk]
key_buffer_size = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M
3) 创建用户组及用户
groupadd -g 101 dba
useradd -u 514 -g dba -G root -d /usr/local/mysql mysqladmin
cp /etc/skel/.* /usr/local/mysql
cd /usr/local/mysql/
#添加
vim .bashrc
export MYSQL_BASE=/usr/local/mysql
export PATH=${MYSQL_BASE}/bin:$PATH
chown mysqladmin:dba /etc/my.cnf
chmod 640 /etc/my.cnf
chown -R mysqladmin:dba /usr/local/mysql
chmod -R 755 /usr/local/mysql
cd /usr/local/mysql
cp support-files/mysql.server /etc/rc.d/init.d/mysql
chmod +x /etc/rc.d/init.d/mysql
chkconfig --add mysql
#数据库初始化
## 如果报错：bin/mysqld: error while loading shared libraries: libaio.so.1: cannot open shared object file: No ## such file or directory
## 就yum install -y libaio
#数据库初始化
bin/mysqld \
--defaults-file=/etc/my.cnf \
--user=mysqladmin \
--basedir=/usr/local/mysql/ \
--datadir=/usr/local/mysql/data/ \
--initialize
cd data
cat hostname.err |grep password
su mysqladmin
service mysql start
mysql -uroot -p'初始密码'
alter user root@localhost identified by '123456';
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' ;
flush privileges;
exit;
service mysql restart
mysql -uroot -p123456
create database cmf default character set utf8;
create database amon default character set utf8;
create database hive default character set utf8;
create database rman default character set utf8;
create database nav default character set utf8;
create database navms default character set utf8;
create database hue default character set utf8;
create database scm default character set utf8;
create database sentry default character set utf8;
create database dolphinschedule default character set utf8;
grant all privileges on cmf.* to 'cmf'@'%' identified by '123456';
grant all privileges on amon.* to 'amon'@'%' identified by '123456';
grant all privileges on hive.* to 'hive'@'%' identified by '123456';
grant all privileges on rman.* to 'rman'@'%' identified by '123456';
grant all privileges on nav.* to 'nav'@'%' identified by '123456';
grant all privileges on navms.* to 'navms'@'%' identified by '123456';
grant all privileges on hue.* to 'hue'@'%' identified by '123456';
grant all privileges on scm.* to 'scm'@'%' identified by '123456';
grant all privileges on sentry.* to 'sentry'@'%' identified by '123456';
grant all privileges on dolphinschedule.* to 'dolphinschedule'@'%' identified by '123456';
flush privileges;
exit;
exit;
2.8 部署JDBC(主机)
mkdir -p /usr/share/java
cp /opt/cdh/mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar
2.9 部署httpd(主机)
yum install -y httpd
mkdir -p /var/www/html/cdh6_parcel
cd /opt/cdh/parcels
cp CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel /var/www/html/cdh6_parcel/
cp CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha /var/www/html/cdh6_parcel/
cp manifest.json /var/www/html/cdh6_parcel/
systemctl start httpd
# 访问http://121.89.199.2/cdh6_parcel/
# 访问http://124.222.153.245/cdh6_parcel/
2.10 部署yum源(主机)
# 将/opt/cdh/cloudera-repos Cloudera Manager文件拷贝到安装主机中的httpd访问目录中,即/var/www/html中
cp -r /opt/cdh/cloudera-repos/ /var/www/html/
# 将RPMS下的x64_84复制到/var/www/html/cloudera-repos/
cd /opt/cdh
tar -xzvf cm6.3.1-redhat7.tar.gz
cp -r /opt/cdh/cm6.3.1/RPMS/x86_64/ /var/www/html/cloudera-repos/cm6/6.3.1/redhat7/yum/RPMS
sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cm6
(
cat <<EOF
[cloudera-manager]
name=Cloudera Manager 6.3.1
baseurl=http://121.89.199.2/cloudera-repos/cm6/6.3.1/redhat7/yum
gpgcheck=0
enabled=1
EOF
) >> /etc/yum.repos.d/cloudera-manager.repo
2.11 部署CDH(所有机器)
cd /opt/cdh/cm6.3.1/RPMS/x86_64
# 仅主节点
rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
vim /etc/cloudera-scm-server/db.properties
#修改为：
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=master
com.cloudera.cmf.db.name=cmf
com.cloudera.cmf.db.user=cmf
com.cloudera.cmf.db.password=123456
com.cloudera.cmf.db.setupType=EXTERNAL
#所有节点(不用手动安装，到页面自动安装)
rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
sed -i "s/server_host=localhost/server_host=master/g" /etc/cloudera-scm-agent/config.ini
2.12 初始化cmf库(仅主节点)
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql cmf cmf 123456
# /opt/cloudera/cm/schema/scm_prepare_database.sh -h localhost mysql cmf cmf 123456
2.13 启动CDH(所有机器)
# 仅主节点
systemctl start cloudera-scm-server
systemctl status cloudera-scm-server
tail -F /var/log/cloudera-scm-server/cloudera-scm-server.log
netstat -at | grep 7180
# 所有节点
systemctl start cloudera-scm-agent
systemctl status cloudera-scm-agent
netstat -at | grep 7182
2.14 网页初始化
3. TIPS
3.1 can't be established.
https://www.cnblogs.com/peng-lan/p/10528435.html
3.2 无法安装agent
不要自己手动安装
https://blog.csdn.net/jiayawei90/article/details/104432180
如果已经安装，rpm -aq先查询，再rpm -e再删除
3.3 hash验证失败
https://blog.csdn.net/weixin_40762393/article/details/106761939
3.4 主机受管
http://121.89.199.2:7180/cmf/hardware/hosts
3.5 主机运行不良
https://www.freesion.com/article/9082551477/
3.6 配置时测试数据库连接失败
配置root启动mysql
https://jingyan.baidu.com/article/7c6fb428f5d58f80642c90b7.html
3.7 主机运行状态不良
https://www.cnblogs.com/cxc1357/p/13624362.html
3.8 找不到host-monitor
1）先安装一台host，再安装后两台agent
2）agent可以到页面安装，手动安装可能无法检测到anget信号
3）etc/hosts 最好加上别名。不然也可能可能无法检测到anget信号
4) 公网弄了很久，sever和agent始终失联，私网可以解决
3.9 服务器重启过后
1）启动mysql
su - mysqladmin
service mysql start
2）启动httpd
systemctl start httpd
3）启动CDH server
systemctl start cloudera-scm-server
4.0配置IP时尽量是用内网
搜索主机时，使用内网IP，外网如果不配置别名，会无法检测agent信号
配置parcels，使用内网IP，下载会快很多
浙公网安备 33010602011771号