基于二进制部署Cloudera Manager集群缩容实战案例（退役）
基于二进制部署Cloudera Manager集群缩容实战案例（退役）
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.查看数据库信息
1>.查看CM数据库中存储的表
[root@node101.yinzhengjie.org.cn ~]# mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 15 Server version: 5.7.25-log MySQL Community Server (GPL) Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | cdh | | mysql | | performance_schema | | sys | +--------------------+ 5 rows in set (0.00 sec) mysql> mysql> use cdh Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> mysql> show tables; +--------------------------------+ | Tables_in_cdh | +--------------------------------+ | AUDITS | | CLIENT_CONFIGS | | CLIENT_CONFIGS_TO_HOSTS | | CLUSTERS | | CLUSTERS_AUD | | CLUSTER_ACTIVATED_RELEASES | | CLUSTER_ACTIVATED_RELEASES_AUD | | CLUSTER_MANAGED_RELEASES | | CLUSTER_UNDISTRIBUTED_RELEASES | | CM_PEERS | | CM_VERSION | | COMMANDS | | COMMANDS_DETAIL | | COMMAND_SCHEDULES | | CONFIGS | | CONFIGS_AUD | | CONFIG_CONTAINERS | | CREDENTIALS | | DIAGNOSTICS_EVENTS | | EXTERNAL_ACCOUNTS | | EXTERNAL_ACCOUNTS_AUD | | GLOBAL_SETTINGS | | HOSTS | | HOSTS_AUD | | HOST_TEMPLATES | | HOST_TEMPLATE_TO_ROLE_CONF_GRP | | METRICS | | PARCELS | | PARCEL_COMPONENTS | | PROCESSES | | PROCESSES_DETAIL | | PROCESS_ACTIVE_RELEASES | | RELEASES | | RELEASES_AUD | | REVISIONS | | ROLES | | ROLES_AUD | | ROLE_CONFIG_GROUPS | | ROLE_CONFIG_GROUPS_AUD | | ROLE_STALENESS_STATUS | | SCHEMA_VERSION | | SERVICES | | SERVICES_AUD | | SNAPSHOT_POLICIES | | USERS | | USER_ROLES | | USER_SETTINGS | +--------------------------------+ 47 rows in set (0.01 sec) mysql>
2>.查看现有节点
mysql> SELECT * FROM HOSTS; +---------+-------------------------+--------------------------------------+----------------------------+--------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+ | HOST_ID | OPTIMISTIC_LOCK_VERSION | HOST_IDENTIFIER | NAME | IP_ADDRESS | RACK_ID | STATUS | CONFIG_CONTAINER_ID | MAINTENANCE_COUNT | DECOMMISSION_COUNT | CLUSTER_ID | NUM_CORES | TOTAL_PHYS_MEM_BYTES | PUBLIC_NAME | PUBLIC_IP_ADDRESS | CLOUD_PROVIDER | +---------+-------------------------+--------------------------------------+----------------------------+--------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+ | 1 | 13 | a047f9e1-50a6-4245-91f5-a02fb728e783 | node102.yinzhengjie.org.cn | 172.30.1.102 | /default | NA | 1 | 0 | 0 | 1 | 4 | 1907970048 | NULL | NULL | NULL | | 2 | 13 | 1f8e3d4c-5609-476c-81d6-93bea59b3e31 | node101.yinzhengjie.org.cn | 172.30.1.101 | /default | NA | 1 | 0 | 0 | 1 | 4 | 8182046720 | NULL | NULL | NULL | | 3 | 14 | c0ad9575-495e-4855-b4c8-a81d9e31c344 | node103.yinzhengjie.org.cn | 172.30.1.103 | /default | NA | 1 | 0 | 0 | 1 | 4 | 3954188288 | NULL | NULL | NULL | | 4 | 6 | 4a1d8c50-d50f-4d42-9309-b84fce3e20b5 | node104.yinzhengjie.org.cn | 172.30.1.104 | /default | NA | 1 | 0 | 0 | 1 | 4 | 1907970048 | NULL | NULL | NULL | +---------+-------------------------+--------------------------------------+----------------------------+--------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+ 4 rows in set (0.00 sec) mysql> mysql>
3>.查看现有角色分配
mysql> SELECT * FROM ROLES; +---------+---------------------------------------------------------+---------+-------------------+-------------------+------------+---------------+-------------------+--------------------+-------------------------+----------------------+------------------+ | ROLE_ID | NAME | HOST_ID | ROLE_TYPE | CONFIGURED_STATUS | SERVICE_ID | MERGED_KEYTAB | MAINTENANCE_COUNT | DECOMMISSION_COUNT | OPTIMISTIC_LOCK_VERSION | ROLE_CONFIG_GROUP_ID | HAS_EVER_STARTED | +---------+---------------------------------------------------------+---------+-------------------+-------------------+------------+---------------+-------------------+--------------------+-------------------------+----------------------+------------------+ | 12 | mgmt-EVENTSERVER-455d0118ea9463330d3da351bdc72e66 | 3 | EVENTSERVER | RUNNING | 3 | NULL | 0 | 0 | 10 | 22 | 1 | | 13 | mgmt-HOSTMONITOR-455d0118ea9463330d3da351bdc72e66 | 3 | HOSTMONITOR | RUNNING | 3 | NULL | 0 | 0 | 10 | 24 | 1 | | 14 | mgmt-SERVICEMONITOR-455d0118ea9463330d3da351bdc72e66 | 3 | SERVICEMONITOR | RUNNING | 3 | NULL | 0 | 0 | 10 | 29 | 1 | | 15 | mgmt-ALERTPUBLISHER-455d0118ea9463330d3da351bdc72e66 | 3 | ALERTPUBLISHER | RUNNING | 3 | NULL | 0 | 0 | 10 | 27 | 1 | | 16 | hdfs-HTTPFS-455d0118ea9463330d3da351bdc72e66 | 3 | HTTPFS | STOPPED | 4 | NULL | 0 | 0 | 4 | 37 | 1 | | 17 | hdfs-SECONDARYNAMENODE-455d0118ea9463330d3da351bdc72e66 | 3 | SECONDARYNAMENODE | STOPPED | 4 | NULL | 0 | 0 | 4 | 31 | 1 | | 18 | hdfs-DATANODE-455d0118ea9463330d3da351bdc72e66 | 3 | DATANODE | STOPPED | 4 | NULL | 0 | 0 | 4 | 39 | 1 | | 19 | hdfs-DATANODE-bd837b270f0f909666aa30e81f437e11 | 2 | DATANODE | RUNNING | 4 | NULL | 0 | 0 | 6 | 33 | 1 | | 20 | hdfs-DATANODE-2f4382454d0c8fb764abbd8574a27746 | 1 | DATANODE | STOPPED | 4 | NULL | 0 | 0 | 4 | 40 | 1 | | 21 | hdfs-NAMENODE-bd837b270f0f909666aa30e81f437e11 | 2 | NAMENODE | RUNNING | 4 | NULL | 0 | 0 | 13 | 30 | 1 | | 22 | hdfs-BALANCER-bd837b270f0f909666aa30e81f437e11 | 2 | BALANCER | NA | 4 | NULL | 0 | 0 | 0 | 34 | 0 | | 28 | yarn-NODEMANAGER-2f4382454d0c8fb764abbd8574a27746 | 1 | NODEMANAGER | RUNNING | 6 | NULL | 0 | 0 | 2 | 47 | 1 | | 29 | yarn-NODEMANAGER-455d0118ea9463330d3da351bdc72e66 | 3 | NODEMANAGER | RUNNING | 6 | NULL | 0 | 0 | 2 | 47 | 1 | | 30 | yarn-NODEMANAGER-bd837b270f0f909666aa30e81f437e11 | 2 | NODEMANAGER | RUNNING | 6 | NULL | 0 | 0 | 2 | 50 | 1 | | 31 | yarn-JOBHISTORY-bd837b270f0f909666aa30e81f437e11 | 2 | JOBHISTORY | RUNNING | 6 | NULL | 0 | 0 | 2 | 49 | 1 | | 32 | yarn-RESOURCEMANAGER-455d0118ea9463330d3da351bdc72e66 | 3 | RESOURCEMANAGER | RUNNING | 6 | NULL | 0 | 0 | 4 | 46 | 1 | | 37 | mapreduce-TASKTRACKER-2f4382454d0c8fb764abbd8574a27746 | 1 | TASKTRACKER | RUNNING | 8 | NULL | 0 | 0 | 2 | 56 | 1 | | 38 | mapreduce-TASKTRACKER-bd837b270f0f909666aa30e81f437e11 | 2 | TASKTRACKER | RUNNING | 8 | NULL | 0 | 0 | 2 | 56 | 1 | | 39 | mapreduce-TASKTRACKER-455d0118ea9463330d3da351bdc72e66 | 3 | TASKTRACKER | RUNNING | 8 | NULL | 0 | 0 | 2 | 56 | 1 | | 40 | mapreduce-JOBTRACKER-bd837b270f0f909666aa30e81f437e11 | 2 | JOBTRACKER | RUNNING | 8 | NULL | 0 | 0 | 2 | 55 | 1 | | 44 | zookeeper-SERVER-bd837b270f0f909666aa30e81f437e11 | 2 | SERVER | RUNNING | 10 | NULL | 0 | 0 | 6 | 60 | 1 | | 45 | zookeeper-SERVER-455d0118ea9463330d3da351bdc72e66 | 3 | SERVER | RUNNING | 10 | NULL | 0 | 0 | 5 | 60 | 1 | | 46 | zookeeper-SERVER-2f4382454d0c8fb764abbd8574a27746 | 1 | SERVER | RUNNING | 10 | NULL | 0 | 0 | 6 | 60 | 1 | | 47 | mapreduce-GATEWAY-5ee75e68465bcfc7fe68accfb667a875 | 4 | GATEWAY | NA | 8 | NULL | 0 | 0 | 0 | 57 | 0 | | 48 | hdfs-GATEWAY-5ee75e68465bcfc7fe68accfb667a875 | 4 | GATEWAY | NA | 4 | NULL | 0 | 0 | 0 | 35 | 0 | | 49 | hdfs-DATANODE-5ee75e68465bcfc7fe68accfb667a875 | 4 | DATANODE | RUNNING | 4 | NULL | 0 | 0 | 2 | 40 | 1 | | 50 | yarn-NODEMANAGER-5ee75e68465bcfc7fe68accfb667a875 | 4 | NODEMANAGER | RUNNING | 6 | NULL | 0 | 0 | 2 | 50 | 1 | | 51 | yarn-GATEWAY-5ee75e68465bcfc7fe68accfb667a875 | 4 | GATEWAY | NA | 6 | NULL | 0 | 0 | 0 | 48 | 0 | +---------+---------------------------------------------------------+---------+-------------------+-------------------+------------+---------------+-------------------+--------------------+-------------------------+----------------------+------------------+ 28 rows in set (0.00 sec) mysql>
4>.通过CM Server的WEB UI查看对应的主机和角色（你会发现一目了然）
二.CM agent节点退役
1>.点击主机
2>.选择需要退役的节点
3>.点击"Begin Maintenance (Suppress Alerts/Decommission)"
4>.点击"Begin Maintenance"(如果你的副本数大于1，推荐选择"Take DataNode offline",这样可以节省我们的维护时间哟~)
5>.等待"主机解除授权 "完成
6>.解除授权的主机进入了"维护状态"
7>.停止该节点的服务
[root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# ./cloudera-scm-agent status cloudera-scm-agent (pid 36848) is running... [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# ps aux | grep 'cloudera-scm-agent.pid' | grep -v grep root 36848 0.7 2.6 1949632 50232 ? Ssl 20:17 0:08 python2.7 /opt/cloudera-manager/cm-5.15.1/lib64/cmf/agent/build/env/bin/cmf-agent --package_dir /opt/cloudera-manager/cm-5.15.1/lib64/cmf/service --agent_dir /opt/cloudera-manager/cm-5.15.1/run/cloudera-scm-agent --lib_dir /opt/cloudera-manager/cm-5.15.1/lib/cloudera-scm-agent --logfile /opt/cloudera-manager/cm-5.15.1/log/cloudera-scm-agent/cloudera-scm-agent.log --daemon --comm_name cmf-agent --pidfile /opt/cloudera-manager/cm-5.15.1/run/cloudera-scm-agent/cloudera-scm-agent.pid [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# ./cloudera-scm-agent status cloudera-scm-agent (pid 36848) is running... [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# ./cloudera-scm-agent stop Stopping cloudera-scm-agent: [ OK ] [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# ./cloudera-scm-agent status cloudera-scm-agent is stopped [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# ps aux | grep 'cloudera-scm-agent.pid' | grep -v grep [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]#
8>.点击"Remove Hosts From Cloudera Manager"
9>.点击确定
10>.节点移除成功
三.再次查看数据库信息
mysql> SELECT * FROM HOSTS; +---------+-------------------------+--------------------------------------+----------------------------+--------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+ | HOST_ID | OPTIMISTIC_LOCK_VERSION | HOST_IDENTIFIER | NAME | IP_ADDRESS | RACK_ID | STATUS | CONFIG_CONTAINER_ID | MAINTENANCE_COUNT | DECOMMISSION_COUNT | CLUSTER_ID | NUM_CORES | TOTAL_PHYS_MEM_BYTES | PUBLIC_NAME | PUBLIC_IP_ADDRESS | CLOUD_PROVIDER | +---------+-------------------------+--------------------------------------+----------------------------+--------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+ | 1 | 13 | a047f9e1-50a6-4245-91f5-a02fb728e783 | node102.yinzhengjie.org.cn | 172.30.1.102 | /default | NA | 1 | 0 | 0 | 1 | 4 | 1907970048 | NULL | NULL | NULL | | 2 | 13 | 1f8e3d4c-5609-476c-81d6-93bea59b3e31 | node101.yinzhengjie.org.cn | 172.30.1.101 | /default | NA | 1 | 0 | 0 | 1 | 4 | 8182046720 | NULL | NULL | NULL | | 3 | 14 | c0ad9575-495e-4855-b4c8-a81d9e31c344 | node103.yinzhengjie.org.cn | 172.30.1.103 | /default | NA | 1 | 0 | 0 | 1 | 4 | 3954188288 | NULL | NULL | NULL | +---------+-------------------------+--------------------------------------+----------------------------+--------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+ 3 rows in set (0.00 sec) mysql> mysql> mysql> mysql> mysql> SELECT * FROM ROLES; +---------+---------------------------------------------------------+---------+-------------------+-------------------+------------+---------------+-------------------+--------------------+-------------------------+----------------------+------------------+ | ROLE_ID | NAME | HOST_ID | ROLE_TYPE | CONFIGURED_STATUS | SERVICE_ID | MERGED_KEYTAB | MAINTENANCE_COUNT | DECOMMISSION_COUNT | OPTIMISTIC_LOCK_VERSION | ROLE_CONFIG_GROUP_ID | HAS_EVER_STARTED | +---------+---------------------------------------------------------+---------+-------------------+-------------------+------------+---------------+-------------------+--------------------+-------------------------+----------------------+------------------+ | 12 | mgmt-EVENTSERVER-455d0118ea9463330d3da351bdc72e66 | 3 | EVENTSERVER | RUNNING | 3 | NULL | 0 | 0 | 10 | 22 | 1 | | 13 | mgmt-HOSTMONITOR-455d0118ea9463330d3da351bdc72e66 | 3 | HOSTMONITOR | RUNNING | 3 | NULL | 0 | 0 | 10 | 24 | 1 | | 14 | mgmt-SERVICEMONITOR-455d0118ea9463330d3da351bdc72e66 | 3 | SERVICEMONITOR | RUNNING | 3 | NULL | 0 | 0 | 10 | 29 | 1 | | 15 | mgmt-ALERTPUBLISHER-455d0118ea9463330d3da351bdc72e66 | 3 | ALERTPUBLISHER | RUNNING | 3 | NULL | 0 | 0 | 10 | 27 | 1 | | 16 | hdfs-HTTPFS-455d0118ea9463330d3da351bdc72e66 | 3 | HTTPFS | STOPPED | 4 | NULL | 0 | 0 | 4 | 37 | 1 | | 17 | hdfs-SECONDARYNAMENODE-455d0118ea9463330d3da351bdc72e66 | 3 | SECONDARYNAMENODE | STOPPED | 4 | NULL | 0 | 0 | 4 | 31 | 1 | | 18 | hdfs-DATANODE-455d0118ea9463330d3da351bdc72e66 | 3 | DATANODE | STOPPED | 4 | NULL | 0 | 0 | 4 | 39 | 1 | | 19 | hdfs-DATANODE-bd837b270f0f909666aa30e81f437e11 | 2 | DATANODE | RUNNING | 4 | NULL | 0 | 0 | 6 | 33 | 1 | | 20 | hdfs-DATANODE-2f4382454d0c8fb764abbd8574a27746 | 1 | DATANODE | STOPPED | 4 | NULL | 0 | 0 | 4 | 40 | 1 | | 21 | hdfs-NAMENODE-bd837b270f0f909666aa30e81f437e11 | 2 | NAMENODE | RUNNING | 4 | NULL | 0 | 0 | 42 | 30 | 1 | | 22 | hdfs-BALANCER-bd837b270f0f909666aa30e81f437e11 | 2 | BALANCER | NA | 4 | NULL | 0 | 0 | 0 | 34 | 0 | | 28 | yarn-NODEMANAGER-2f4382454d0c8fb764abbd8574a27746 | 1 | NODEMANAGER | RUNNING | 6 | NULL | 0 | 0 | 2 | 47 | 1 | | 29 | yarn-NODEMANAGER-455d0118ea9463330d3da351bdc72e66 | 3 | NODEMANAGER | RUNNING | 6 | NULL | 0 | 0 | 2 | 47 | 1 | | 30 | yarn-NODEMANAGER-bd837b270f0f909666aa30e81f437e11 | 2 | NODEMANAGER | RUNNING | 6 | NULL | 0 | 0 | 2 | 50 | 1 | | 31 | yarn-JOBHISTORY-bd837b270f0f909666aa30e81f437e11 | 2 | JOBHISTORY | RUNNING | 6 | NULL | 0 | 0 | 2 | 49 | 1 | | 32 | yarn-RESOURCEMANAGER-455d0118ea9463330d3da351bdc72e66 | 3 | RESOURCEMANAGER | RUNNING | 6 | NULL | 0 | 0 | 32 | 46 | 1 | | 37 | mapreduce-TASKTRACKER-2f4382454d0c8fb764abbd8574a27746 | 1 | TASKTRACKER | RUNNING | 8 | NULL | 0 | 0 | 2 | 56 | 1 | | 38 | mapreduce-TASKTRACKER-bd837b270f0f909666aa30e81f437e11 | 2 | TASKTRACKER | RUNNING | 8 | NULL | 0 | 0 | 2 | 56 | 1 | | 39 | mapreduce-TASKTRACKER-455d0118ea9463330d3da351bdc72e66 | 3 | TASKTRACKER | RUNNING | 8 | NULL | 0 | 0 | 2 | 56 | 1 | | 40 | mapreduce-JOBTRACKER-bd837b270f0f909666aa30e81f437e11 | 2 | JOBTRACKER | RUNNING | 8 | NULL | 0 | 0 | 2 | 55 | 1 | | 44 | zookeeper-SERVER-bd837b270f0f909666aa30e81f437e11 | 2 | SERVER | RUNNING | 10 | NULL | 0 | 0 | 6 | 60 | 1 | | 45 | zookeeper-SERVER-455d0118ea9463330d3da351bdc72e66 | 3 | SERVER | RUNNING | 10 | NULL | 0 | 0 | 5 | 60 | 1 | | 46 | zookeeper-SERVER-2f4382454d0c8fb764abbd8574a27746 | 1 | SERVER | RUNNING | 10 | NULL | 0 | 0 | 6 | 60 | 1 | +---------+---------------------------------------------------------+---------+-------------------+-------------------+------------+---------------+-------------------+--------------------+-------------------------+----------------------+------------------+ 23 rows in set (0.00 sec) mysql>
当你的才华还撑不起你的野心的时候，你就应该静下心来学习。当你的能力还驾驭不了你的目标的时候，你就应该沉下心来历练。问问自己，想要怎样的人生。
欢迎加入基础架构自动化运维：598432640，大数据SRE进阶之路：959042252，DevOps进阶之路：526991186
