dolphinscheduler部署踩坑记录
1. 背景
本地的调度方式一直采用crontab,现在调度任务量很大,crontab给运维带来很大麻烦,而且很多老任务可以删掉,趁此机会,使用新的开源调度工具。经过一番调研,dolphinscheduler比较适合。
2. 部署过程
# dolphinscheduler版本 2.0.5
# 主机及相关组件准备
mysql8 ip 账号
zookeeper ips
dolphinscheduler集群(jdk 1.8.0_102)
ips
# 安装进程树
sudo yum -y install psmisc
# 添加用户dolphinscheduler
useradd dolphinscheduler
echo "dolphinscheduler" | passwd --stdin dolphinscheduler
sed -i '$adolphinscheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL' /etc/sudoers
sed -i 's/Defaults requirett/#Defaults requirett/g' /etc/sudoers
# 配置ssh免密登录
su - dolphinscheduler
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
ssh-copy-id dolphinscheduler@ips
# 上传mysql jdbc jar
mv mysql-connector-java-8.0.29.jar ./lib
# 创建用户
CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE USER 'dolphinscheduler'@'%' IDENTIFIED BY '123';
GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'dolphinscheduler'@'%';
CREATE USER 'dolphinscheduler'@'localhost' IDENTIFIED BY '123';
GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'dolphinscheduler'@'localhost';
FLUSH PRIVILEGES;
# 初始化mysql数据库
sh script/create-dolphinscheduler.sh
# 修改配置文件
ips="hostnames"
masters="hostnames"
workers="hostname:default,hostname:default,hostname:default"
alertServer="hostname"
apiServers="hostname"
# 启动 或 停服 都在某一主机
sh ./install.sh
# 修改配置重启
cd /data03/dolphinscheduler/dolphinscheduler-2.0.5
su dolphinscheduler
: dolphinscheduler
# 停止服务
sh script/stop-all.sh
sh ./install.sh
# 访问地址
http://132.151.46.137:12345/dolphinscheduler/ui/view/login/index.html
bonc_zj bonc@123
3. 踩坑
主要是配置文件调整
# 配置文件,这里采用了mysql8
SPRING_DATASOURCE_URL=jdbc:mysql://132.151.46.136:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false&allowPublicKeyRetrieval=true
SPRING_DATASOURCE_DRIVER_CLASS_NAME=com.mysql.cj.jdbc.Driver
# worker tenant auto create 设置为true,否则会报错tenantCode: tenant does not exists
workerTenantAutoCreate="true"
还有一个hostname的问题,在配置masters的时候我用的ip但不是192段是132段,所以后面出现奇奇怪怪的错误,后来改成hostname
官方答疑