datasophon1.2.1集成Doris3.0.2版本升级手册

Doris3.0.2版本升级手册

前言

此次DataSophon集成高版本Doris3.0.2是基于Rocky Linux8系统进行部署,同时MySQL版本是8.0.29

环境准备

关闭 swap 分区【必须】

在部署 Doris 时,建议关闭 swap 分区。swap 分区是内核发现内存紧张时,会按照自己的策略将部分内存数据移动到配置的 swap 分区,由于内核策略不能充分了解应用的行为,会对数据库性能造成较大影响。所以建议关闭。

通过以下命令可以临时或者永久关闭。

查看是否存在swap分区

swapon --show

永久关闭,使用 Linux root 账户,注释掉 /etc/fstab 中的 swap 分区,然后重启即可彻底关闭 swap 分区

vi  /etc/fstab

/dev/mapper/rl-root     /                       xfs     defaults        0 0
UUID=aa81c65f-2271-49ce-81b2-1c8ba71eff8d /boot                   xfs     defaults        0 0
/dev/mapper/rl-home     /home                   xfs     defaults        0 0
#/dev/mapper/rl-swap     none                    swap    defaults        0 0

注意:修改完成记得重启(reboot)服务器

临时关闭,下次机器启动时,swap 还会被打开。

swapoff -a 

修改虚拟内存【必须】

修改虚拟内存区域至少 2000000

永久关闭

vi /etc/sysctl.conf

# 在文件中添加以下行
vm.max_map_count = 2000000

# 或
echo 'vm.max_map_count = 2000000' >> /etc/sysctl.conf

修改文件后,需要运行sudo sysctl -p命令来重新加载系统配置,使更改生效

临时关闭

sysctl -w vm.max_map_count=2000000

配置最大进程数和最大文件句柄数【必须】

  • 设置系统最大进程数

    在终端中输入ulimit -u,可以查看当前用户的最大进程数限制

    echo '* soft nproc 1000000' >> /etc/security/limits.conf
    echo '* hard nproc 1000000' >> /etc/security/limits.conf
    
  • 设置系统最大打开文件句柄数

    在终端中输入ulimit -n,可以查看当前shell会话的文件描述符(句柄)限制

    echo '* soft nofile 1000000' >> /etc/security/limits.conf
    echo '* hard nofile 1000000' >> /etc/security/limits.conf
    

​ 修改配置文件后,你需要重新登录或重启系统以使更改生效。

关闭透明大页

在部署 Doris 时,建议关闭透明大页。

永久有效解决方法

vi /etc/rc.d/rc.local

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
 echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
 echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

# 或
echo 'if test -f /sys/kernel/mm/transparent_hugepage/enabled; then' >> /etc/rc.d/rc.local
echo '    echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.d/rc.local
echo 'fi' >> /etc/rc.d/rc.local"
echo 'if test -f /sys/kernel/mm/transparent_hugepage/defrag; then' >> /etc/rc.d/rc.local
echo '    echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.d/rc.local
echo 'fi' >> /etc/rc.d/rc.local

chmod +x /etc/rc.d/rc.local

临时有效解决方法

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

禁用SELinux

在安装Doris时,建议禁用SELinux。SELinux(Security-Enhanced Linux)是一个安全模块,用于为Linux系统提供访问控制安全策略。然而,SELinux的严格安全策略有时可能会与某些应用程序(包括Doris)的正常运行产生冲突。

sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/sysconfig/selinux

关闭系统防火墙【推荐】

如果发现端口不通,确认如果是防火墙造成,可以根据配置的 Doris 各组件端口打开相应的端口通信。也可以直接关闭防火墙

sudo systemctl status firewalld
sudo systemctl stop firewalld
sudo systemctl disable firewalld

sudo systemctl status iptables
sudo systemctl stop iptables
sudo systemctl disable iptables

集群时钟同步

Doris 的元数据要求时间精度要小于 5000ms,所以所有集群所有机器要进行时钟同步,避免因为时钟问题引发的元数据不一致导致服务出现异常。

当然某些云服务,比如阿里云的云服务会默认配置,不需要再进行操作。

两种时间同步方式,选择其一进行执行以下操作(每台节点机器都需要执行)。

  • 方式一:使用NTP
sudo yum -y install ntp
sudo systemctl start ntpd
sudo systemctl enable ntpd

测试:

[root@nodeX ~]# date

image-20240603105642291

  • 方式二:使用Chrony

Centos7默认使用Chrony工具而非NTP进行时间同步,Chrony和NTP两者有冲突只能选其一,CDH6推荐使用Chrony,安装和启动命令如下:

# 安装
yum -y install chrony
# 启动
systemctl start chronyd

如果已安装可跳过,修改硬件时钟为UTC,时区为本地时区

# 硬件时钟设置为UTC
pssh -h ./nodes -i "timedatectl set-local-rtc 0"
# 设置本地时区,显示本地时间。如果需要设置上海时区Etc/UTC改成Asia/Shanghai
pssh -h ./nodes -i "timedatectl set-timezone Asia/Shanghai"
# 手动加载RTC设置
pssh -h ./nodes -i "hwclock --systohc"

通过timedatectl查看时间

[root@node1 ~]# timedatectl
      Local time: Tue 2023-08-22 15:14:57 CST
  Universal time: Tue 2023-08-22 07:14:57 UTC
        RTC time: Tue 2023-08-22 07:14:57
       Time zone: Asia/Shanghai (CST, +0800)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: no
      DST active: n/a

安装JDK17

image-20241126011217391

如上Doris官网所述,安装Doris3以上版本,需要使用Java 17。如果还是使用Java 8,则会报错:

StdoutLogger 2024-11-25 16:33:25,817 ERROR: The jdk_version is 8, must be 17.

下载JDK 17

如果有Oracle官网账号,则直接使用下面链接进行没有,没有可以网上进行搜索下载。

wget https://download.oracle.com/java/17/archive/jdk-17.0.10_linux-x64_bin.tar.gz

解压安装

tar -zxvf jdk-17.0.10_linux-x64_bin.tar.gz -C /opt/datasophon/

scp -r /opt/datasophon/jdk-17.0.10 bigdata2:/opt/datasophon/
scp -r /opt/datasophon/jdk-17.0.10 bigdata3:/opt/datasophon/

注意:解压完成之后不需要进行配置环境变量,因为发现其他组件目前还是需要依赖Java 8启动。后续直接在Doris的启动脚本中进行指定Java版本就可以啦!

制作doris部署包

下载官方安装包

官网下载:https://doris.apache.org/zh-CN/download

wget https://apache-doris-releases.oss-accelerate.aliyuncs.com/apache-doris-3.0.2-bin-x64.tar.gz

上传至服务器解压

image-20241126021331080

tar -zxvf apache-doris-3.0.2-bin-x64.tar.gz && mv apache-doris-3.0.2-bin-x64 doris-3.0.2

将状态脚本放进be和fe的bin目录

image-20241126022036307

image-20241126022111499

如果是粘贴下面的内容新建状态脚本文件,可以考虑给两个文件755的权限。

chmod 755 /root/doris-3.0.2/fe/bin/status_fe.sh

chmod 755 /root/doris-3.0.2/be/bin/status_be.sh

status_fe.sh文件

vi /root/doris-3.0.2/fe/bin/status_fe.sh

#  (the "License"); you may not use this file except in compliance with
#  the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.
#

usage="Usage: start.sh (start|stop|restart) <command> "

# if no args specified, show usage
if [ $# -le 1 ]; then
  echo $usage
  exit 1
fi
startStop=$1
shift
command=$1
SH_DIR=`dirname $0`

curdir=`dirname "$0"`
curdir=`cd "$curdir"; pwd`
PID_DIR=`cd "$curdir"; pwd`
pid=$PID_DIR/fe.pid


status(){
  if [ -f $pid ]; then
    ARGET_PID=`cat $pid`
    echo "pid is $ARGET_PID"
    kill -0 $ARGET_PID
    if [ $? -eq 0 ]
    then
      echo "$command is  running "
    else
      echo "$command  is not running"
      exit 1
    fi
  else
    echo "$command  pid file is not exists"
    exit 1
        fi
}
case $startStop in
  (status)
          status
        ;;
  (*)
    echo $usage
    exit 1
    ;;
esac


echo "End $startStop $command."

status_be.sh文件

vi /root/doris-3.0.2/be/bin/status_be.sh

#!/bin/bash
usage="Usage: start.sh (status) <command> "
 
# if no args specified, show usage
if [ $# -lt 1 ]; then
  echo $usage
  exit 1
fi
startStop=$1
shift
 
SH_DIR=`dirname $0`
echo $SH_DIR
export PID_DIR=$SH_DIR
pid=$PID_DIR/be.pid
function get_json(){
  echo "${1//\"/}" | sed "s/.*$2:\([^,}]*\).*/\1/"
}
status(){
  if [ -f $pid ]; then
    ARGET_PID=`cat $pid`
    kill -0 $ARGET_PID
    if [ $? -eq 0 ]
    then
	  # 发送GET请求到指定的URL
      #response=$(curl -s  http://localhost:18030/api/bootstrap)
      response=$(curl -s  http://localhost:18040/api/health)
      # 检查返回值是否为200
      code=$(get_json "${response}" "code")
      echo "BE be pid $ARGET_PID $startStop is  running "
    else
      echo "BE be pid $ARGET_PID $startStop  is not running"
      exit 1
    fi
  else
    echo "BE fe pid $ARGET_PID $startStop  pid file is not exists"
    exit 1
        fi
}
 
case $startStop in
 
  (status)
          status
        ;;
  (*)
    echo $usage
    exit 1
    ;;
esac
 
echo "End $startStop $command."
 

使用命令转换文件为二进制格式

使用命令转换文件为二进制格式,be和fe都需要转换

yum install -y dos2unix

dos2unix /root/doris-3.0.2/fe/bin/*
dos2unix /root/doris-3.0.2/be/bin/*

image-20241126022825438

修改fe、be启动脚本-指定Java的版本路径

vi /root/doris-3.0.2/fe/bin/start_fe.sh

# 大概在124行添加以下内容
export JAVA_HOME=/opt/datasophon/jdk-17.0.10

image-20241126032619761

vi /root/doris-3.0.2/be/bin/start_be.sh

# 可以考虑大概在137行添加以下内容
export JAVA_HOME=/opt/datasophon/jdk-17.0.10

image-20241126033429183

重新构建压缩包

tar -czf doris-3.0.2.tar.gz doris-3.0.2

# 对构建好的压缩包进行md5加密
md5sum doris-3.0.2.tar.gz | awk '{print $1}' > doris-3.0.2.tar.gz.md5

# 将构建好的压缩包拷贝到datasophon的packages路径下
cp ./doris-3.0.2.tar.gz ./doris-3.0.2.tar.gz.md5 /opt/datasophon/DDP/packages/

image-20241126023521161

image-20241126023758816

删除Doris服务及安装包-如果部署过

进入底座运管平台,先停止doris服务,再删除doris服务

image-20241108103117919

进入服务器中删除旧版本doris的数据目录

image-20241108103202182

同时将datasophon目录下部署的旧版本doris安装包删除(所有机器都要删除)

rm -rf /opt/datasophon/flink-1.16.2/
rm -rf /opt/datasophon/flink

升级manager包

替换manager包中的状态脚本

cp -r /root/doris-3.0.2/fe/bin/status_fe.sh /opt/datasophon/datasophon-manager-1.2.1/conf/meta/DDP-1.2.1/DORIS/
cp -r /root/doris-3.0.2/be/bin/status_be.sh /opt/datasophon/datasophon-manager-1.2.1/conf/meta/DDP-1.2.1/DORIS/

image-20241126025150085

编辑service_ddl.json文件

image-20241126025628504

编辑service_ddl.json文件,将其中的1.2.6全部替换为3.0.2

vi /opt/datasophon/datasophon-manager-1.2.1/conf/meta/DDP-1.2.1/DORIS/service_ddl.json

{
  "name": "DORIS",
  "label": "Doris",
  "description": "简单易用、高性能和统一的分析数据库",
  "version": "3.0.2",
  "sortNum": 20,
  "dependencies":[],
  "packageName": "doris-3.0.2.tar.gz",
  "decompressPackageName": "doris-3.0.2",
  "roles": [
    {
      "name": "DorisFE",
      "label": "DorisFE",
      "roleType": "master",
      "cardinality": "1+",
      "logFile": "fe/log/fe.log",
      "jmxPort": 18030,
      "startRunner": {
        "timeout": "600",
        "program": "fe/bin/start_fe.sh",
        "args": [
          "--daemon"
        ]
      },
      "stopRunner": {
        "timeout": "600",
        "program": "fe/bin/stop_fe.sh",
        "args": [
          "--daemon"
        ]
      },
      "statusRunner": {
        "timeout": "60",
        "program": "fe/bin/status_fe.sh",
        "args": [
          "status",
          "fe"
        ]
      },
      "externalLink": {
        "name": "FE UI",
        "label": "FE UI",
        "url": "http://${host}:18030"
      }
    },
    {
      "name": "DorisFEObserver",
      "label": "DorisFEObserver",
      "roleType": "worker",
      "logFile": "fe/log/fe.log",
      "jmxPort": 18031,
      "startRunner": {
        "timeout": "60",
        "program": "fe/bin/start_fe.sh",
        "args": [
          "--daemon"
        ]
      },
      "stopRunner": {
        "timeout": "600",
        "program": "fe/bin/stop_fe.sh",
        "args": [
          "--daemon"
        ]
      },
      "statusRunner": {
        "timeout": "60",
        "program": "fe/bin/status_observer.sh",
        "args": [
          "status",
          "fe"
        ]
      }
    },
    {
      "name": "DorisBE",
      "label": "DorisBE",
      "roleType": "worker",
      "cardinality": "1+",
      "logFile": "be/log/be.INFO",
      "jmxPort": 18040,
      "startRunner": {
        "timeout": "600",
        "program": "be/bin/start_be.sh",
        "args": [
          "--daemon"
        ]
      },
      "stopRunner": {
        "timeout": "600",
        "program": "be/bin/stop_be.sh",
        "args": [
          "--daemon"
        ]
      },
      "statusRunner": {
        "timeout": "60",
        "program": "be/bin/status_be.sh",
        "args": [
          "status",
          "be"
        ]
      }
    }
  ],
  "configWriter": {
    "generators": [
      {
        "filename": "be.conf",
        "configFormat": "properties",
        "outputDirectory": "be/conf",
        "includeParams": [
          "be_priority_networks",
          "sys_log_level",
          "be_port",
          "webserver_port",
          "brpc_port",
          "storage_root_path",
          "mem_limit",
          "custom.be.conf"
        ]
      },
      {
        "filename": "fe.conf",
        "configFormat": "custom",
        "outputDirectory": "fe/conf",
        "templateName": "doris_fe.ftl",
        "includeParams": [
          "fe_heap_size",
          "cluster_id",
          "meta_dir",
          "rpc_port",
          "query_port",
          "fe_priority_networks",
          "lower_case_table_names",
          "enable_outfile_to_local",
          "custom.fe.conf"
        ]
      }
    ]
  },
  "parameters": [
    {
      "name": "cluster_id",
      "label": "集群ID",
      "description": "相同集群ID的FE/BE节点属于同一个集群",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "1"
    },
    {
      "name": "meta_dir",
      "label": "FE元数据的保存目录",
      "description": "FE元数据的保存目录",
      "configType": "path",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "${DORIS_HOME}/fe/doris-meta"
    },
    {
      "name": "rpc_port",
      "label": "FE节点上Thrift服务器的端口",
      "description": "FE 节点上 Thrift 服务器的端口。",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "9020"
    },
    {
      "name": "query_port",
      "label": "FE节点上MySQL服务器的端口",
      "description": "FE 节点上 MySQL 服务器的端口",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "9030"
    },
    {
      "name": "fe_priority_networks",
      "label": "FE优先网段",
      "description": "为那些有多个 IP 地址的服务器声明一个选择策略。\n请注意,最多应该有一个 IP 地址与此列表匹配。这是一个以分号分隔格式的列表,用 CIDR 表示法,例如 10.10.10.0/24。 如果没有匹配这条规则的ip,会随机选择一个。",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "0.0.0.0/24"
    },
    {
      "name": "fe_heap_size",
      "label": "FE最大堆内存",
      "description": "FE最大堆内存,单位为GB",
      "required": true,
      "configType": "map",
      "type": "slider",
      "minValue": 0,
      "maxValue": 64,
      "value": "",
      "unit": "GB",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "8"
    },
    {
      "name": "lower_case_table_names",
      "label": "表名大小写是否敏感,只有初始化安装时有效",
      "description": "值为 0 时,表名大小写敏感。默认为0。\n值为 1 时,表名大小写不敏感,doris在存储和查询时会将表名转换为小写。\n优点是在一条语句中可以使用表名的任意大小写形式\n缺点是建表后无法获得建表语句中指定的表名,show tables 查看的表名为指定表名的小写。\n值为 2 时,表名大小写不敏感,doris存储建表语句中指定的表名,查询时转换为小写进行比较。\n优点是show tables 查看的表名为建表语句中指定的表名;\n缺点是同一语句中只能使用表名的一种大小写形式。",
      "required": true,
      "type": "select",
      "value": "0",
      "configurableInWizard": true,
      "selectValue": [
        "0",
        "1",
        "2"
      ],
      "hidden": false,
      "defaultValue": "0"
    },
    {
      "name": "enable_outfile_to_local",
      "label": "是否允许outfile函数将结果导出到本地磁盘",
      "description": "是否允许 outfile 函数将结果导出到本地磁盘",
      "required": true,
      "type": "switch",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": false
    },
    {
      "name": "be_priority_networks",
      "label": "BE优先网段",
      "description": "为那些有多个 IP 地址的服务器声明一个选择策略。\n请注意,最多应该有一个 IP 地址与此列表匹配。这是一个以分号分隔格式的列表,用 CIDR 表示法,例如 10.10.10.0/24。 如果没有匹配这条规则的ip,会随机选择一个。",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "0.0.0.0/24"
    },
    {
      "name": "sys_log_level",
      "label": "BE服务日志级别",
      "description": "",
      "required": true,
      "type": "select",
      "value": "",
      "configurableInWizard": true,
      "selectValue": [
        "INFO",
        "WARNING",
        "ERROR",
        "FATAL"
      ],
      "hidden": false,
      "defaultValue": "INFO"
    },
    {
      "name": "be_port",
      "label": "BE admin端口",
      "description": "BE admin端口",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "9060"
    },
    {
      "name": "webserver_port",
      "label": "BE WebServer端口",
      "description": "BE WebServer端口",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "18040"
    },
    {
      "name": "brpc_port",
      "label": "BE Rpc端口",
      "description": "BE Rpc端口",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "8060"
    },
    {
      "name": "storage_root_path",
      "label": "BE数据存储目录",
      "description": "BE数据存储目录,可配置多个,按分号分隔,例如/data1,medium:HDD;/data2,medium:SSD;/data3",
      "configType": "path",
      "separator": ";",
      "required": true,
      "type": "multiple",
      "value": [
        "/data/be/storage"
      ],
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": ""
    },
    {
      "name": "mem_limit",
      "label": "BE进程使用服务器最大内存百分比",
      "description": "限制BE进程使用服务器最大内存百分比。用于防止BE内存挤占太多的机器内存,该参数必须大于0,当百分大于100%之后,该值会默认为100%。",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "100%"
    },
    {
      "name": "custom.fe.conf",
      "label": "自定义配置fe.conf",
      "description": "自定义配置",
      "configType": "custom",
      "required": false,
      "type": "multipleWithKey",
      "value": [],
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": ""
    },
    {
      "name": "custom.be.conf",
      "label": "自定义配置be.conf",
      "description": "自定义配置",
      "configType": "custom",
      "required": false,
      "type": "multipleWithKey",
      "value": [],
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": ""
    }
  ]
}

修改doris_fe.ftl模板文件-每个节点都需要修改

JAVA_OPTS="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -Xloggc:$LOG_DIR/log/fe.gc.log.$CUR_DATE -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=50M -Dlog4j2.formatMsgNoLookups=true"

# For jdk 17, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_17="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -Xmx8192m -Xms8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=$LOG_DIR -Xlog:gc*:$LOG_DIR/fe.gc.log.$CUR_DATE:time,uptime:filecount=10,filesize=50M --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens java.base/jdk.internal.ref=ALL-UNNAMED"

# Set your own JAVA_HOME
JAVA_HOME=/opt/datasophon/jdk-17.0.10

以上内容摘自与Doris3.0.2的fe.conf。将以上内容覆盖到doris_fe.ftl文件中,最主要是添加JAVA_OPTS_FOR_JDK_17。

DataSophon中ftl模板脚本主要是针对Java8部署的,它会下面的内容覆盖追加到fe.conf,而我们现在部署的是Doris3.0.2版本,它依赖于Java 17版本,所以需要手动指定JAVA_OPTS_FOR_JDK_17。同时可以考虑在文件中指定Java 17的路径

--doris_fe.ftl文件原内容-其中一部分

JAVA_OPTS="-Xmx${fe_heap_size}G -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"


# For jdk 9+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_9="-Xmx${fe_heap_size}G -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$DATE:time"

doris_fe.ftl文件

vi /opt/datasophon/datasophon-worker/conf/templates/doris_fe.ftl

image-20241126034948575

fe.ftl文件

里面也有一个fe.ftl文件,发现Doris部署用不上它,可以考虑不更改它也没事。暂时不知道它是干什么用的

重启服务,重装work

各节点worker重启

sh /opt/datasophon/datasophon-worker/bin/datasophon-worker.sh restart worker

主节点重启api

sh /opt/datasophon/datasophon-manager-1.2.1/bin/datasophon-api.sh restart api

如有必要(比如识别不到或者节点内存异常等)可以重装work, 等待2分钟。否则这里可以省略

image-20241108103334305

安装新版本Doris

添加新版本的doris服务,更改相关配置,进行安装即可

image-20241126040818999

image-20241126041006316

image-20241126041030784

image-20241126041051513

设置密码和查看节点注册情况

MySQL Client 连接 FE

# 使用 MySQL Client 连接 FE
 
mysql -h 192.168.10.21 -P 9030 -uroot

默认 root 无密码,通过以下命令修改 root 密码。

SET PASSWORD FOR 'root' = PASSWORD('yixiao666');

查看 BE 状态

SHOW PROC '/backends';

image-20241126133305247

Alive 为true表示添加成功。或使用数据库连接工具连接上Doris,如下图查看BE注册连接情况。

image-20241126041453346

到此,DataSophon集成高版本Doris3.0.2就成功了。

posted @ 2024-11-26 14:15  xiongshengxiao  阅读(81)  评论(0)    收藏  举报