[数据集成] Apache SeaTunnel : 为解决多源异构数据源间的数据同步与轻量集成而生

0 序

tags:

大数据
数据集成
数据仓库
数据采集
综述/概述

1 概述

产品介绍

Apache SeaTunnel（原名为 Waterdrop）是一个多模态、高性能、分布式、海量数据集成工具，隶属于 Apache 软件基金会（ASF）的顶级项目（Top-Level Project）。它专注于解决多源、异构数据源之间的高效、可靠数据同步问题，提供 EtLT（extract, lightweight Transform, Load） 架构模式，支持离线批处理、实时流处理、CDC（Change Data Capture）以及多模态（文本、图像、向量等）数据集成。
产品定位：统一的数据集成层（Data Integration Layer），用于连接各类数据源与目标系统。
诞生的背景：在大数据生态中，数据源与目标系统种类繁多（数据库、消息队列、文件系统、SaaS API、数据湖、向量数据库等），缺少一个统一的、高性能的、易用的数据集成工具。SeaTunnel 应运而生，目标是为数据工程师提供一个"写一次配置，在任何引擎上运行"的体验。

seatunnel-arch.20260814014949960

Apache SeaTunnel | Apache SeaTunnel

解决的核心问题：
- 异构数据源间的高吞吐低延迟数据同步
- CDC（增量变更捕获）的 Exactly-Once 语义保障
- 多引擎（Zeta、Flink、Spark）适配，避免厂商锁定
- 配置化而非编码化，降低使用门槛
URLs：
- 官网：https://seatunnel.apache.org/
  - https://seatunnel.apache.org/zh-CN/docs/getting-started/overview/
- GitHub：https://github.com/apache/seatunnel
- 下载页面：https://seatunnel.apache.org/download

发展历程

时间	里程碑
2017-08	项目创立（原名 Waterdrop）
2021	进入 Apache 孵化器，更名为 Apache SeaTunnel
2022	发布里程碑版本 2.0.x，引入 SeaTunnel Zeta Engine（自研引擎）
2023	成为 Apache 顶级项目（TLP），发布 2.3.x 系列，支持 CDC、Schema Evolution
2024-2025	持续迭代，新增多模态集成、LLM/RAG 支持、Embedding 能力，连接器数量突破 200+
2026-02	发布 2.3.13 版本，支持 Checkpoint API、Flink 1.20、HugeGraph、DuckDB、Lance 数据集等

核心功能

约 200 种数据连接器：覆盖数据库（MySQL、PostgreSQL、Oracle、SQL Server、ClickHouse、Doris、StarRocks 等）、消息队列（Kafka、Pulsar、RocketMQ 等）、文件系统（HDFS、S3、Local、FTP、SFTP 等）、数据湖（Iceberg、Hudi、Paimon 等）、向量数据库（Milvus、Qdrant 等）、SaaS API（Salesforce、Slack、Feishu、DingTalk 等）、AI 相关（Embedding、LLM 集成）等。
CDC（Change Data Capture）：支持 MySQL-CDC、PostgreSQL-CDC、Oracle-CDC、SQL Server-CDC、MongoDB-CDC，提供 Exactly-Once 语义保障。
多引擎执行：同一份配置文件可以在 SeaTunnel Zeta Engine、Apache Flink、Apache Spark 上无缝运行。
Schema Evolution：自动适配源端 Schema 变更（新增列、类型变更），并同步到目标端。
实时流处理与批处理：在单一 Pipeline 定义中支持 BATCH 和 STREAMING 模式。
Transform 能力：内置 SQL Transform、FieldMapper、Embedding、RegexExtract、JSONPath 等多种数据转换插件。
SaveMode / DataSaveMode：支持多种数据写入策略（Append、Overwrite、Create、ErrorIfExists）。
多表同步：一个连接器可一次性同步多张表。
多模态集成：支持文本、图像、向量数据的统一集成（适用于 LLM/RAG 场景）。
REST API & UI：提供作业管理、监控的 REST API 和 Web UI。
Metalake：元数据湖概念，统一管理数据源元数据信息。
MCP 支持：SeaTunnel MCP Server，提供 AI Agent 与 SeaTunnel 交互的能力。

核心优势

优势	说明
EtLT 架构	相比传统 ETL，EtLT 仅做轻量抽取和加载，将复杂转换交给目标数仓/数据湖，显著降低 Pipeline 复杂度
引擎无关	同一配置可在 Zeta/Flink/Spark 三个引擎上运行，无厂商锁定
超高吞吐	Zeta Engine 自研引擎在 Benchmark 中展现出极高吞吐能力，适合海量数据迁移
易用性好	只需编写 HOCON 配置文件即可定义数据 Pipeline，无需编码
Exactly-Once	CDC 与流式场景下提供精确一次语义
连接器丰富	约 200 个开箱即用的连接器，涵盖主流数据系统
开源	Apache 2.0 协议，社区活跃，贡献者超 600 人

主要短板

文档尚待完善：部分连接器文档更新滞后，中英文文档差异较大，配置参数缺少详细说明。
Zeta Engine 相对年轻：比之 Flink/Spark 经大规模验证，Zeta Engine 作为自研引擎仍在快速迭代中，部分高级功能尚不成熟。
Transform 能力有限：相比专业的 ETL 工具，SeaTunnel 的内置 Transform 偏轻量，复杂转换仍需依赖目标系统或外部工具。
多引擎一致性差异：不同引擎之间对同一配置的行为存在细微差异。

局限性

主要面向大批量数据集成场景，不适合 OLTP 级别的小事务处理。
实时处理的时间语义（EventTime、Watermark、Session Window）不如 Flink 原生 API 灵活。
不支持跨 Pipeline 的复杂 DAG 依赖管理。

适用场景

数仓/数据湖同步：将 MySQL/Oracle/PostgreSQL 等 OLTP 数据库实时同步到 ClickHouse/Doris/StarRocks/Iceberg/Paimon 等分析型存储。
CDC 实时同步：数据库变更日志实时捕获并同步到下游目标系统。
多源数据汇聚：将分散在不同数据库、消息队列、文件系统中的数据统一汇聚到一个目标系统。
SaaS 数据集成：从 Salesforce、Slack、飞书等 SaaS 平台抽取数据并加载到数仓。
AI/RAG 数据管道：将多模态数据通过 Embedding Transform 处理后写入向量数据库。
日志/文件采集：从 HDFS/S3/Local 采集日志文件并写入分析系统。

产品	定位	区别点
Apache NiFi	流式数据流管理	NiFi 更侧重可视化 Flow 设计；SeaTunnel 更侧重高性能批量同步与 CDC
Apache Flink	流式计算引擎	Flink 是有状态计算引擎，SeaTunnel 在其之上提供了简化配置的集成层
Debezium + Kafka Connect	CDC 与数据集成	Debezium 专注 CDC，通常需 Kafka 作为传输通道；SeaTunnel CDC 内置，无需额外中间件
Airbyte	数据集成平台	Airbyte 提供商业 UI 与 Connector Builder；SeaTunnel 侧重高性能与多引擎部署
DataX（阿里）	离线数据同步	DataX 仅支持离线批同步，不支持流式与 CDC； 20260814: 5.6k fork / 17.3k star
Flink CDC	流式 CDC	Flink CDC 需 Flink 环境；SeaTunnel CDC 可在 Zeta/Flink/Spark 上运行 20260814: 2.2k fork / 5.6k star

发展趋势

开源社区活跃度：项目持续高速迭代，最新版本 2.3.13 于 2026 年 3 月发布。GitHub 拥有 9.6k+ Stars、2.3k+ Forks、650+ Open Issues，贡献者 600+，社区非常活跃。
项目发展趋势总结：SeaTunnel 正从"数据库同步工具"演进为"全场景数据集成平台"，方向涵盖多模态、LLM/RAG、Embedding、MCP AI Agent 集成等前沿领域，同时保持对传统离线/实时同步场景的深度覆盖。

2 工作原理与架构

概念术语

术语	说明
SeaTunnel Zeta Engine	SeaTunnel 自研的分布式执行引擎，支持批/流处理
Connector（连接器）	数据源（Source）或目标端（Sink）的插件实现
Transform（转换）	在 Source 与 Sink 之间对数据进行轻量处理
Pipeline（管道）	Source -> Transform -> Sink 构成的数据处理链路
Catalog	用于获取目标表元数据信息（Schema、主键、分区等）
SaveMode	数据写入模式（Append、Overwrite、Create、ErrorIfExists）
Schema Evolution	Schema 变更自动同步能力
CDC	增量数据变更捕获，支持 Exactly-Once 语义
Multi-Table Sink	一个连接器同时写入多张目标表
Metalake	元数据湖，集中管理数据源/目标元数据信息
SeaTunnel MCP Server	AI Agent 通过 MCP 协议与 SeaTunnel 交互

架构设计

总体架构

SeaTunnel Workflow

技术架构

                       +------------+  +------------+
                       | HOCON 配置  |  | Web UI /   |
                       | 文件       |  | REST API   |
                       +------+-----+  +------+-----+
                              |               |
                              v               v
                       +------+-----+  +------+-----+
                       | SeaTunnel API / Pipeline 构建器  |
                       +------+-----+  +------+-----+
                              |               |
                   +----------+-------+-------+
                   |          |               |
                   v          v               v
             +-----+--+ +----+----+ +----+----+
             | Zeta   | | Flink   | | Spark   |
             | Engine | | Starter | | Starter |
             +-----+--+ +----+----+ +----+----+
                   |          |               |
                   v          v               v
             +-----+----+----+----+-----------+
             |    Source / Sink / Transform    |
             |        连接器体系(约200种)       |
             +-----+----+----+----+-----------+
                   |    |    |    |
                   v    v    v    v
             +-----+----+----+----+-----------+
             |  DB / MQ / FS / Lake / VecDB   |
             |   外部数据系统                    |
             +----------------------------------+

运行原理

用户编写 HOCON 配置文件，定义 Source、Transform、Sink。
SeaTunnel API 层解析配置，构建 Pipeline DAG。
Pipeline 被提交到指定的执行引擎（Zeta / Flink / Spark）。
执行引擎根据配置加载对应的连接器插件。
数据从 Source 流入，经过可选 Transform 处理，最终写入 Sink。
引擎提供容错（Checkpoint）、并行度控制、Metrics 等运行时保障。

Zeta Engine 详解

SeaTunnel Zeta Engine 是 SeaTunnel 自研的分布式执行引擎。

去中心化架构：Master-Worker 架构，Master 调度，Worker 执行 Task。
Checkpoint 机制：基于 Chandy-Lamport 分布式快照算法，提供 Exactly-Once 语义。
动态资源管理：支持动态扩缩容。
REST API：提交、停止、查询作业、获取 Metrics。
Kubernetes 支持：可部署在 K8s 集群。
默认 G1 GC：优化大堆内存 GC 性能。

3 使用指南

CASE 安装部署 + 最简示例(批处理)

前提条件：JDK 8+（推荐 JDK 11）
Linux / macOS / Windows(Git-Bash)：

step1 下载安装包

# 下载安装包
curl -O https://dlcdn.apache.org/seatunnel/2.3.13/apache-seatunnel-2.3.13-bin.tar.gz
tar -xzf apache-seatunnel-2.3.13-bin.tar.gz
cd apache-seatunnel-2.3.13

# 配置环境变量 (可选步骤)
# export SEATUNNEL_HOME=/opt/apache-seatunnel-2.3.6
export SEATUNNEL_HOME=/h/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13
#export PATH=$SEATUNNEL_HOME/bin:$PATH
export PATH=$SEATUNNEL_HOME/bin:$PATH

step2 配置、下载 connectors

# 修改 config/plugin_config，只保留你需要的：（也可以不改，全部下载） [可选步骤]
--connectors-v2--
connector-jdbc-mysql
connector-jdbc-oracle
connector-console # 这个建议保留，用于调试输出

# 安装连接器插件 (注: 执行后，产生的 connectors 在 ./connectors/ 目录下)
sh bin/install-plugin.sh

step3 配置数据同步任务（模拟数据源 => Console）

配置config/v2.batch.config.template

此处需要说明，再执行任务时可以指定配置文件，所以配置文件目录和命名没有要求

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
######
###### This config file is a demonstration of streaming processing in SeaTunnel config
######

env {
 # You can set SeaTunnel environment configuration here
  parallelism = 2
  job.mode = "BATCH"
  checkpoint.interval = 10000
}

source {
 # This is a example source plugin **only for test and demonstrate the feature source plugin**
  FakeSource {
    parallelism = 2
    plugin_output = "fake"
    row.num = 16
    schema = {
      fields {
        name = "string"
        age = "int"
      }
    }
  }

 # If you would like to get more information about how to configure SeaTunnel and see full list of source plugins,
 # please go to https://seatunnel.apache.org/docs/connector-v2/source
}

# transform 组件的配置为可选，可删除本段配置
transform {
  FieldMapper {
    source_table_name = "fake"
    result_table_name = "fake1"
    field_mapper = {
      age = age
      name = name2
    }
  }
}

sink {
  Console {
  }

 # If you would like to get more information about how to configure SeaTunnel and see full list of sink plugins,
 # please go to https://seatunnel.apache.org/docs/connector-v2/sink
}

此处输出的name和age是根据指定的数据类型string和int模拟出来的数据，此处不必考虑实际意义

step4 运行测试任务

修改 JVM参数的配置 [可选步骤]

// 编辑 bin/seatunnel.[sh|cmd] 文件，在文件开头附近添加或修改
export JVM_ARGS="-Xmx2g -Xms1g -XX:MaxDirectMemorySize=1g"
...

内存配置建议:

机器内存	JVM堆内存	并行度	Batch Size
4G	-Xmx2g -Xms1g	2	500
8G	-Xmx4g -Xms2g	4	1000
16G	-Xmx8g -Xms4g	8	2000
32G	-Xmx16g -Xms8g	16	5000

启动测试任务

cd $SEATUNNEL_HOME

（Linux 环境）: （若 Windows 上运行会报错: 错误: 找不到或无法加载主类 org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient）
./bin/seatunnel.sh --config ./config./v2.batch.config.template -m local

（Windows环境[Git-Bash]）:
./bin/seatunnel.cmd --config ./config./v2.batch.config.template -m local

日志输出如下，红框内即为输出到控制台的name和age：

sea-tunnel-dmeo-running-log.20260814120609521

为了了解任务执行情况，还可以在控制台找到相关信息

***********************************************
           Job Statistic Information
***********************************************
Start Time                : 2026-08-14 12:01:00
End Time                  : 2026-08-14 12:01:03
Total Time(s)             :                   3
Total Read Count          :                  32
Total Write Count         :                  32
Total Failed Count        :                   0
***********************************************

附件:详细运行日志

$ ./bin/seatunnel.cmd --config ./config./v2.batch.config.template -m local
2026-08-14 12:00:23,366 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration 'H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\config\seatunnel.yaml' from System property 'seatunnel.config'
2026-08-14 12:00:23,370 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\config\seatunnel.yaml
2026-08-14 12:00:23,372 INFO  [o.a.s.e.c.c.SeaTunnelConfig   ] [main] - seatunnel.home is H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13
2026-08-14 12:00:23,477 INFO  [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-08-14 12:00:23,478 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration 'H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\config\hazelcast.yaml' from System property 'hazelcast.config'
2026-08-14 12:00:23,478 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\config\hazelcast.yaml
2026-08-14 12:00:23,860 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration 'H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\config\hazelcast-client.yaml' from System property 'hazelcast.client.config'
2026-08-14 12:00:23,860 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\config\hazelcast-client.yaml
2026-08-14 12:00:23,926 WARN  [c.h.i.AddressPicker           ] [main] - [LOCAL] [seatunnel-813134] [5.1] You configured your member address as host name. Please be aware of that your dns can be spoofed. Make sure that your dns configurations are correct.
2026-08-14 12:00:23,926 INFO  [c.h.i.AddressPicker           ] [main] - [LOCAL] [seatunnel-813134] [5.1] Resolving domain name 'localhost' to address(es): [127.0.0.1, 0:0:0:0:0:0:0:1]
2026-08-14 12:00:23,927 INFO  [c.h.i.AddressPicker           ] [main] - [LOCAL] [seatunnel-813134] [5.1] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1]
2026-08-14 12:00:23,962 INFO  [o.a.s.e.s.SeaTunnelServer     ] [main] - SeaTunnel server start...
2026-08-14 12:00:23,964 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Based on Hazelcast IMDG version: 5.1.0 (20220228 - 21f20e7)
2026-08-14 12:00:23,964 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Cluster name: seatunnel-813134
2026-08-14 12:00:23,965 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-813134] [5.1]

_____               _____                             _
/  ___|             |_   _|                           | |
\ `--.   ___   __ _   | |   _   _  _ __   _ __    ___ | |
 `--. \ / _ \ / _` |  | |  | | | || '_ \ | '_ \  / _ \| |
/\__/ /|  __/| (_| |  | |  | |_| || | | || | | ||  __/| |
\____/  \___| \__,_|  \_/   \__,_||_| |_||_| |_| \___||_|


2026-08-14 12:00:23,965 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Copyright ? 2021-2024 The Apache Software Foundation. Apache SeaTunnel, SeaTunnel, and its feather logo are trademarks of The Apache Software Foundation.
2026-08-14 12:00:23,965 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Integrity Checker is disabled. Fail-fast on corrupted executables will not be performed.
To enable integrity checker do one of the following:
  - Change member config using Java API: config.setIntegrityCheckerEnabled(true);
  - Change XML/YAML configuration property: Set hazelcast.integrity-checker.enabled to true
  - Add system property: -Dhz.integritychecker.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_INTEGRITYCHECKER_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2026-08-14 12:00:23,967 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] The Jet engine is disabled.
To enable the Jet engine on the members, do one of the following:
  - Change member config using Java API: config.getJetConfig().setEnabled(true)
  - Change XML/YAML configuration property: Set hazelcast.jet.enabled to true
  - Add system property: -Dhz.jet.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_JET_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2026-08-14 12:00:51,366 INFO  [c.h.s.security                ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Enable DEBUG/FINE log level for log category com.hazelcast.system.security  or use -Dhazelcast.security.recommendations system property to see security recommendations and the status of current config.
2026-08-14 12:00:51,457 INFO  [o.a.s.e.s.SeaTunnelNodeContext] [main] - Using LiteNodeDropOutTcpIpJoiner TCP/IP discovery
2026-08-14 12:00:51,460 WARN  [c.h.c.CPSubsystem             ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
2026-08-14 12:00:51,705 INFO  [.c.c.DefaultClassLoaderService] [main] - start classloader service with cache mode
2026-08-14 12:00:51,712 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration 'H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\config\seatunnel.yaml' from System property 'seatunnel.config'
2026-08-14 12:00:51,712 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\config\seatunnel.yaml
2026-08-14 12:00:51,718 INFO  [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-08-14 12:00:51,718 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration 'H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\config\hazelcast.yaml' from System property 'hazelcast.config'
2026-08-14 12:00:51,718 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\config\hazelcast.yaml
2026-08-14 12:00:51,733 WARN  [o.a.s.e.s.TaskExecutionService] [pool-3-thread-1] - [localhost]:5801 [seatunnel-813134] [5.1] The Node is not ready yet, Node state STARTING,looking forward to the next scheduling
2026-08-14 12:00:51,734 INFO  [o.a.s.e.s.TaskExecutionService] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Created new BusWork : 1276666395
2026-08-14 12:00:53,280 WARN  [a.s.e.s.s.s.DefaultSlotService] [hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource manager, will retry later. this address: [localhost]:5801
2026-08-14 12:00:53,281 INFO  [o.a.s.e.s.CoordinatorService  ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Start pending job schedule thread
2026-08-14 12:00:54,960 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-7-thread-1] - [localhost]:5801 [seatunnel-813134] [5.1]
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   1
corePoolSize              :                  10
maximumPoolSize           :          2147483647
poolSize                  :                   1
completedTaskCount        :                   0
taskCount                 :                   1
***********************************************

2026-08-14 12:00:54,969 INFO  [o.a.s.e.s.JettyService        ] [main] - SeaTunnel REST service will start on port 8080
2026-08-14 12:00:54,978 INFO  [o.a.s.s.o.e.j.u.log           ] [main] - Logging initialized @32230ms to org.apache.seatunnel.shade.org.eclipse.jetty.util.log.Slf4jLog
2026-08-14 12:00:55,051 WARN  [a.s.s.o.e.j.s.h.ContextHandler] [main] - Empty contextPath
2026-08-14 12:00:55,101 INFO  [o.a.s.s.o.e.j.s.Server        ] [main] - jetty-9.4.56.v20240826; built: 2024-08-26T17:15:05.868Z; git: ec6782ff5ead824dabdcf47fa98f90a4aedff401; jvm 1.8.0_261-b12
2026-08-14 12:00:55,133 INFO  [o.a.s.s.o.e.j.s.session       ] [main] - DefaultSessionIdManager workerName=node0
2026-08-14 12:00:55,133 INFO  [o.a.s.s.o.e.j.s.session       ] [main] - No SessionScavenger set, using defaults
2026-08-14 12:00:55,135 INFO  [o.a.s.s.o.e.j.s.session       ] [main] - node0 Scavenging every 660000ms
2026-08-14 12:00:55,319 INFO  [a.s.s.o.e.j.s.h.ContextHandler] [main] - Started o.a.s.s.o.e.j.s.ServletContextHandler@30db5536{/,null,AVAILABLE}
2026-08-14 12:00:55,348 INFO  [.s.s.o.e.j.s.AbstractConnector] [main] - Started ServerConnector@2eed37f4{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2026-08-14 12:00:55,349 INFO  [o.a.s.s.o.e.j.s.Server        ] [main] - Started @32601ms
2026-08-14 12:00:55,394 INFO  [c.h.i.d.Diagnostics           ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-08-14 12:00:55,402 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5801 is STARTING
2026-08-14 12:00:57,511 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-3] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5823 is added to the blacklist.
2026-08-14 12:00:57,510 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-6] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5813 is added to the blacklist.
2026-08-14 12:00:57,511 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-2] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5827 is added to the blacklist.
2026-08-14 12:00:57,511 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-7] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5819 is added to the blacklist.
2026-08-14 12:00:57,511 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-9] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5806 is added to the blacklist.
2026-08-14 12:00:57,511 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-11] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5810 is added to the blacklist.
2026-08-14 12:00:57,512 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-13] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5829 is added to the blacklist.
2026-08-14 12:00:57,510 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-5] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5815 is added to the blacklist.
2026-08-14 12:00:57,514 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-4] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5825 is added to the blacklist.
2026-08-14 12:00:57,514 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-10] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5804 is added to the blacklist.
2026-08-14 12:00:57,511 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-16] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5822 is added to the blacklist.
2026-08-14 12:00:57,515 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-1] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5821 is added to the blacklist.
2026-08-14 12:00:57,516 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-14] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5802 is added to the blacklist.
2026-08-14 12:00:57,516 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-8] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5817 is added to the blacklist.
2026-08-14 12:00:57,516 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-12] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5808 is added to the blacklist.
2026-08-14 12:00:57,516 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-15] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5820 is added to the blacklist.
2026-08-14 12:00:58,277 WARN  [a.s.e.s.s.s.DefaultSlotService] [hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource manager, will retry later. this address: [localhost]:5801
2026-08-14 12:00:59,539 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-3] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5826 is added to the blacklist.
2026-08-14 12:00:59,539 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-9] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5812 is added to the blacklist.
2026-08-14 12:00:59,540 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-5] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5807 is added to the blacklist.
2026-08-14 12:00:59,540 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-6] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5830 is added to the blacklist.
2026-08-14 12:00:59,540 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-4] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5805 is added to the blacklist.
2026-08-14 12:00:59,540 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-14] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5803 is added to the blacklist.
2026-08-14 12:00:59,540 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-7] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5814 is added to the blacklist.
2026-08-14 12:00:59,540 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-2] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5824 is added to the blacklist.
2026-08-14 12:00:59,540 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-13] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5816 is added to the blacklist.
2026-08-14 12:00:59,541 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-1] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5828 is added to the blacklist.
2026-08-14 12:00:59,541 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-10] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5811 is added to the blacklist.
2026-08-14 12:00:59,541 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-16] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5809 is added to the blacklist.
2026-08-14 12:00:59,556 INFO  [s.j.LiteNodeDropOutTcpIpJoiner] [hz.main.cached.thread-11] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5818 is added to the blacklist.
2026-08-14 12:01:00,505 INFO  [c.h.i.c.ClusterService        ] [main] - [localhost]:5801 [seatunnel-813134] [5.1]

Members {size:1, ver:1} [
        Member [localhost]:5801 - bcd4cf41-9ce4-4509-8631-7d178b381cec [master node] [active master] this
]

2026-08-14 12:01:00,526 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5801 is STARTED
2026-08-14 12:01:00,560 INFO  [.h.i.p.i.PartitionStateManager] [hz.main.cached.thread-12] - [localhost]:5801 [seatunnel-813134] [5.1] Initializing cluster partition table arrangement...
2026-08-14 12:01:00,574 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel-813134] [5.1] Running with 2 response threads, dynamic=true
2026-08-14 12:01:00,587 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-813134] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2026-08-14 12:01:00,587 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-813134] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2026-08-14 12:01:00,594 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-813134] [5.1] Trying to connect to cluster: seatunnel-813134
2026-08-14 12:01:00,595 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-813134] [5.1] Trying to connect to [localhost]:5801
2026-08-14 12:01:00,598 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-813134] [5.1] This node become a new active master node, begin init coordinator service
2026-08-14 12:01:00,613 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-813134] [5.1] Loaded event handlers: [org.apache.seatunnel.api.event.LoggingEventHandler@1ef13248]
2026-08-14 12:01:00,616 INFO  [.p.t.AuthenticationMessageTask] [hz.main.priority-generic-operation.thread-0] - [localhost]:5801 [seatunnel-813134] [5.1] Received auth from Connection[id=1, /127.0.0.1:5801->/127.0.0.1:45457, qualifier=null, endpoint=[127.0.0.1]:45457, remoteUuid=cfdd5a2e-ab9f-4a42-9654-ce8e218e290c, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: cfdd5a2e-ab9f-4a42-9654-ce8e218e290c, client name: hz.client_1, client version: 5.1
2026-08-14 12:01:00,618 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-813134] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2026-08-14 12:01:00,618 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-813134] [5.1] Authenticated with server [localhost]:5801:bcd4cf41-9ce4-4509-8631-7d178b381cec, server version: 5.1, local address: /127.0.0.1:45457
2026-08-14 12:01:00,619 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel-813134] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-08-14 12:01:00,628 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-7] - hz.client_1 [seatunnel-813134] [5.1]

Members [1] {
        Member [localhost]:5801 - bcd4cf41-9ce4-4509-8631-7d178b381cec [master node]
}

2026-08-14 12:01:00,651 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
2026-08-14 12:01:00,729 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Loading config file from path: .\config.\v2.batch.config.template
2026-08-14 12:01:00,892 INFO  [o.a.s.c.s.u.ConfigShadeUtils  ] [main] - Load config shade spi: [base64]
2026-08-14 12:01:00,926 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Parsed config file:
{
    "env" : {
        "parallelism" : 2,
        "job.mode" : "BATCH",
        "checkpoint.interval" : 10000
    },
    "source" : [
        {
            "parallelism" : 2,
            "plugin_output" : "fake",
            "row.num" : 16,
            "schema" : {
                "fields" : {
                    "name" : "string",
                    "age" : "int"
                }
            },
            "plugin_name" : "FakeSource"
        }
    ],
    "transform" : [
        {
            "source_table_name" : "fake",
            "result_table_name" : "fake1",
            "field_mapper" : {
                "age" : "age",
                "name" : "name2"
            },
            "plugin_name" : "FieldMapper"
        }
    ],
    "sink" : [
        {
            "plugin_name" : "Console"
        }
    ]
}

2026-08-14 12:01:00,936 INFO  [p.MultipleTableJobConfigParser] [main] - add common jar in plugins :[]
2026-08-14 12:01:00,954 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\connectors
2026-08-14 12:01:00,958 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='FakeSource'} at: [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-fake-2.3.13.jar]
2026-08-14 12:01:00,958 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='FakeSource'}: [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-fake-2.3.13.jar]
2026-08-14 12:01:00,964 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\connectors
2026-08-14 12:01:00,964 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='transform', pluginName='FieldMapper'}: []
2026-08-14 12:01:00,969 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\connectors
2026-08-14 12:01:00,971 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Console'} at: [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-console-2.3.13.jar]
2026-08-14 12:01:00,971 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Console'}: [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-console-2.3.13.jar]
2026-08-14 12:01:00,974 WARN  [o.a.s.a.c.ReadonlyConfig      ] [main] - Please use the new key 'plugin_input' instead of the deprecated key 'source_table_name'.
2026-08-14 12:01:00,974 WARN  [o.a.s.a.c.u.ConfigUtil        ] [main] - Option 'plugin_input' is a List, and it is recommended to configure it as ["string1","string2"]; we will only use ',' to split the String into a list.
2026-08-14 12:01:00,975 WARN  [o.a.s.a.c.ReadonlyConfig      ] [main] - Please use the new key 'plugin_output' instead of the deprecated key 'result_table_name'.
2026-08-14 12:01:00,975 INFO  [o.a.s.e.c.p.ConfigParserUtil  ] [main] - Currently, incorrect configuration of plugin_input and plugin_output options don't affect job running. In the future we will ban incorrect configurations.
2026-08-14 12:01:00,975 WARN  [o.a.s.e.c.p.ConfigParserUtil  ] [main] - This configuration is not recommended. A source/transform(FieldMapper) is configured with 'plugin_output' option value of 'fake1', but subsequent transform/sink(Console) is not configured with 'plugin_input' option.
2026-08-14 12:01:00,975 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sources.
2026-08-14 12:01:01,048 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - get the CatalogTable from source FakeSource: FakeSource.null.fake
2026-08-14 12:01:01,062 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSource Plugin from H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\connectors
2026-08-14 12:01:01,064 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='FakeSource'} at: [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-fake-2.3.13.jar]
2026-08-14 12:01:01,064 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='FakeSource'}: [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-fake-2.3.13.jar]
2026-08-14 12:01:01,066 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all transforms.
2026-08-14 12:01:01,071 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelTransform Plugin from H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\connectors
2026-08-14 12:01:01,072 WARN  [o.a.s.a.c.ReadonlyConfig      ] [main] - Please use the new key 'plugin_input' instead of the deprecated key 'source_table_name'.
2026-08-14 12:01:01,072 WARN  [o.a.s.a.c.u.ConfigUtil        ] [main] - Option 'plugin_input' is a List, and it is recommended to configure it as ["string1","string2"]; we will only use ',' to split the String into a list.
2026-08-14 12:01:01,072 WARN  [o.a.s.a.c.ReadonlyConfig      ] [main] - Please use the new key 'plugin_output' instead of the deprecated key 'result_table_name'.
2026-08-14 12:01:01,086 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sinks.
2026-08-14 12:01:01,088 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from H:\Local\bigdata-develop\seatunnel-research\apache-seatunnel-2.3.13\connectors
2026-08-14 12:01:01,089 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Console'} at: [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-console-2.3.13.jar]
2026-08-14 12:01:01,090 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Console'}: [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-console-2.3.13.jar]
2026-08-14 12:01:01,101 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Console' with upstream input catalog-table[database: null, schema: null, table: fake]
2026-08-14 12:01:01,155 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Start submit job, job id: 1140495265673773057, with plugin jar [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-fake-2.3.13.jar, file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-console-2.3.13.jar]
2026-08-14 12:01:01,165 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - Init ResourceManager
2026-08-14 12:01:01,166 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - initWorker...
2026-08-14 12:01:01,166 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - init live nodes: [[localhost]:5801]
2026-08-14 12:01:01,167 INFO  [.e.s.r.AbstractResourceManager] [SeaTunnel-CompletableFuture-Thread-2] - received new worker register: [localhost]:5801
2026-08-14 12:01:01,167 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=477626368}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=477626368}}, dynamicSlot=true, assignedSlots=[], unassignedSlots=[], attributes={}, systemLoadInfo=null)}
2026-08-14 12:01:01,183 INFO  [o.a.s.e.s.m.JobMaster         ] [seatunnel-coordinator-service-2] - Init JobMaster for Job SeaTunnel_Job (1140495265673773057)
2026-08-14 12:01:01,184 INFO  [o.a.s.e.s.m.JobMaster         ] [seatunnel-coordinator-service-2] - Job SeaTunnel_Job (1140495265673773057) needed jar urls [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-fake-2.3.13.jar, file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-console-2.3.13.jar]
2026-08-14 12:01:01,184 INFO  [.c.c.DefaultClassLoaderService] [seatunnel-coordinator-service-2] - Create classloader for job 1 with jars [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-fake-2.3.13.jar]
2026-08-14 12:01:01,185 INFO  [.c.c.DefaultClassLoaderService] [seatunnel-coordinator-service-2] - Create classloader for job 1 with jars []
2026-08-14 12:01:01,185 INFO  [.c.c.DefaultClassLoaderService] [seatunnel-coordinator-service-2] - Create classloader for job 1 with jars [file:/H:/Local/bigdata-develop/seatunnel-research/apache-seatunnel-2.3.13/connectors/connector-console-2.3.13.jar]
2026-08-14 12:01:01,275 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-2] - Create CheckpointCoordinator for job(1@1140495265673773057) with plan(CheckpointPlan(pipelineId=1, pipelineSubtasks=[TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}, taskID=1000300000001, index=1}, TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}, taskID=1000300010001, index=1}, TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}, taskID=1000200000000, index=0}, TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}, taskID=1000200010000, index=0}, TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}, taskID=1000100000000, index=0}], startingSubtasks=[TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}, taskID=1000100000000, index=0}], pipelineActions={ActionStateKey(name=ActionStateKey - pipeline-1 [TransformChain[Transform[0]-FieldMapper]])=2, ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-FakeSource])=2, ActionStateKey(name=ActionStateKey - pipeline-1 [Sink[0]-Console-MultiTableSink])=2}, subtaskActions={TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}, taskID=1000300000001, index=1}=[(ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-FakeSource]), 1), (ActionStateKey(name=ActionStateKey - pipeline-1 [TransformChain[Transform[0]-FieldMapper]]), 1)], TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}, taskID=1000300010001, index=1}=[(ActionStateKey(name=ActionStateKey - pipeline-1 [Sink[0]-Console-MultiTableSink]), 1)], TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}, taskID=1000200000000, index=0}=[(ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-FakeSource]), 0), (ActionStateKey(name=ActionStateKey - pipeline-1 [TransformChain[Transform[0]-FieldMapper]]), 0)], TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}, taskID=1000200010000, index=0}=[(ActionStateKey(name=ActionStateKey - pipeline-1 [Sink[0]-Console-MultiTableSink]), 0)], TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}, taskID=1000100000000, index=0}=[(ActionStateKey(name=ActionStateKey - pipeline-1 [Source[0]-FakeSource]), -1)]}))
2026-08-14 12:01:01,276 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-2] - The task Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (1/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] is in state CREATED when init state future
2026-08-14 12:01:01,277 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-2] - The task Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (2/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] is in state CREATED when init state future
2026-08-14 12:01:01,277 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-2] - The task Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] is in state CREATED when init state future
2026-08-14 12:01:01,281 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Submit job finished, job id: 1140495265673773057, job name: SeaTunnel_Job
2026-08-14 12:01:01,282 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-2] - Job SeaTunnel_Job (1140495265673773057) turned from state CREATED to PENDING.
2026-08-14 12:01:01,282 WARN  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-2] - Job SeaTunnel_Job (1140495265673773057) state process is stopped
2026-08-14 12:01:01,283 INFO  [o.a.s.e.s.CoordinatorService  ] [seatunnel-coordinator-service-2] - [localhost]:5801 [seatunnel-813134] [5.1] The submit job enter the pending queue , jobId: 1140495265673773057 , jobName: SeaTunnel_Job
2026-08-14 12:01:01,290 INFO  [o.a.s.e.s.CoordinatorService  ] [pending-job-schedule-runner] - [localhost]:5801 [seatunnel-813134] [5.1] Resources enough, start running: 1140495265673773057
2026-08-14 12:01:01,292 INFO  [o.a.s.e.s.CoordinatorService  ] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] The SUBMIT Job SeaTunnel_Job (1140495265673773057) is in PENDING state, restore pipeline and take over this job running
2026-08-14 12:01:01,293 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-3] - Job SeaTunnel_Job (1140495265673773057) state process is start
2026-08-14 12:01:01,294 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-3] - Job SeaTunnel_Job (1140495265673773057) turned from state PENDING to SCHEDULED.
2026-08-14 12:01:01,295 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-3] - Job SeaTunnel_Job (1140495265673773057), Pipeline: [(1/1)] state process is start
2026-08-14 12:01:01,296 WARN  [o.a.s.e.c.j.JobMetricsRunner  ] [job-metrics-runner-1140495265673773057] - Failed to get job metrics summary, it maybe first-run
2026-08-14 12:01:01,298 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-3] - Job SeaTunnel_Job (1140495265673773057), Pipeline: [(1/1)] turned from state CREATED to SCHEDULED.
2026-08-14 12:01:01,308 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-3] - Resource allocation for pipeline Job SeaTunnel_Job (1140495265673773057), Pipeline: [(1/1)] completed. Task execution locations:
2026-08-14 12:01:01,308 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-3] -   Task [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] will be executed on worker [[localhost]:5801], slotID [1], resourceProfile [ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}], sequence [77664dbf-04c9-442a-bdf5-2029eabff181], assigned [1140495265673773057]
2026-08-14 12:01:01,308 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-3] -   Task [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] will be executed on worker [[localhost]:5801], slotID [2], resourceProfile [ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}], sequence [77664dbf-04c9-442a-bdf5-2029eabff181], assigned [1140495265673773057]
2026-08-14 12:01:01,308 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-3] -   Task [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] will be executed on worker [[localhost]:5801], slotID [3], resourceProfile [ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}], sequence [77664dbf-04c9-442a-bdf5-2029eabff181], assigned [1140495265673773057]
2026-08-14 12:01:01,310 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-3] - Job SeaTunnel_Job (1140495265673773057), Pipeline: [(1/1)] turned from state SCHEDULED to DEPLOYING.
2026-08-14 12:01:01,310 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] state process is start
2026-08-14 12:01:01,312 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] turned from state CREATED to DEPLOYING.
2026-08-14 12:01:01,323 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] received deploying task executionId [1140495268223909889]
2026-08-14 12:01:01,328 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] deploying task TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}, executionId [1140495268223909889]
2026-08-14 12:01:01,329 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] deploying TaskGroup TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1} init success
2026-08-14 12:01:01,335 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] deploying TaskGroup TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1} success
2026-08-14 12:01:01,336 INFO  [.s.t.SourceSplitEnumeratorTask] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] - starting seatunnel source split enumerator task, source name: pipeline-1 [Source[0]-FakeSource]
2026-08-14 12:01:01,337 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] turned from state DEPLOYING to RUNNING.
2026-08-14 12:01:01,338 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] current state equals target state: RUNNING, skip
2026-08-14 12:01:01,338 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (1/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] state process is start
2026-08-14 12:01:01,339 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (1/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] turned from state CREATED to DEPLOYING.
2026-08-14 12:01:01,354 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-25] - log event: EnumeratorOpenEvent(createdTime=1786680061352, jobId=1140495265673773057, eventType=LIFECYCLE_ENUMERATOR_OPEN)
2026-08-14 12:01:01,360 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] received deploying task executionId [1140495268223975425]
2026-08-14 12:01:01,368 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] deploying task TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}, executionId [1140495268223975425]
2026-08-14 12:01:01,369 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] deploying TaskGroup TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2} init success
2026-08-14 12:01:01,371 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] deploying TaskGroup TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2} success
2026-08-14 12:01:01,373 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (1/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] turned from state DEPLOYING to RUNNING.
2026-08-14 12:01:01,373 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (1/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] current state equals target state: RUNNING, skip
2026-08-14 12:01:01,373 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (2/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] state process is start
2026-08-14 12:01:01,379 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (2/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] turned from state CREATED to DEPLOYING.
2026-08-14 12:01:01,381 INFO  [s.e.s.t.TransformSeaTunnelTask] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] - starting seatunnel transform task, index 0
2026-08-14 12:01:01,383 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] received deploying task executionId [1140495268224040961]
2026-08-14 12:01:01,389 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] deploying task TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}, executionId [1140495268224040961]
2026-08-14 12:01:01,389 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] deploying TaskGroup TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3} init success
2026-08-14 12:01:01,393 INFO  [o.a.s.e.s.TaskExecutionService] [seatunnel-coordinator-service-3] - [localhost]:5801 [seatunnel-813134] [5.1] deploying TaskGroup TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3} success
2026-08-14 12:01:01,394 INFO  [s.e.s.t.TransformSeaTunnelTask] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] - starting seatunnel transform task, index 1
2026-08-14 12:01:01,395 INFO  [.a.s.e.s.t.SourceSeaTunnelTask] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] - starting seatunnel source task, index 1
2026-08-14 12:01:01,395 INFO  [.a.s.e.s.t.SourceSeaTunnelTask] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] - starting seatunnel source task, index 0
2026-08-14 12:01:01,397 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (2/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] turned from state DEPLOYING to RUNNING.
2026-08-14 12:01:01,398 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [seatunnel-coordinator-service-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (2/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] current state equals target state: RUNNING, skip
2026-08-14 12:01:01,400 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-3] - Job SeaTunnel_Job (1140495265673773057), Pipeline: [(1/1)] turned from state DEPLOYING to RUNNING.
2026-08-14 12:01:01,402 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-3] - Job SeaTunnel_Job (1140495265673773057) turned from state SCHEDULED to RUNNING.
2026-08-14 12:01:01,410 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [hz.main.seaTunnel.task.thread-7] - output rowType: age<INT>, name2<STRING>
2026-08-14 12:01:01,410 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [hz.main.seaTunnel.task.thread-6] - output rowType: age<INT>, name2<STRING>
2026-08-14 12:01:01,416 INFO  [a.s.a.s.m.MultiTableSinkWriter] [hz.main.seaTunnel.task.thread-7] - init multi table sink writer, queue size: 1
2026-08-14 12:01:01,417 INFO  [a.s.a.s.m.MultiTableSinkWriter] [hz.main.seaTunnel.task.thread-6] - init multi table sink writer, queue size: 1
2026-08-14 12:01:01,508 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-21] - log event: ReaderOpenEvent(createdTime=1786680061507, jobId=1140495265673773057, eventType=LIFECYCLE_READER_OPEN)
2026-08-14 12:01:01,508 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-21] - log event: ReaderOpenEvent(createdTime=1786680061507, jobId=1140495265673773057, eventType=LIFECYCLE_READER_OPEN)
2026-08-14 12:01:01,510 INFO  [.s.t.SourceSplitEnumeratorTask] [hz.main.seaTunnel.task.thread-7] - received reader register, readerID: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}, taskID=1000200000000, index=0}
2026-08-14 12:01:01,510 INFO  [.s.t.SourceSplitEnumeratorTask] [hz.main.seaTunnel.task.thread-6] - received reader register, readerID: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}, taskID=1000300000001, index=1}
2026-08-14 12:01:01,560 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-7] - checkpoint is enabled, start schedule trigger pending checkpoint.
2026-08-14 12:01:01,617 INFO  [o.a.s.c.s.f.s.FakeSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] - wait split!
2026-08-14 12:01:01,617 INFO  [o.a.s.c.s.f.s.FakeSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] - wait split!
2026-08-14 12:01:01,665 INFO  [.s.t.SourceSplitEnumeratorTask] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] - received enough reader, starting enumerator...
2026-08-14 12:01:01,665 INFO  [.f.s.FakeSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] - Starting to calculate splits.
2026-08-14 12:01:01,668 INFO  [.f.s.FakeSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] - Calculated splits for table fake successfully, the size of splits is 2.
2026-08-14 12:01:01,669 INFO  [.f.s.FakeSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] - Assigned [FakeSourceSplit(tableId=fake, splitId=1, rowNum=16), FakeSourceSplit(tableId=fake, splitId=0, rowNum=16)] to 2 readers.
2026-08-14 12:01:01,670 INFO  [.f.s.FakeSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] - Calculated splits successfully, the size of splits is 2.
2026-08-14 12:01:01,671 INFO  [.f.s.FakeSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] - Assigning splits to readers 0 [FakeSourceSplit(tableId=fake, splitId=0, rowNum=16)]
2026-08-14 12:01:01,682 INFO  [.f.s.FakeSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] - Assigning splits to readers 1 [FakeSourceSplit(tableId=fake, splitId=1, rowNum=16)]
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=1:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1755694728, RcAmm
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=1:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1870709191, INCEm
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=2:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1428186829, zmebn
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=2:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1765100536, NeseY
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=3:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1207757873, eUsaE
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=3:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 831705485, xcuRk
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=4:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 2032854171, GDuWV
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=4:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 130556650, Ugcsh
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=5:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 77115810, uWAuu
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=5:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1055821179, aSZBD
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=6:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 2093674794, uEEQh
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=7:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 2135310066, OcvXC
2026-08-14 12:01:02,655 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=6:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 117163797, GfrFT
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=8:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1055063622, jvYjQ
2026-08-14 12:01:02,656 INFO  [o.a.s.c.s.f.s.FakeSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] - 16 rows of data have been generated in split(fake_1) for table fake. Generation time: 1786680062630
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=9:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1349836289, vlVcO
2026-08-14 12:01:02,656 INFO  [o.a.s.c.s.f.s.FakeSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] - 16 rows of data have been generated in split(fake_0) for table fake. Generation time: 1786680062630
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=7:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1682831313, TrxUr
2026-08-14 12:01:02,656 INFO  [o.a.s.c.s.f.s.FakeSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] - Closed the bounded fake source
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=10:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 167512193, BWxaW
2026-08-14 12:01:02,656 INFO  [o.a.s.c.s.f.s.FakeSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] - Closed the bounded fake source
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=8:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1508427260, HZPtk
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=11:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1719078349, SKUPg
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=9:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 978361042, pRTaQ
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=12:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1368664289, AErTh
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=10:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 8887186, JpTaO
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=13:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1610756312, GtXMm
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=11:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1317411056, USltu
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=12:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1379976106, fvbQF
2026-08-14 12:01:02,656 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=14:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1995660815, iChCr
2026-08-14 12:01:02,657 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=13:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1020387821, PJiPm
2026-08-14 12:01:02,657 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=15:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 2136780512, syaHQ
2026-08-14 12:01:02,657 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=14:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1364038694, iUTwZ
2026-08-14 12:01:02,657 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=1  rowIndex=16:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1661859757, cgFAi
2026-08-14 12:01:02,657 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=15:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1153010304, qVEoq
2026-08-14 12:01:02,657 INFO  [.a.s.c.s.c.s.ConsoleSinkWriter] [st-multi-table-sink-writer-1] - subtaskIndex=0  rowIndex=16:  SeaTunnelRow#tableId=fake SeaTunnelRow#kind=INSERT : 1067227184, tgyic
2026-08-14 12:01:02,681 INFO  [.s.e.s.c.CheckpointCoordinator] [hz.main.generic-operation.thread-37] - skip schedule trigger checkpoint because checkpoint type is COMPLETED_POINT_TYPE
2026-08-14 12:01:02,697 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-9] - wait checkpoint completed: 9223372036854775807
2026-08-14 12:01:02,729 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-9] - pending checkpoint(9223372036854775807/1@1140495265673773057) notify finished!
2026-08-14 12:01:02,738 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-9] - start notify checkpoint completed, job id: 1140495265673773057, pipeline id: 1, checkpoint id:9223372036854775807
2026-08-14 12:01:02,747 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-9] - start clean pending checkpoint cause CheckpointCoordinator completed.
2026-08-14 12:01:02,750 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-9] - Turn checkpoint_state_1140495265673773057_1 state from null to FINISHED
2026-08-14 12:01:02,788 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] - [localhost]:5801 [seatunnel-813134] [5.1] taskDone, taskId = 1000100000000, taskGroup = TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}
2026-08-14 12:01:02,788 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-4] - log event: EnumeratorCloseEvent(createdTime=1786680062788, jobId=1140495265673773057, eventType=LIFECYCLE_ENUMERATOR_CLOSE)
2026-08-14 12:01:02,793 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] - [localhost]:5801 [seatunnel-813134] [5.1] taskGroup TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1} complete with FINISHED
2026-08-14 12:01:02,793 INFO  [o.a.s.e.s.TaskExecutionService] [hz.main.seaTunnel.task.thread-6] - [localhost]:5801 [seatunnel-813134] [5.1] Task TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1} complete with state FINISHED
2026-08-14 12:01:02,793 INFO  [o.a.s.e.s.CoordinatorService  ] [hz.main.seaTunnel.task.thread-6] - [localhost]:5801 [seatunnel-813134] [5.1] Received task end from execution TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}, state FINISHED
2026-08-14 12:01:02,796 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [hz.main.seaTunnel.task.thread-6] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] turned from state RUNNING to FINISHED.
2026-08-14 12:01:02,796 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [hz.main.seaTunnel.task.thread-6] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] state process is stopped
2026-08-14 12:01:02,796 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-9] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=1}] future complete with state FINISHED
2026-08-14 12:01:02,838 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] - [localhost]:5801 [seatunnel-813134] [5.1] taskDone, taskId = 1000200010000, taskGroup = TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}
2026-08-14 12:01:02,838 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] - [localhost]:5801 [seatunnel-813134] [5.1] taskDone, taskId = 1000300010001, taskGroup = TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}
2026-08-14 12:01:02,838 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-2] - log event: WriterCloseEvent(createdTime=1786680062838, jobId=1140495265673773057, eventType=LIFECYCLE_WRITER_CLOSE)
2026-08-14 12:01:02,838 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-2] - log event: WriterCloseEvent(createdTime=1786680062838, jobId=1140495265673773057, eventType=LIFECYCLE_WRITER_CLOSE)
2026-08-14 12:01:03,672 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] - [localhost]:5801 [seatunnel-813134] [5.1] taskDone, taskId = 1000200000000, taskGroup = TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}
2026-08-14 12:01:03,672 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] - [localhost]:5801 [seatunnel-813134] [5.1] taskDone, taskId = 1000300000001, taskGroup = TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}
2026-08-14 12:01:03,672 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-15] - log event: ReaderCloseEvent(createdTime=1786680063671, jobId=1140495265673773057, eventType=LIFECYCLE_READER_CLOSE)
2026-08-14 12:01:03,672 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-15] - log event: ReaderCloseEvent(createdTime=1786680063671, jobId=1140495265673773057, eventType=LIFECYCLE_READER_CLOSE)
2026-08-14 12:01:03,678 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] - [localhost]:5801 [seatunnel-813134] [5.1] taskGroup TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2} complete with FINISHED
2026-08-14 12:01:03,678 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] - [localhost]:5801 [seatunnel-813134] [5.1] taskGroup TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3} complete with FINISHED
2026-08-14 12:01:03,678 INFO  [o.a.s.e.s.TaskExecutionService] [hz.main.seaTunnel.task.thread-5] - [localhost]:5801 [seatunnel-813134] [5.1] Task TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2} complete with state FINISHED
2026-08-14 12:01:03,678 INFO  [o.a.s.e.s.TaskExecutionService] [hz.main.seaTunnel.task.thread-3] - [localhost]:5801 [seatunnel-813134] [5.1] Task TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3} complete with state FINISHED
2026-08-14 12:01:03,679 INFO  [o.a.s.e.s.CoordinatorService  ] [hz.main.seaTunnel.task.thread-5] - [localhost]:5801 [seatunnel-813134] [5.1] Received task end from execution TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}, state FINISHED
2026-08-14 12:01:03,679 INFO  [o.a.s.e.s.CoordinatorService  ] [hz.main.seaTunnel.task.thread-3] - [localhost]:5801 [seatunnel-813134] [5.1] Received task end from execution TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}, state FINISHED
2026-08-14 12:01:03,682 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [hz.main.seaTunnel.task.thread-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (2/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] turned from state RUNNING to FINISHED.
2026-08-14 12:01:03,682 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [hz.main.seaTunnel.task.thread-5] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (1/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] turned from state RUNNING to FINISHED.
2026-08-14 12:01:03,682 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [hz.main.seaTunnel.task.thread-3] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (2/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] state process is stopped
2026-08-14 12:01:03,682 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [hz.main.seaTunnel.task.thread-5] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (1/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] state process is stopped
2026-08-14 12:01:03,682 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-9] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (2/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}] future complete with state FINISHED
2026-08-14 12:01:03,682 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-8] - Job (1140495265673773057), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource]-SourceTask (1/2)], taskGroupLocation: [TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}] future complete with state FINISHED
2026-08-14 12:01:03,682 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-8] - Job SeaTunnel_Job (1140495265673773057), Pipeline: [(1/1)] will end with state FINISHED
2026-08-14 12:01:03,683 INFO  [o.a.s.e.s.m.JobMaster         ] [hz.main.seaTunnel.task.thread-5] - release the task group resource TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=2}
2026-08-14 12:01:03,683 INFO  [o.a.s.e.s.m.JobMaster         ] [hz.main.seaTunnel.task.thread-3] - release the task group resource TaskGroupLocation{jobId=1140495265673773057, pipelineId=1, taskGroupId=3}
2026-08-14 12:01:03,684 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-8] - Job SeaTunnel_Job (1140495265673773057), Pipeline: [(1/1)] turned from state RUNNING to FINISHED.
2026-08-14 12:01:03,684 INFO  [a.s.e.s.s.s.DefaultSlotService] [hz.main.generic-operation.thread-3] - received slot release request, jobID: 1140495265673773057, slot: SlotProfile{worker=[localhost]:5801, slotID=2, ownerJobID=1140495265673773057, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='77664dbf-04c9-442a-bdf5-2029eabff181'}
2026-08-14 12:01:03,685 INFO  [a.s.e.s.s.s.DefaultSlotService] [hz.main.generic-operation.thread-20] - received slot release request, jobID: 1140495265673773057, slot: SlotProfile{worker=[localhost]:5801, slotID=3, ownerJobID=1140495265673773057, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='77664dbf-04c9-442a-bdf5-2029eabff181'}
2026-08-14 12:01:03,884 INFO  [o.a.s.e.s.m.JobMaster         ] [seatunnel-coordinator-service-8] - release the pipeline Job SeaTunnel_Job (1140495265673773057), Pipeline: [(1/1)] resource
2026-08-14 12:01:03,885 INFO  [a.s.e.s.s.s.DefaultSlotService] [hz.main.generic-operation.thread-12] - received slot release request, jobID: 1140495265673773057, slot: SlotProfile{worker=[localhost]:5801, slotID=1, ownerJobID=1140495265673773057, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='77664dbf-04c9-442a-bdf5-2029eabff181'}
2026-08-14 12:01:03,886 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-8] - Job SeaTunnel_Job (1140495265673773057), Pipeline: [(1/1)] state process is stop
2026-08-14 12:01:03,886 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-9] - Job SeaTunnel_Job (1140495265673773057), Pipeline: [(1/1)] future complete with state FINISHED
2026-08-14 12:01:03,888 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-9] - Job SeaTunnel_Job (1140495265673773057) turned from state RUNNING to FINISHED.
2026-08-14 12:01:03,888 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-9] - Job SeaTunnel_Job (1140495265673773057) state process is stop
2026-08-14 12:01:03,905 INFO  [o.a.s.a.e.LoggingEventHandler ] [seatunnel-coordinator-service-9] - log event: JobStateEvent(jobId=1140495265673773057, jobName=SeaTunnel_Job, jobStatus=FINISHED, createdTime=1786680063905)
2026-08-14 12:01:03,905 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Job (1140495265673773057) end with state FINISHED
2026-08-14 12:01:03,953 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] -
***********************************************
           Job Statistic Information
***********************************************
Start Time                : 2026-08-14 12:01:00
End Time                  : 2026-08-14 12:01:03
Total Time(s)             :                   3
Total Read Count          :                  32
Total Write Count         :                  32
Total Failed Count        :                   0
***********************************************

2026-08-14 12:01:03,954 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-813134] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2026-08-14 12:01:03,958 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-813134] [5.1] Removed connection to endpoint: [localhost]:5801:bcd4cf41-9ce4-4509-8631-7d178b381cec, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:45457->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2026-08-14 12:01:03.945, lastWriteTime=2026-08-14 12:01:03.905, closedTime=2026-08-14 12:01:03.955, connected server version=5.1}
2026-08-14 12:01:03,958 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-813134] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2026-08-14 12:01:03,959 INFO  [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-0] - [localhost]:5801 [seatunnel-813134] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:45457, qualifier=null, endpoint=[127.0.0.1]:45457, remoteUuid=cfdd5a2e-ab9f-4a42-9654-ce8e218e290c, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
2026-08-14 12:01:03,963 INFO  [c.h.c.i.ClientEndpointManager ] [hz.main.event-3] - [localhost]:5801 [seatunnel-813134] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:45457, qualifier=null, endpoint=[127.0.0.1]:45457, remoteUuid=cfdd5a2e-ab9f-4a42-9654-ce8e218e290c, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=cfdd5a2e-ab9f-4a42-9654-ce8e218e290c, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1786680060611, latest clientAttributes=lastStatisticsCollectionTime=1786680060650,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1786680060597,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=534577152,os.freePhysicalMemorySize=4595171328,os.freeSwapSpaceSize=6503546880,os.maxFileDescriptorCount=0,os.openFileDescriptorCount=0,os.processCpuTime=15984375000,os.systemLoadAverage=-1.0,os.totalPhysicalMemorySize=17054289920,os.totalSwapSpaceSize=32086675456,runtime.availableProcessors=8,runtime.freeMemory=187145312,runtime.maxMemory=477626368,runtime.totalMemory=324534272,runtime.uptime=37996,runtime.usedMemory=137388960, labels=[]}
2026-08-14 12:01:03,964 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-813134] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2026-08-14 12:01:03,964 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
2026-08-14 12:01:03,964 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5801 is SHUTTING_DOWN
2026-08-14 12:01:03,969 INFO  [c.h.i.p.i.MigrationManager    ] [hz.main.cached.thread-10] - [localhost]:5801 [seatunnel-813134] [5.1] Shutdown request of Member [localhost]:5801 - bcd4cf41-9ce4-4509-8631-7d178b381cec [master node] [active master] this is handled
2026-08-14 12:01:03,976 INFO  [c.h.i.i.Node                  ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Shutting down connection manager...
2026-08-14 12:01:03,979 INFO  [c.h.i.i.Node                  ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Shutting down node engine...
2026-08-14 12:01:03,986 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-813134] [5.1] This node become leave active master node, begin clear coordinator service
2026-08-14 12:01:04,000 INFO  [.s.s.o.e.j.s.AbstractConnector] [main] - Stopped ServerConnector@2eed37f4{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2026-08-14 12:01:04,001 INFO  [o.a.s.s.o.e.j.s.session       ] [main] - node0 Stopped scavenging
2026-08-14 12:01:04,002 INFO  [a.s.s.o.e.j.s.h.ContextHandler] [main] - Stopped o.a.s.s.o.e.j.s.ServletContextHandler@30db5536{/,null,STOPPED}
2026-08-14 12:01:04,006 INFO  [.c.c.DefaultClassLoaderService] [main] - close classloader service
2026-08-14 12:01:04,006 INFO  [o.a.s.e.s.EventService        ] [event-forwarder-0] - Event forward thread interrupted
2026-08-14 12:01:07,017 INFO  [c.h.i.i.NodeExtension         ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Destroying node NodeExtension.
2026-08-14 12:01:07,018 INFO  [c.h.i.i.Node                  ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] Hazelcast Shutdown is completed in 3050 ms.
2026-08-14 12:01:07,019 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-813134] [5.1] [localhost]:5801 is SHUTDOWN
2026-08-14 12:01:07,019 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed HazelcastInstance ......
2026-08-14 12:01:07,019 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed metrics executor service ......
2026-08-14 12:01:09,167 INFO  [s.c.s.s.c.ClientExecuteCommand] [SeaTunnel-CompletableFuture-Thread-7] - run shutdown hook because get close signal

CASE 最简示例(流式处理)

step0 安装 + 构建所需的connectors （略）

step1 配置流式的数据同步任务

编辑/配置config/my.v2.streaming.conf

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
######
###### This config file is a demonstration of streaming processing in SeaTunnel config
######

env {
 # You can set SeaTunnel environment configuration here
 # 为避免报错: Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.IllegalArgumentException: A single split source allows only one single reader to be created. Please make sure source parallelism = 1（原因：org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.IllegalArgumentException: 单个分片源仅允许创建一个读取器。请确保源并行度 = 1），必须设置为1
  parallelism = 1
  job.mode = "STREAMING"
  checkpoint.interval = 2000
}

source {
  Socket {
    host = "localhost"
    port = 9999

    schema = {
      fields {
        name = "string"
        age = "int"
      }
    }
  }
}

sink {
  Console {
  }

 # If you would like to get more information about how to configure SeaTunnel and see full list of sink plugins,
 # please go to https://seatunnel.apache.org/docs/connector-v2/sink
}

step2 基于Netcat 创建 Socket 流式数据输入环境

C:\Users\Xxx> nc -L -p 9999 -v
listening on [any] 9999 ...

等流式作业启动完成后，此窗口方可自由输入数据。（每行数据通过 Enter 键间隔）

step3 启动流式同步作业

启动流式同步作业

 ./bin/seatunnel.cmd --config ./config./my.v2.streaming.conf -m local

在 netcat 窗口输入数据，即可查看效果

sea-tunnel-streaming-demo-from-netcat-socket-to-console.20260814123019623

CASE 作业提交命令

# Zeta Engine 本地模式
./bin/seatunnel.sh --config ./config/my-job.conf -m local

# Zeta Engine 集群模式
./bin/seatunnel.sh --config ./config/my-job.conf -m cluster

# Flink 引擎
./bin/start-seatunnel-flink-connector-v2.sh --config ./config/my-job.conf

# Spark 引擎
./bin/start-seatunnel-spark-connector-v2.sh --config ./config/my-job.conf

CASE Sea Tunnel 与 Flink 的集成

未亲测，此处仅展示大致思路。

也可直接下载、查看 Github 仓库源码: seatunnel-examples/seatunnel-flink-connector-v2-example，示例入口类：

org.apache.seatunnel.example.flink.v2.SeaTunnelApiExample

在 SeaTunnel 中，Flink 被用作流处理引擎。以下是基于 Flink 的 SeaTunnel 配置和运行的关键步骤

推荐文献
- https://seatunnel.apache.org/zh-CN/docs/engines/flink/
- https://github.com/apache/seatunnel/tree/dev/seatunnel-examples/seatunnel-flink-examples
  - https://github.com/apache/seatunnel/tree/dev/seatunnel-examples/seatunnel-flink-examples/seatunnel-flink-15-example/src/main/java/org/apache/seatunnel/example/flink

配置文件

SeaTunnel 作业里的 Flink 专属配置，需要写在 env 块中，并使用 flink. 前缀。

env {
  parallelism = 1
  checkpoint.interval = 5000

  flink.execution.checkpointing.mode = "EXACTLY_ONCE"
  flink.execution.checkpointing.timeout = 600000
 # flink.execution.checkpointing.unaligned.enabled = true
}

source {
  FakeSource {
    row.num = 16
    plugin_output = "fake_table"
    schema = {
      fields {
        c_map = "map<string, string>"
        c_array = "array<int>"
        c_string = string
        c_boolean = boolean
        c_int = int
        c_bigint = bigint
        c_double = double
        c_bytes = bytes
        c_date = date
        c_decimal = "decimal(33, 18)"
        c_timestamp = timestamp
      }
    }
  }
}

transform {
  FieldMapper {
    plugin_input = "fake_table"
    plugin_output = "fake_output"
    field_mapper = {
      c_string = c_string
      c_int = c_int
    }
  }
}

sink {
  Console {
    plugin_input = "fake_output"
  }
}

某些枚举类配置项暂时不适合直接内联在 SeaTunnel 作业配置里，遇到这类参数时，建议放到 Flink 自身配置中。当前常见的内联支持类型主要包括：

Integer

Boolean

String

Duration

如果你需要更多 transform 能力，继续查看 Transforms 目录和 Transform 通用参数。

通过上述配置文件，SeaTunnel 定义了数据源（FakeSource）和目标（Console），并设置了并行度和数据模式。

Flink 作业程序中集成 Sea Tunnel

seatunnel/seatunnel-examples/seatunnel-flink-examples/seatunnel-flink-15-example/pom.xml at dev · apache/seatunnel
<dependency>
    <groupId>org.apache.seatunnel</groupId>
    <artifactId>seatunnel-flink-15-starter</artifactId>
    <version>${project.version}</version>
</dependency>
https://github.com/apache/seatunnel/blob/dev/seatunnel-examples/seatunnel-flink-examples/seatunnel-flink-15-example/src/main/java/org/apache/seatunnel/example/flink/SeaTunnelStreamingJobExample.java

/*
 *  Licensed to the Apache Software Foundation (ASF) under one or more
 *  contributor license agreements.  See the NOTICE file distributed with
 *  this work for additional information regarding copyright ownership.
 *  The ASF licenses this file to You under the Apache License, Version 2.0
 *  (the "License"); you may not use this file except in compliance with
 *  the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 *  Unless required by applicable law or agreed to in writing, software
 *  distributed under the License is distributed on an "AS IS" BASIS,
 *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *  See the License for the specific language governing permissions and
 *  limitations under the License.
 */

package org.apache.seatunnel.example.flink;

import org.apache.seatunnel.core.starter.SeaTunnel;
import org.apache.seatunnel.core.starter.exception.CommandException;
import org.apache.seatunnel.core.starter.flink.args.FlinkCommandArgs;

import java.io.FileNotFoundException;
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.Paths;

public class SeaTunnelStreamingJobExample {

    public static void main(String[] args)
            throws FileNotFoundException, URISyntaxException, CommandException {
        String configurePath =
                args.length > 0 ? args[0] : "/examples/fake_to_console_streaming.conf";
        String configFile = getTestConfigFile(configurePath);
        FlinkCommandArgs flinkCommandArgs = new FlinkCommandArgs();
        flinkCommandArgs.setConfigFile(configFile);
        flinkCommandArgs.setCheckConfig(false);
        flinkCommandArgs.setVariables(null);
        SeaTunnel.run(flinkCommandArgs.buildCommand());
    }

    public static String getTestConfigFile(String configFile)
            throws FileNotFoundException, URISyntaxException {
        URL resource = SeaTunnelStreamingJobExample.class.getResource(configFile);
        if (resource == null) {
            throw new FileNotFoundException("Can't find config file: " + configFile);
        }
        return Paths.get(resource.toURI()).toString();
    }
}

CASE SeaTunnel + Web + Docker实现: Kafka 同步到 ES/Hive

SeaTunnel 2.3.11 + Web 1.0.3 Docker 部署实战：Kafka 同步 Hive/ES 完整指南 - 博客园

seatunnel-web-implements-kafka-sync-to-hive-and-es.202608141312

Z FAQ for Apache Sea Tunnel

Q: 是否需要 Flink 或 Spark？

不需要。Zeta Engine 是内置的自研引擎，开箱即用。Flink 和 Spark 是可选引擎。

Q: Windows 能否部署？

可以。2.3.x 版本提供了 .bat 启动脚本，支持 Windows 运行。

Q: SeaTunnel 与 Flink CDC 区别？

SeaTunnel CDC 可以在 Zeta/Flink/Spark 三个引擎上运行，Flink CDC 需要 Flink 环境。

Q: 支持 Exactly-Once 吗？

支持。通过两阶段提交和 Checkpoint 机制实现。

Q: Apache Sea Tunnel的数据集成Job的开发、配置，是通过什么方式？不支持从Web UI提交，Web UI仅提供作业的监控/查询

SeaTunnel 的 Job 开发与配置主要有 4 种方式，按使用频率排序：

1. 配置文件方式（最主流）

用 .conf（HOCON 格式）文件描述一个完整 Job，通过客户端提交。文件放在 config/ 下，由四大部分组成：

env {
  parallelism = 2
  job.mode = "BATCH"
  checkpoint.interval = 10000
}

source {
  Jdbc {
    url = "jdbc:mysql://localhost:3306/db"
    driver = "com.mysql.cj.jdbc.Driver"
    user = "root"
    password = "123456"
    query = "select * from user"
  }
}

transform {
  FieldMapper { field_mapper = { old = new } }
}

sink {
  Jdbc {
    url = "jdbc:postgresql://localhost:5432/db"
    ...
  }
}

2. 客户端命令行提交

配置写好后用 seatunnel.sh 提交：

bin/seatunnel.sh --config config/job.conf \
  -e localhost:5801 \
  --name my_job

-e 指定集群 master 地址（默认 5801）
--name 指定 Job 名称（不传则用文件名）

3. REST API 提交（脚本化/平台集成用）

SeaTunnel Engine 提供 REST API（默认 8080，需在 seatunnel.yaml 开 enable-http），支持：

POST /hazelcast/rest/maps/submit-job —— 提交配置文本
POST /hazelcast/rest/maps/submit-job-file —— 提交配置文件（multipart）
查询/停止 Job 也有对应接口，适合接调度平台

curl -X POST http://master:8080/hazelcast/rest/maps/submit-job \
  -H "Content-Type: application/json" \
  -d @job.json

4. Java SDK / Engine Client（程序化嵌入）

引入 seatunnel-engine-client 依赖，在 Java/Spring 里直接构建 Job 并提交：

EngineClient engineClient = new EngineClient();
SubmitJobRequest req = ...;
engineClient.submitJob(req);

适合把 SeaTunnel 嵌进自己的平台/数据中台。

怎么选

场景	推荐方式
一次性/临时跑数	配置文件 + `seatunnel.sh`
上线到调度平台（Airflow/DolphinScheduler/自研）	REST API 或客户端命令
平台内嵌、二次开发	Java SDK
复杂转换逻辑	配置内 `transform` 或写自定义 Transform 插件

补充：新版本（2.3.x 后期起）也在演进 SeaTunnel SQL / 动态脚本能力，支持用 SQL 定义任务，但目前生产上最稳、资料最多的仍是 HOCON 配置文件方式。

特别说明： Web UI 能做什么（只读 + 查看）：不支持从Web UI提交，Web UI仅提供作业的监控/查询

集群概览（版本、worker 数、slot、任务数）
查看运行中 / 已完成的 Job 列表
打开 Job 详情页：DAG 图、source/sink 吞吐、实时指标、异常信息、配置、日志
查看 worker /master 节点状态、系统监控

Q: Apache Sea Tunnel 的端口用了哪些？在哪个配置文件配置？

SeaTunnel（SeaTunnel Engine / Zeta）用到的端口不多，主要就几个，分散在 hazelcast.yaml 和 seatunnel.yaml 两个配置文件里。

端口清单

端口	用途	配置文件	配置项
5801	集群节点间通信（Hazelcast 成员端口，含数据面/控制面）	`hazelcast.yaml`	`hazelcast.network.port.port`
5802	分离集群模式下 worker 节点的成员端口	`hazelcast.yaml`（worker 节点）	同上，值改 5802
8080	REST API / Web UI 的 HTTP 服务	`seatunnel.yaml`	`seatunnel.engine.http.port`
8443	REST API 的 HTTPS 服务（默认关闭）	`seatunnel.yaml`	`seatunnel.engine.http.https-port`

具体配置位置

1. 集群通信端口 —— hazelcast.yaml

hazelcast:
  network:
    join:
      tcp-ip:
        enabled: true
        member-list:
          - master-node-1:5801
          - worker-node-1:5802
    port:
      auto-increment: false
      port: 5801       # master 用 5801；worker 单独配置时改 5802

这个文件在 SeaTunnel 安装目录的 config/hazelcast.yaml。

2. HTTP/Web UI 端口 —— seatunnel.yaml

seatunnel:
  engine:
    http:
      enable-http: true
      port: 8080

位置在 config/seatunnel.yaml。注意：想用 Web UI 或 REST API 必须显式把 enable-http 打开，否则 8080 不监听。

补充说明

分离集群模式下 master 用 5801、worker 用 5802，两者在各自节点的 hazelcast.yaml 里配置。
K8s 部署时 Service 对外暴露的也是 5801（restapi 端口转发 5801:5801）。
早期版本（2.3.x 之前）REST API 也曾直接挂在 5801 上，端口归属以当前版本官方文档为准。

Q: Apache Sea Tunnel (flink) 支持从 kafka 拉取数据并转储到 apache lotDB 或 td-engine 吗？

✅ 结论

Apache SeaTunnel 使用 Flink 引擎，完全支持 Kafka Source 拉取实时流数据，转储到 Apache IoTDB 或者 TDengine，2个时序库都有官方 V2 Connector，Flink 引擎原生兼容这一套连接器。
链路：Kafka Source → Transform（JSON解析/字段映射）→ Apache IoTDB / TDengine Sink，流模式运行，支持 checkpoint、exactly-once。

注，本节的代码目前未实测。仅提供思路

1. 连接器支持情况

组件	引擎支持	说明
Kafka Source	Flink / Zeta / Spark	官方V2，流批一体，支持消费kafka json/avro消息，offset由checkpoint管理
Apache IoTDB Sink	Flink / Zeta / Spark	两个版本：`IoTDB`(0.13.x)、`IoTDBv2`(IoTDB 2.x)；支持设备、时间戳、测点映射，幂等写入，相同device+ts覆盖，保证exactly-once
TDengine Sink	Flink / Zeta / Spark	SeaTunnel官方V2 Sink，底层JDBC/TAOS驱动，支持超级表、子表写入，时序数据写入

⚠️ 依赖包注意（Flink引擎）
插件和驱动jar，放到 ${SEATUNNEL_HOME}/plugins/ 目录；Zeta引擎放到 lib 目录
# 一键安装连接器
sh bin/install-plugin.sh kafka iotdb tdengine

2. 典型Job配置示例（Flink引擎，Kafka → IoTDB）

env {
  execution.parallelism = 2
  job.mode = "STREAMING"
  checkpoint.interval = 5000
  engine = "flink"
}

source {
  Kafka {
    bootstrap.servers = "127.0.0.1:9092"
    topic = "sensor_data"
    consumer.group = "seatunnel_flink_group"
    format = "json"
  }
}

transform {
 # 解析json，提取ts、device、指标字段
  FieldMapper {
    source_fields = ["ts", "device_id", "temp", "hum"]
  }
}

sink {
  IoTDBv2 {
    node_urls = ["127.0.0.1:6667"]
    username = "root"
    password = "root"
    key_device = "device_id"
    key_timestamp = "ts"
    key_measurement_fields = ["temp", "hum"]
    batch_size = 1024
    batch_interval_ms = 3000
  }
}

Kafka → TDengine sink片段

sink {
  TDengine {
    url = "jdbc:TAOS://127.0.0.1:6030/test_db"
    username = "root"
    password = "taosdata"
    table_name = "sensor"
   # 映射timestamp、tag、field
    timestamp_column = "ts"
    tag_columns = ["device_id"]
    field_columns = ["temp", "hum"]
  }
}

3. 提交方式（Flink引擎）

# 提交到flink集群
bin/seatunnel.sh --config ./kafka2iotdb.conf -e flink

4. 关键注意点

Kafka消息格式：IoT场景绝大多数是JSON，format="json"直接解析；如果是二进制avro需要配置schema。
时序字段：IoTDB/TDengine都必须提取时间戳字段，同时区分设备/tag字段，这是时序库建模核心。
Exactly-Once实现：
- Kafka source：依赖Flink checkpoint提交offset
- IoTDB：依靠 device + timestamp 幂等覆盖实现exactly-once
- TDengine：依赖checkpoint+幂等主键
版本选型建议
- IoTDB 0.13.x → 使用 IoTDB sink
- IoTDB 2.x → 使用 IoTDBv2 sink
对比 Zeta 引擎：如果你没有存量Flink集群，优先推荐Zeta引擎，更轻量，不需要Flink集群部署；只有当你需要复用现有Flink、复杂Flink算子时才选Flink引擎。

5. 局限性

IoTDB Sink不支持CDC的update/delete，所有数据都当成插入；
TDengine Sink基于JDBC，高吞吐场景注意调优batch_size、batch_interval；
Flink引擎模式下，任务监控看Flink UI；SeaTunnel自带8080 Web UI只针对Zeta引擎，Flink任务不会在SeaTunnel 8080页面展示。

Y 推荐文献

X 参考文献

posted @ 2026-10-26 21:01 千千寰宇阅读(15) 评论(0) 收藏举报

刷新页面返回顶部

千千寰宇

面向Data+AI转型的资深数据与知识管理者、软件工程师，数智化转型实践者。 【文章分类】 【网络测验方法】 【好物推荐】