用 Apache SeaTunnel 同步 DynamoDB 到 Redis，一个配置文件就够

作者：Ricardo Ferreira
编译：Debra Chen

假设你把客户数据存储在 Amazon DynamoDB 中，作为应用的唯一数据源（single source of truth）。但如今你需要同样的数据放在 Redis 里，进行快速缓存、实时分析，或是为向量搜索推荐引擎赋能。举例来说，DynamoDB 中这张名为 mySourceTable 的表当前包含两条数据记录。

那么这个任务的挑战在哪呢？如何在不大写一堆代码的情况下，让两边保持同步？

就我个人而言，我最理想的方案是使用 Redis Data Integration（RDI），但遗憾的是，在撰写本文时，RDI 还不支持把 Amazon DynamoDB 作为数据源。这就让开发者只剩下两个选择：自己编写复杂的数据集成管道，或者投入昂贵（而且往往溢价过高）的 ETL 解决方案。这两个选项都不怎么让人满意。

但还有另一条路。

那就是 Apache SeaTunnel——一个开源数据集成项目，它让数据同步变得像写一个配置文件一样简单。在这篇博客中，我会深入介绍这个项目，并一步步讲解如何把数据从 Amazon DynamoDB 同步到 Redis。

Apache SeaTunnel：你的开源数据管道好帮手

Apache SeaTunnel 是一个多模态、高性能、分布式的海量数据集成工具，同时支持批处理和流式两种模式。你可以把它想象成你数据系统的"万能翻译官"——它精通 DynamoDB、Redis 以及其他 100 多种数据系统。

在这个场景下，SeaTunnel 的突出优势在于：

  • 零代码：用简单的配置文件即可定义你的数据管道
  • 分布式：当数据增长时，同步可横向扩展
  • 容错：内置 checkpoint 机制，确保数据不丢失

你可以用两种不同的配置方式使用 SeaTunnel：单机模式（standalone），用于一次性的数据搬运；或者集群模式（cluster），它能搭建一套常驻在线的数据管道基础设施，随时准备执行新任务。

方案一：快速上手（单机模式）

这个方案非常适合一次性迁移、本地开发或定时批处理任务。一条命令，搞定一切。

第一步，创建你的管道配置文件（jobs/my.config）：

env {
  parallelism = 1
  job.mode = "BATCH"
}
source {
  Amazondynamodb {
    url = "http://dynamodb.us-east-1.amazonaws.com"
    region = "us-east-1"
    access_key_id = "YOUR_ACCESS_KEY"
    secret_access_key = "YOUR_SECRET_KEY"
    table = "mySourceTable"
    schema = {
      fields {
        customerId = int
        customerName = string
        address = string
      }
    }
  }
}
sink {
  Redis {
    host = host.docker.internal  # 或者你的 Redis 主机地址
    port = 6379
    support_custom_key = true
    key = "customer:{customerId}"
    data_type = hash
  }
}

用一条 Docker 命令即可运行：

docker run --rm -it \
  -v $(pwd)/jobs:/config \
  apache/seatunnel:2.3.12 \
  ./bin/seatunnel.sh -m local -c /config/my.config

这条命令会启动一个新容器，里面运行一个单机模式 SeaTunnel 实例，执行 my.config 文件中描述的任务。你会看到类似下面的输出：

2026-01-09 16:57:50,786 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/seatunnel/config/seatunnel.yaml' from System property 'seatunnel.config'
2026-01-09 16:57:50,788 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/seatunnel/config/seatunnel.yaml
2026-01-09 16:57:50,789 INFO  [o.a.s.e.c.c.SeaTunnelConfig   ] [main] - seatunnel.home is /opt/seatunnel
2026-01-09 16:57:50,830 INFO  [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-01-09 16:57:50,830 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/seatunnel/config/hazelcast.yaml' from System property 'hazelcast.config'
2026-01-09 16:57:50,830 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/seatunnel/config/hazelcast.yaml
2026-01-09 16:57:50,962 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/seatunnel/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
2026-01-09 16:57:50,962 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/seatunnel/config/hazelcast-client.yaml
2026-01-09 16:57:50,985 WARN  [c.h.i.AddressPicker           ] [main] - [LOCAL] [seatunnel-420804] [5.1] You configured your member address as host name. Please be aware of that your dns can be spoofed. Make sure that your dns configurations are correct.
2026-01-09 16:57:50,985 INFO  [c.h.i.AddressPicker           ] [main] - [LOCAL] [seatunnel-420804] [5.1] Resolving domain name 'localhost' to address(es): [127.0.0.1, 0:0:0:0:0:0:0:1]
2026-01-09 16:57:50,986 INFO  [c.h.i.AddressPicker           ] [main] - [LOCAL] [seatunnel-420804] [5.1] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1]
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2026-01-09 16:57:51,189 INFO  [o.a.s.e.s.SeaTunnelServer     ] [main] - SeaTunnel server start...
2026-01-09 16:57:51,191 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Based on Hazelcast IMDG version: 5.1.0 (20220228 - 21f20e7)
2026-01-09 16:57:51,191 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Cluster name: seatunnel-420804
2026-01-09 16:57:51,191 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] 

 _____               _____                             _ 
/  ___|             |_   _|                           | |
\ `--.   ___   __ _   | |   _   _  _ __   _ __    ___ | |
 `--. \ / _ \ / _` |  | |  | | | || '_ \ | '_ \  / _ \| |
/\__/ /|  __/| (_| |  | |  | |_| || | | || | | ||  __/| |
\____/  \___| \__,_|  \_/   \__,_||_| |_||_| |_| \___||_|


2026-01-09 16:57:51,191 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Copyright © 2021-2024 The Apache Software Foundation. Apache SeaTunnel, SeaTunnel, and its feather logo are trademarks of The Apache Software Foundation.
2026-01-09 16:57:51,191 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Integrity Checker is disabled. Fail-fast on corrupted executables will not be performed.
To enable integrity checker do one of the following: 
  - Change member config using Java API: config.setIntegrityCheckerEnabled(true);
  - Change XML/YAML configuration property: Set hazelcast.integrity-checker.enabled to true
  - Add system property: -Dhz.integritychecker.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_INTEGRITYCHECKER_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2026-01-09 16:57:51,193 INFO  [c.h.system                    ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] The Jet engine is disabled.
To enable the Jet engine on the members, do one of the following:
  - Change member config using Java API: config.getJetConfig().setEnabled(true)
  - Change XML/YAML configuration property: Set hazelcast.jet.enabled to true
  - Add system property: -Dhz.jet.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_JET_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2026-01-09 16:57:51,331 INFO  [c.h.s.security                ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Enable DEBUG/FINE log level for log category com.hazelcast.system.security  or use -Dhazelcast.security.recommendations system property to see 🔒 security recommendations and the status of current config.
2026-01-09 16:57:51,392 INFO  [o.a.s.e.s.SeaTunnelNodeContext] [main] - Using LiteNodeDropOutTcpIpJoiner TCP/IP discovery
2026-01-09 16:57:51,393 WARN  [c.h.c.CPSubsystem             ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
2026-01-09 16:57:51,462 INFO  [.c.c.DefaultClassLoaderService] [main] - start classloader service with cache mode
2026-01-09 16:57:51,464 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/seatunnel/config/seatunnel.yaml' from System property 'seatunnel.config'
2026-01-09 16:57:51,464 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/seatunnel/config/seatunnel.yaml
2026-01-09 16:57:51,466 INFO  [amlSeaTunnelDomConfigProcessor] [main] - Dynamic slot is enabled, the schedule strategy is set to REJECT
2026-01-09 16:57:51,466 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/seatunnel/config/hazelcast.yaml' from System property 'hazelcast.config'
2026-01-09 16:57:51,466 INFO  [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/seatunnel/config/hazelcast.yaml
2026-01-09 16:57:51,471 WARN  [o.a.s.e.s.TaskExecutionService] [pool-3-thread-1] - [localhost]:5801 [seatunnel-420804] [5.1] The Node is not ready yet, Node state STARTING,looking forward to the next scheduling
2026-01-09 16:57:51,471 INFO  [o.a.s.e.s.TaskExecutionService] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Created new BusWork : 1257526338
2026-01-09 16:57:51,474 WARN  [a.s.e.s.s.s.DefaultSlotService] [hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource manager, will retry later. this address: [localhost]:5801
2026-01-09 16:57:51,475 INFO  [o.a.s.e.s.CoordinatorService  ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Start pending job schedule thread
2026-01-09 16:57:51,554 WARN  [o.a.h.u.NativeCodeLoader      ] [main] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2026-01-09 16:57:51,607 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-7-thread-1] - [localhost]:5801 [seatunnel-420804] [5.1] 
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   1
corePoolSize              :                  10
maximumPoolSize           :          2147483647
poolSize                  :                   1
completedTaskCount        :                   0
taskCount                 :                   1
***********************************************

2026-01-09 16:57:51,612 INFO  [o.a.s.e.s.JettyService        ] [main] - SeaTunnel REST service will start on port 8080
2026-01-09 16:57:51,617 INFO  [o.a.s.s.o.e.j.u.log           ] [main] - Logging initialized @1069ms to org.apache.seatunnel.shade.org.eclipse.jetty.util.log.Slf4jLog
2026-01-09 16:57:51,643 WARN  [a.s.s.o.e.j.s.h.ContextHandler] [main] - Empty contextPath
2026-01-09 16:57:51,652 INFO  [o.a.s.s.o.e.j.s.Server        ] [main] - jetty-9.4.56.v20240826; built: 2024-08-26T17:15:05.868Z; git: ec6782ff5ead824dabdcf47fa98f90a4aedff401; jvm 1.8.0_342-b07
2026-01-09 16:57:51,665 INFO  [o.a.s.s.o.e.j.s.session       ] [main] - DefaultSessionIdManager workerName=node0
2026-01-09 16:57:51,665 INFO  [o.a.s.s.o.e.j.s.session       ] [main] - No SessionScavenger set, using defaults
2026-01-09 16:57:51,666 INFO  [o.a.s.s.o.e.j.s.session       ] [main] - node0 Scavenging every 660000ms
2026-01-09 16:57:51,727 INFO  [a.s.s.o.e.j.s.h.ContextHandler] [main] - Started o.a.s.s.o.e.j.s.ServletContextHandler@7069f076{/,null,AVAILABLE}
2026-01-09 16:57:51,731 INFO  [.s.s.o.e.j.s.AbstractConnector] [main] - Started ServerConnector@6e31d989{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2026-01-09 16:57:51,732 INFO  [o.a.s.s.o.e.j.s.Server        ] [main] - Started @1184ms
2026-01-09 16:57:51,747 INFO  [c.h.i.d.Diagnostics           ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-01-09 16:57:51,749 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] [localhost]:5801 is STARTING
Members {size:1, ver:1} [
        Member [localhost]:5801 - 579e9374-7d47-4d91-80d5-c7a9155d42d0 [master node] [active master] this
]

2026-01-09 16:57:52,775 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-420804] [5.1] This node become a new active master node, begin init coordinator service
2026-01-09 16:57:52,779 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] [localhost]:5801 is STARTED
2026-01-09 16:57:52,797 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-5-thread-1] - [localhost]:5801 [seatunnel-420804] [5.1] Loaded event handlers: [org.apache.seatunnel.api.event.LoggingEventHandler@1ebbcf65]
2026-01-09 16:57:52,803 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel-420804] [5.1] Running with 2 response threads, dynamic=true
2026-01-09 16:57:52,807 INFO  [.h.i.p.i.PartitionStateManager] [seatunnel-coordinator-service-1] - [localhost]:5801 [seatunnel-420804] [5.1] Initializing cluster partition table arrangement...
2026-01-09 16:57:52,811 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2026-01-09 16:57:52,811 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2026-01-09 16:57:52,814 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-420804] [5.1] Trying to connect to cluster: seatunnel-420804
2026-01-09 16:57:52,815 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-420804] [5.1] Trying to connect to [localhost]:5801
2026-01-09 16:57:52,823 INFO  [.p.t.AuthenticationMessageTask] [hz.main.priority-generic-operation.thread-0] - [localhost]:5801 [seatunnel-420804] [5.1] Received auth from Connection[id=1, /127.0.0.1:5801->/127.0.0.1:45255, qualifier=null, endpoint=[127.0.0.1]:45255, remoteUuid=3be41dca-f0e7-4603-ad27-4e5b3aecbb4a, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: 3be41dca-f0e7-4603-ad27-4e5b3aecbb4a, client name: hz.client_1, client version: 5.1
2026-01-09 16:57:52,824 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2026-01-09 16:57:52,824 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-420804] [5.1] Authenticated with server [localhost]:5801:579e9374-7d47-4d91-80d5-c7a9155d42d0, server version: 5.1, local address: /127.0.0.1:45255
2026-01-09 16:57:52,825 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel-420804] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2026-01-09 16:57:52,829 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-10] - hz.client_1 [seatunnel-420804] [5.1] 

Members [1] {
        Member [localhost]:5801 - 579e9374-7d47-4d91-80d5-c7a9155d42d0 [master node]
}

2026-01-09 16:57:52,841 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
2026-01-09 16:57:52,858 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Loading config file from path: /config/my.config
2026-01-09 16:57:52,951 INFO  [o.a.s.c.s.u.ConfigShadeUtils  ] [main] - Load config shade spi: [base64]
2026-01-09 16:57:52,967 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Parsed config file: 
{
    "env" : {
        "parallelism" : 1,
        "job.mode" : "BATCH"
    },
    "source" : [
        {
            "url" : "http://dynamodb.us-east-1.amazonaws.com",
            "region" : "us-east-1",
            "access_key_id" : "<REDACTED>",
            "secret_access_key" : "<REDACTED>",
            "table" : "mySourceTable",
            "schema" : {
                "fields" : {
                    "customerId" : "int",
                    "customerName" : "string",
                    "address" : "string"
                }
            },
            "plugin_name" : "Amazondynamodb"
        }
    ],
    "sink" : [
        {
            "host" : "host.docker.internal",
            "port" : 6379,
            "support_custom_key" : true,
            "key" : "customer:{customerId}",
            "data_type" : "hash",
            "plugin_name" : "Redis"
        }
    ]
}

2026-01-09 16:57:52,971 INFO  [p.MultipleTableJobConfigParser] [main] - add common jar in plugins :[]
2026-01-09 16:57:52,978 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/seatunnel/connectors
2026-01-09 16:57:52,980 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Amazondynamodb'} at: [file:/opt/seatunnel/connectors/connector-amazondynamodb-2.3.12.jar]
2026-01-09 16:57:52,980 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Amazondynamodb'}: [file:/opt/seatunnel/connectors/connector-amazondynamodb-2.3.12.jar]
2026-01-09 16:57:52,982 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/seatunnel/connectors
2026-01-09 16:57:52,985 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/seatunnel/connectors
2026-01-09 16:57:52,986 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Redis'} at: [file:/opt/seatunnel/connectors/connector-redis-2.3.12.jar]
2026-01-09 16:57:52,986 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Redis'}: [file:/opt/seatunnel/connectors/connector-redis-2.3.12.jar]
2026-01-09 16:57:52,988 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sources.
2026-01-09 16:57:53,003 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - get the CatalogTable from source AmazonDynamodb: .default.default.default
2026-01-09 16:57:53,008 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSource Plugin from /opt/seatunnel/connectors
2026-01-09 16:57:53,011 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Amazondynamodb'} at: [file:/opt/seatunnel/connectors/connector-amazondynamodb-2.3.12.jar]
2026-01-09 16:57:53,011 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Amazondynamodb'}: [file:/opt/seatunnel/connectors/connector-amazondynamodb-2.3.12.jar]
2026-01-09 16:57:53,012 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all transforms.
2026-01-09 16:57:53,012 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sinks.
2026-01-09 16:57:53,013 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/seatunnel/connectors
2026-01-09 16:57:53,014 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Redis'} at: [file:/opt/seatunnel/connectors/connector-redis-2.3.12.jar]
2026-01-09 16:57:53,014 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - find connector jar and dependency for PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Redis'}: [file:/opt/seatunnel/connectors/connector-redis-2.3.12.jar]
2026-01-09 16:57:53,024 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - Create sink 'Redis' with upstream input catalog-table[database: default, schema: default, table: default]
2026-01-09 16:57:53,042 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Start submit job, job id: 1062052604328017921, with plugin jar [file:/opt/seatunnel/connectors/connector-amazondynamodb-2.3.12.jar, file:/opt/seatunnel/connectors/connector-redis-2.3.12.jar]
2026-01-09 16:57:53,047 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - Init ResourceManager
2026-01-09 16:57:53,047 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - initWorker... 
2026-01-09 16:57:53,047 INFO  [.e.s.r.AbstractResourceManager] [hz.main.client.thread-4] - init live nodes: [[localhost]:5801]
2026-01-09 16:57:53,048 INFO  [.e.s.r.AbstractResourceManager] [SeaTunnel-CompletableFuture-Thread-2] - received new worker register: [localhost]:5801
2026-01-09 16:57:53,152 INFO  [o.a.s.c.s.r.c.RedisParameters ] [hz.main.seaTunnel.task.thread-4] - Try to get redis version information from the jedis.info() method
2026-01-09 16:57:53,152 INFO  [o.a.s.c.s.r.c.RedisParameters ] [hz.main.seaTunnel.task.thread-4] - The version of Redis is :8.4.0
2026-01-09 16:57:53,154 INFO  [a.s.a.s.m.MultiTableSinkWriter] [hz.main.seaTunnel.task.thread-4] - init multi table sink writer, queue size: 1
2026-01-09 16:57:53,233 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-12] - log event: ReaderOpenEvent(createdTime=1767977873233, jobId=1062052604328017921, eventType=LIFECYCLE_READER_OPEN)
2026-01-09 16:57:53,402 INFO  [.s.t.SourceSplitEnumeratorTask] [hz.main.seaTunnel.task.thread-4] - received reader register, readerID: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}, taskID=1000200000000, index=0}
2026-01-09 16:57:53,429 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-9] - checkpoint is disabled, because in batch mode and 'checkpoint.interval' of env is missing.
2026-01-09 16:57:53,506 INFO  [a.s.AmazonDynamoDBSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - AmazonDynamoDB Source Reader [0] waiting for splits
2026-01-09 16:57:53,506 INFO  [a.s.AmazonDynamoDBSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - AmazonDynamoDB Source Reader [0] waiting for splits
2026-01-09 16:57:53,529 INFO  [.s.t.SourceSplitEnumeratorTask] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - received enough reader, starting enumerator...
2026-01-09 16:57:53,530 INFO  [nDynamoDBSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - Assigning org.apache.seatunnel.connectors.seatunnel.amazondynamodb.source.AmazonDynamoDBSourceSplit@36fe5b1d to 0 reader.
2026-01-09 16:57:53,530 INFO  [nDynamoDBSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - Assigning org.apache.seatunnel.connectors.seatunnel.amazondynamodb.source.AmazonDynamoDBSourceSplit@81bd425 to 0 reader.
2026-01-09 16:57:53,530 INFO  [nDynamoDBSourceSplitEnumerator] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - Assign splits [org.apache.seatunnel.connectors.seatunnel.amazondynamodb.source.AmazonDynamoDBSourceSplit@36fe5b1d, org.apache.seatunnel.connectors.seatunnel.amazondynamodb.source.AmazonDynamoDBSourceSplit@81bd425] to reader 0
2026-01-09 16:57:53,533 INFO  [a.s.AmazonDynamoDBSourceReader] [hz.main.generic-operation.thread-21] - Reader [0] received noMoreSplit event.
2026-01-09 16:57:53,934 INFO  [a.s.AmazonDynamoDBSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - AmazonDynamoDB Source Reader [0] waiting for splits
2026-01-09 16:57:53,934 INFO  [a.s.AmazonDynamoDBSourceReader] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - Closed the bounded amazonDynamodb source
2026-01-09 16:57:53,941 INFO  [.s.e.s.c.CheckpointCoordinator] [hz.main.generic-operation.thread-23] - skip schedule trigger checkpoint because checkpoint type is COMPLETED_POINT_TYPE
2026-01-09 16:57:53,943 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-8] - wait checkpoint completed: 9223372036854775807
2026-01-09 16:57:55,974 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-8] - pending checkpoint(9223372036854775807/1@1062052604328017921) notify finished!
2026-01-09 16:57:55,974 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-8] - start notify checkpoint completed, job id: 1062052604328017921, pipeline id: 1, checkpoint id:9223372036854775807
2026-01-09 16:57:55,979 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-8] - start clean pending checkpoint cause CheckpointCoordinator completed.
2026-01-09 16:57:55,979 INFO  [.s.e.s.c.CheckpointCoordinator] [seatunnel-coordinator-service-8] - Turn checkpoint_state_1062052604328017921_1 state from null to FINISHED
2026-01-09 16:57:55,987 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - [localhost]:5801 [seatunnel-420804] [5.1] taskDone, taskId = 1000100000000, taskGroup = TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}
2026-01-09 16:57:55,987 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-32] - log event: EnumeratorCloseEvent(createdTime=1767977875987, jobId=1062052604328017921, eventType=LIFECYCLE_ENUMERATOR_CLOSE)
2026-01-09 16:57:55,998 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] - [localhost]:5801 [seatunnel-420804] [5.1] taskGroup TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1} complete with FINISHED
2026-01-09 16:57:55,998 INFO  [o.a.s.e.s.TaskExecutionService] [hz.main.seaTunnel.task.thread-5] - [localhost]:5801 [seatunnel-420804] [5.1] Task TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1} complete with state FINISHED
2026-01-09 16:57:55,998 INFO  [o.a.s.e.s.CoordinatorService  ] [hz.main.seaTunnel.task.thread-5] - [localhost]:5801 [seatunnel-420804] [5.1] Received task end from execution TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}, state FINISHED
2026-01-09 16:57:56,000 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [hz.main.seaTunnel.task.thread-5] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] turned from state RUNNING to FINISHED.
2026-01-09 16:57:56,000 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [hz.main.seaTunnel.task.thread-5] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] state process is stopped
2026-01-09 16:57:56,000 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-8] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SplitEnumerator (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=1}] future complete with state FINISHED
2026-01-09 16:57:56,039 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-33] - log event: ReaderCloseEvent(createdTime=1767977876039, jobId=1062052604328017921, eventType=LIFECYCLE_READER_CLOSE)
2026-01-09 16:57:56,040 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - [localhost]:5801 [seatunnel-420804] [5.1] taskDone, taskId = 1000200000000, taskGroup = TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}
2026-01-09 16:57:56,076 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - [localhost]:5801 [seatunnel-420804] [5.1] taskDone, taskId = 1000200010000, taskGroup = TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}
2026-01-09 16:57:56,077 INFO  [o.a.s.a.e.LoggingEventHandler ] [hz.main.generic-operation.thread-34] - log event: WriterCloseEvent(createdTime=1767977876076, jobId=1062052604328017921, eventType=LIFECYCLE_WRITER_CLOSE)
2026-01-09 16:57:56,079 INFO  [o.a.s.e.s.TaskExecutionService] [BlockingWorker-TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] - [localhost]:5801 [seatunnel-420804] [5.1] taskGroup TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2} complete with FINISHED
2026-01-09 16:57:56,079 INFO  [o.a.s.e.s.TaskExecutionService] [hz.main.seaTunnel.task.thread-2] - [localhost]:5801 [seatunnel-420804] [5.1] Task TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2} complete with state FINISHED
2026-01-09 16:57:56,079 INFO  [o.a.s.e.s.CoordinatorService  ] [hz.main.seaTunnel.task.thread-2] - [localhost]:5801 [seatunnel-420804] [5.1] Received task end from execution TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}, state FINISHED
2026-01-09 16:57:56,080 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [hz.main.seaTunnel.task.thread-2] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] turned from state RUNNING to FINISHED.
2026-01-09 16:57:56,080 INFO  [o.a.s.e.s.d.p.PhysicalVertex  ] [hz.main.seaTunnel.task.thread-2] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] state process is stopped
2026-01-09 16:57:56,080 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-8] - Job (1062052604328017921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Amazondynamodb]-SourceTask (1/1)], taskGroupLocation: [TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}] future complete with state FINISHED
2026-01-09 16:57:56,080 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-8] - Job SeaTunnel_Job (1062052604328017921), Pipeline: [(1/1)] will end with state FINISHED
2026-01-09 16:57:56,081 INFO  [o.a.s.e.s.m.JobMaster         ] [hz.main.seaTunnel.task.thread-2] - release the task group resource TaskGroupLocation{jobId=1062052604328017921, pipelineId=1, taskGroupId=2}
2026-01-09 16:57:56,081 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-8] - Job SeaTunnel_Job (1062052604328017921), Pipeline: [(1/1)] turned from state RUNNING to FINISHED.
2026-01-09 16:57:56,081 INFO  [a.s.e.s.s.s.DefaultSlotService] [hz.main.generic-operation.thread-35] - received slot release request, jobID: 1062052604328017921, slot: SlotProfile{worker=[localhost]:5801, slotID=2, ownerJobID=1062052604328017921, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='e992c180-0d29-42db-ac3f-249a1130c0b1'}
2026-01-09 16:57:56,105 INFO  [o.a.s.e.s.m.JobMaster         ] [seatunnel-coordinator-service-8] - release the pipeline Job SeaTunnel_Job (1062052604328017921), Pipeline: [(1/1)] resource
2026-01-09 16:57:56,105 INFO  [a.s.e.s.s.s.DefaultSlotService] [hz.main.generic-operation.thread-39] - received slot release request, jobID: 1062052604328017921, slot: SlotProfile{worker=[localhost]:5801, slotID=1, ownerJobID=1062052604328017921, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='e992c180-0d29-42db-ac3f-249a1130c0b1'}
2026-01-09 16:57:56,106 INFO  [o.a.s.e.s.d.p.SubPlan         ] [seatunnel-coordinator-service-8] - Job SeaTunnel_Job (1062052604328017921), Pipeline: [(1/1)] state process is stop
2026-01-09 16:57:56,106 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-7] - Job SeaTunnel_Job (1062052604328017921), Pipeline: [(1/1)] future complete with state FINISHED
2026-01-09 16:57:56,107 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-7] - Job SeaTunnel_Job (1062052604328017921) turned from state RUNNING to FINISHED.
2026-01-09 16:57:56,107 INFO  [o.a.s.e.s.d.p.PhysicalPlan    ] [seatunnel-coordinator-service-7] - Job SeaTunnel_Job (1062052604328017921) state process is stop
2026-01-09 16:57:56,116 INFO  [o.a.s.a.e.LoggingEventHandler ] [seatunnel-coordinator-service-7] - log event: JobStateEvent(jobId=1062052604328017921, jobName=SeaTunnel_Job, jobStatus=FINISHED, createdTime=1767977876116)
2026-01-09 16:57:56,117 INFO  [o.a.s.e.c.j.ClientJobProxy    ] [main] - Job (1062052604328017921) end with state FINISHED
2026-01-09 16:57:56,132 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - 
***********************************************
           Job Statistic Information
***********************************************
Start Time                : 2026-01-09 16:57:52
End Time                  : 2026-01-09 16:57:56
Total Time(s)             :                   3
Total Read Count          :                   2
Total Write Count         :                   2
Total Failed Count        :                   0
***********************************************

2026-01-09 16:57:56,133 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2026-01-09 16:57:56,135 INFO  [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-1] - [localhost]:5801 [seatunnel-420804] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:45255, qualifier=null, endpoint=[127.0.0.1]:45255, remoteUuid=3be41dca-f0e7-4603-ad27-4e5b3aecbb4a, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
2026-01-09 16:57:56,135 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-420804] [5.1] Removed connection to endpoint: [localhost]:5801:579e9374-7d47-4d91-80d5-c7a9155d42d0, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:45255->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2026-01-09 16:57:56.131, lastWriteTime=2026-01-09 16:57:56.117, closedTime=2026-01-09 16:57:56.134, connected server version=5.1}
2026-01-09 16:57:56,135 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2026-01-09 16:57:56,136 INFO  [c.h.c.i.ClientEndpointManager ] [hz.main.event-1] - [localhost]:5801 [seatunnel-420804] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:45255, qualifier=null, endpoint=[127.0.0.1]:45255, remoteUuid=3be41dca-f0e7-4603-ad27-4e5b3aecbb4a, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=3be41dca-f0e7-4603-ad27-4e5b3aecbb4a, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1767977872821, latest clientAttributes=lastStatisticsCollectionTime=1767977872841,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1767977872816,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=9714790400,os.freePhysicalMemorySize=2077376512,os.freeSwapSpaceSize=1073737728,os.maxFileDescriptorCount=1048576,os.openFileDescriptorCount=103,os.processCpuTime=4280000000,os.systemLoadAverage=3.26,os.totalPhysicalMemorySize=8321540096,os.totalSwapSpaceSize=1073737728,runtime.availableProcessors=14,runtime.freeMemory=286196824,runtime.maxMemory=477626368,runtime.totalMemory=376438784,runtime.uptime=2297,runtime.usedMemory=90241960, labels=[]}
2026-01-09 16:57:56,136 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-420804] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2026-01-09 16:57:56,136 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
2026-01-09 16:57:56,136 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] [localhost]:5801 is SHUTTING_DOWN
2026-01-09 16:57:56,138 INFO  [c.h.i.p.i.MigrationManager    ] [hz.main.cached.thread-15] - [localhost]:5801 [seatunnel-420804] [5.1] Shutdown request of Member [localhost]:5801 - 579e9374-7d47-4d91-80d5-c7a9155d42d0 [master node] [active master] this is handled
2026-01-09 16:57:56,141 INFO  [c.h.i.i.Node                  ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Shutting down connection manager...
2026-01-09 16:57:56,142 INFO  [c.h.i.i.Node                  ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Shutting down node engine...
2026-01-09 16:57:56,149 INFO  [.s.s.o.e.j.s.AbstractConnector] [main] - Stopped ServerConnector@6e31d989{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2026-01-09 16:57:56,149 INFO  [o.a.s.s.o.e.j.s.session       ] [main] - node0 Stopped scavenging
2026-01-09 16:57:56,150 INFO  [a.s.s.o.e.j.s.h.ContextHandler] [main] - Stopped o.a.s.s.o.e.j.s.ServletContextHandler@7069f076{/,null,STOPPED}
2026-01-09 16:57:56,151 INFO  [.c.c.DefaultClassLoaderService] [main] - close classloader service
2026-01-09 16:57:56,152 INFO  [o.a.s.e.s.EventService        ] [event-forwarder-0] - Event forward thread interrupted
2026-01-09 16:57:56,155 INFO  [c.h.i.i.NodeExtension         ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Destroying node NodeExtension.
2026-01-09 16:57:56,156 INFO  [c.h.i.i.Node                  ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] Hazelcast Shutdown is completed in 18 ms.
2026-01-09 16:57:56,156 INFO  [c.h.c.LifecycleService        ] [main] - [localhost]:5801 [seatunnel-420804] [5.1] [localhost]:5801 is SHUTDOWN
2026-01-09 16:57:56,156 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed HazelcastInstance ......
2026-01-09 16:57:56,156 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed metrics executor service ......
2026-01-09 16:57:56,156 INFO  [s.c.s.s.c.ClientExecuteCommand] [SeaTunnel-CompletableFuture-Thread-7] - run shutdown hook because get close signal

任务执行完成后，用于执行它的 Docker 容器会被终止，此时你就能在 Redis 中看到写入的数据记录了。

正如 SeaTunnel 官方所说，整个过程简单到只需写一个配置文件。当然，要真正吃透 source 和 sink 连接器提供的众多配置项，可能还需要花些时间。你可以点击此处查看 Amazon DynamoDB 的配置属性，点击此处查看 Redis 的配置属性。

方案二：生产级引擎（集群模式）

当你需要一套常驻在线的数据管道基础设施，能够处理多个任务、按需扩容，并提供 REST API 访问时，集群模式就是你的选择。

第一步：搭建基础设施

创建一个 docker-compose.yaml，拉起一个完整的 SeaTunnel 集群。它同时也会带上 Redis——在这个用例里，Redis 是数据管道的目标端（sink）。

version: '3.8'
services:
  seatunnel-master:
    image: apache/seatunnel:2.3.12
    container_name: seatunnel-master
    environment:
      - ST_DOCKER_MEMBER_LIST=172.16.0.3,172.16.0.4,172.16.0.5
    volumes:
      - ./config/hazelcast-master.yaml:/opt/seatunnel/config/hazelcast-master.yaml
      - ./config/seatunnel.yaml:/opt/seatunnel/config/seatunnel.yaml
    entrypoint: >
      /bin/sh -c "
      /opt/seatunnel/bin/seatunnel-cluster.sh -r master
      "
    ports:
      - "5801:5801"
      - "8080:8080"  # REST API 端点
    networks:
      seatunnel_network:
        ipv4_address: 172.16.0.3
    healthcheck:
      test: [ "CMD-SHELL", "curl -f http://localhost:8080/system-monitoring-information || exit 1" ]
      interval: 30s
      timeout: 10s
      retries: 10
      start_period: 60s

  seatunnel-worker1:
    image: apache/seatunnel:2.3.12
    container_name: seatunnel-worker1
    environment:
      - ST_DOCKER_MEMBER_LIST=172.16.0.3,172.16.0.4,172.16.0.5
    volumes:
      - ./config/hazelcast-worker.yaml:/opt/seatunnel/config/hazelcast-worker.yaml
      - ./config/seatunnel.yaml:/opt/seatunnel/config/seatunnel.yaml
    entrypoint: >
      /bin/sh -c "
      /opt/seatunnel/bin/seatunnel-cluster.sh -r worker
      "
    depends_on:
      - seatunnel-master
    networks:
      seatunnel_network:
        ipv4_address: 172.16.0.4
    healthcheck:
      test: ["CMD-SHELL", "pgrep -f seatunnel || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 45s

  seatunnel-worker2:
    image: apache/seatunnel:2.3.12
    container_name: seatunnel-worker2
    environment:
      - ST_DOCKER_MEMBER_LIST=172.16.0.3,172.16.0.4,172.16.0.5
    volumes:
      - ./config/hazelcast-worker.yaml:/opt/seatunnel/config/hazelcast-worker.yaml
      - ./config/seatunnel.yaml:/opt/seatunnel/config/seatunnel.yaml
    entrypoint: >
      /bin/sh -c "
      /opt/seatunnel/bin/seatunnel-cluster.sh -r worker
      "
    depends_on:
      - seatunnel-master
    networks:
      seatunnel_network:
        ipv4_address: 172.16.0.5
    healthcheck:
      test: ["CMD-SHELL", "pgrep -f seatunnel || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 45s

  redis-database:
    container_name: redis-database
    hostname: redis-database
    image: redis:8.4.0
    ports:
      - "6379:6379"
    networks:
      seatunnel_network:
        ipv4_address: 172.16.0.2
    healthcheck:
      test: [ "CMD-SHELL", "redis-cli ping | grep PONG" ]
      interval: 10s
      retries: 5
      start_period: 5s
      timeout: 5s
networks:
  seatunnel_network:
    driver: bridge
    ipam:
      config:
        - subnet: 172.16.0.0/24

第二步：配置集群

创建 config/seatunnel.yaml 以启用 REST API：

seatunnel:
  engine:
    http:
      enable-http: true
      port: 8080

创建 config/hazelcast-master.yaml 和 config/hazelcast-worker.yaml：

hazelcast:
  network:
    rest-api:
      enabled: true
      endpoint-groups:
        CLUSTER_READ:
          enabled: true
        CLUSTER_WRITE:
          enabled: true
        HEALTH_CHECK:
          enabled: true
        DATA:
          enabled: true
    port:
      auto-increment: true
      port-count: 100
      port: 5801
    join:
      tcp-ip:
        enabled: true
        member-list:
          - 172.16.0.3
          - 172.16.0.4
          - 172.16.0.5

第三步：启动集群

docker compose up -d

现在你的 SeaTunnel 集群已运行，包含：

  • 1 个 Master 节点（协调器 + API 服务器）
  • 2 个 Worker 节点（实际执行数据管道）
  • Redis 数据库（你的 sink 和目标系统）

第四步：通过 REST API 提交任务

接下来是见证奇迹的时刻！用一条简单的 HTTP POST 提交任务：

curl -X POST http://localhost:8080/submit-job?jobName=dynamodb-to-redis \
  -H "Content-Type: application/json" \
  -d '{
 "env": {
 "parallelism": 2,
 "job.mode": "BATCH"
 },
 "source": [
 {
 "plugin_name": "Amazondynamodb",
 "plugin_output": "dynamodb_source",
 "url": "http://dynamodb.us-east-1.amazonaws.com",
 "region": "us-east-1",
 "access_key_id": "YOUR_ACCESS_KEY",
 "secret_access_key": "YOUR_SECRET_KEY",
 "table": "mySourceTable",
 "schema": {
 "fields": {
 "customerId": "int",
 "customerName": "string",
 "address": "string"
 }
 }
 }
 ],
 "transform": [],
 "sink": [
 {
 "plugin_name": "Redis",
 "host": "redis-database",
 "port": 6379,
 "support_custom_key": true,
 "key": "customer:{customerId}",
 "data_type": "hash"
 }
 ]
}'

任务一旦提交，就会立刻开始执行。你可以在 SeaTunnel 提供的 Web 控制台里跟进任务执行情况：访问 http://localhost:8080/#/overview，然后切换到 Jobs 标签页。

总结

有了 Apache SeaTunnel，把数据从 Amazon DynamoDB 同步到 Redis 就变成了一件"配置"层面的工作。无论你需要一次性迁移，还是一套常驻在线的数据集成管道，Apache SeaTunnel 都有满足你需求的工具。

使用 SeaTunnel 的妙处在于：

  • 无需维护代码——只需简单配置即可使用
  • 久经实战检验的连接器——专注于你的业务逻辑
  • 部署灵活——无论是笔记本还是生产集群都能跑
  • 天生可扩展——可随数据增长横向扩容

准备好试一试了吗？建议先用单机模式快速实验一下，等要上生产时再升级到集群模式。

posted @ 2026-09-02 15:06  ApacheSeaTunnel  阅读(47)  评论(0)    收藏  举报