Spring Integration 集成 Kafka

• 1. Spring Integration 集成 Kafka
  • 1.1. 版本说明
  • 1.2. 概览
  • 1.3. Spring 配置
  • 1.4. 定义常量
  • 1.5. 消息入站
    • 1.5.1. 定义 Message Channel
    • 1.5.2. 定义 Service Activator
    • 1.5.3. 定义 Message Router
    • 1.5.4. 基于 Java DSL 定义 Inbound Channel Adapter
  • 1.6. 消息出站
    • 1.6.1. 定义 Message Channel
    • 1.6.2. 基于 Java DSL 定义 Outbound Channel Adapter
    • 1.6.3. 定义 Messaging Gateway
  • 1.7. 测试

1. Spring Integration 集成 Kafka

1.1. 版本说明

构件	版本
spring-boot	2.7.18
spring-boot-starter-integration	2.7.18
spring-integration-kafka	5.5.20

1.2. 概览

flowchart TB subgraph outBound["Out Bound"] direction TB KafkaGateway["MessagingGateway\n(kafkaGateway)"] --> outboundMessageChannel["MessageChannel\n(outboundMessageChannel)"] --> KafkaProducerMessageHandler["MessageHandler\n(KafkaProducerMessageHandler)"] end subgraph kafka["Kafka"] direction TB topic1["topic-1"] topic2["topic-2"] end subgraph inBound["In Bound"] KafkaInboundChannelAdapter["MessageProducer\n(KafkaInboundChannelAdapter)"] --> inboundMessageRouter["MessageRouter\n(inboundMessageRouter)"] --"topic-1"--> inboundMessageChannel1["MessageChannel\n(inboundMessageChannel1)"] --> inboundMessageHandler1["MessageHandler\n(inboundMessageHandler1)"] inboundMessageRouter --"topic-2"--> inboundMessageChannel2["MessageChannel\n(inboundMessageChannel2)"] --> inboundMessageHandler2["MessageHandler\n(inboundMessageHandler2)"] end KafkaProducerMessageHandler --> topic1 --> KafkaInboundChannelAdapter KafkaProducerMessageHandler --> topic2 --> KafkaInboundChannelAdapter

1.3. Spring 配置

spring:
  application:
    name: spring-integratiton-kafka-demo
  kafka:
    bootstrap-servers: 127.0.0.1:9092
    consumer:
      client-id: consumer
      group-id: consumer
    producer:
      client-id: producer

1.4. 定义常量

public static final String TOPIC_1 = "topic-1";
public static final String TOPIC_2 = "topic-2";

1.5. 消息入站

1.5.1. 定义 Message Channel

@Bean
public DirectChannelSpec inboundMessageChannel1() {
    return MessageChannels.direct();
}

@Bean
public DirectChannelSpec inboundMessageChannel2() {
    return MessageChannels.direct();
}

1.5.2. 定义 Service Activator

@Bean
@ServiceActivator(inputChannel = "inboundMessageChannel1")
public MessageHandler inboundMessageHandler1() {
    return message -> log.info("received a message, topic: {}, payload: {}", message.getHeaders().get(RECEIVED_TOPIC), message.getPayload());
}

@Bean
@ServiceActivator(inputChannel = "inboundMessageChannel2")
public MessageHandler inboundMessageHandler2() {
    return message -> log.info("received a message, topic: {}, payload: {}", message.getHeaders().get(RECEIVED_TOPIC), message.getPayload());
}

1.5.3. 定义 Message Router

@Bean
public HeaderValueRouter inboundMessageRouter() {
    HeaderValueRouter router = new HeaderValueRouter(RECEIVED_TOPIC);
    router.setChannelMapping(TOPIC_1, "inboundMessageChannel1");
    router.setChannelMapping(TOPIC_2, "inboundMessageChannel2");
    return router;
}

1.5.4. 基于 Java DSL 定义 Inbound Channel Adapter

@Bean
public IntegrationFlow inboundIntegrationFlow(ConsumerFactory<?, ?> consumerFactory) {
    return IntegrationFlows.from(Kafka.inboundChannelAdapter(consumerFactory, new ConsumerProperties(TOPIC_1, TOPIC_2)))
            .route(inboundMessageRouter())
            .get();
}

1.6. 消息出站

1.6.1. 定义 Message Channel

@Bean
public DirectChannelSpec outboundMessageChannel() {
    return MessageChannels.direct();
}

1.6.2. 基于 Java DSL 定义 Outbound Channel Adapter

@Bean
public IntegrationFlow outboundIntegrationFlow(ProducerFactory<?, ?> producerFactory) {
    return IntegrationFlows.from(outboundMessageChannel())
            .handle(Kafka.outboundChannelAdapter(producerFactory))
            .get();
}

1.6.3. 定义 Messaging Gateway

@MessagingGateway(defaultRequestChannel = "outboundMessageChannel")
public interface KafkaGateway {
    void sent2Kafka(@Header(TOPIC) String topic, @Payload String payload);
}

1.7. 测试

@Component
@Slf4j
public class SpringIntegrationKafkaDemo implements ApplicationRunner {

    @Resource
    private KafkaConfiguration.KafkaGateway kafkaGateway;

    @Override
    public void run(ApplicationArguments args) throws Exception {
        String payload1 = "hello topic 1";
        kafkaGateway.sent2Kafka(TOPIC_1, payload1);
        log.info("send a message, topic: {}, payload: {}", TOPIC_1, payload1);

        String payload2 = "hello topic 2";
        kafkaGateway.sent2Kafka(KafkaConstants.TOPIC_2, payload2);
        log.info("send a message, topic: {}, payload: {}", TOPIC_2, payload2);
    }
}

启动程序，控制台将输出：

send a message, topic: topic-1, payload: hello topic 1
send a message, topic: topic-2, payload: hello topic 2
received a message, topic: topic-1, payload: hello topic 1
received a message, topic: topic-2, payload: hello topic 2