Saga pattern with springboot

 

https://github.com/fanqingsong/springboot-kafka-streams-microservices-demo

 

Kafka Streams Microservices Demo

 

    

An E-commerce Microservices Application that demonstrates the usage of Kafka Streams.

---

Overview

Business Logic Flow

1. Client creates Order → POST /orders.
2. Order published → orders Kafka topic.
3. Payment Service → Listens, reserves funds → publishes to payments topic.
4. Stock Service → Listens, reserves inventory → publishes to stock topic.
5. Orders Service → Joins both responses → publishes to orders topic.
6. Results → Stored in KTable for querying.
7. Client retrieves order + its status → GET /orders.

1. (ms-orders) - Order Orchestrator

• RESTful Web Service for order management.
• Orchestrates the order processing flow.
• Publishes orders to Kafka.
• Joins responses from ms-payment and ms-stock services.
• Uses Kafka Streams with KTable for state persistence.
• Port: 9091.

2. (ms-payment) - Payment Processing Service

• Listens to order events.
• Reserves customer funds.
• Validates payment availability.
• Sends payment decisions back to orders topic.
• Maintains customer account state.
• Database: MySQL (customers table).

3. (ms-stock) - Inventory Management Service

• Listens to order events.
• Reserves product inventory.
• Validates stock availability.
• Sends stock decisions back to orders topic.
• Maintains product inventory state.
• Database: MySQL (products table).

---

Why Kafka Streams?

The Business Logic Problem

Look at the Business Logic Flow (steps 3-6):

3. Payment Service → publishes PAYMENT decision to payments topic.
4. Stock Service → publishes STOCK decision to stock topic.
5. Orders Service → Joins BOTH responses → publishes FINAL order.
6. Results → Stored in KTable for querying.

 

The Challenge:

• Payment service responds to order #123 → publishes to payments topic.
• Stock service responds to order #123 (might be delayed) → publishes to stock topic.
• Orders service MUST wait for BOTH and join them based on order ID.
• Must handle timing: What if stock response arrives after payment? Or never arrives?.
• Must persist: Final order result queried later in step 7.

Without a framework, this becomes incredibly complex.

What Kafka Streams Does For You (Per Business Flow)

Business Step	Challenge	Without KS	With KS
Step 3-4	Consume payment & stock concurrently in orders	Manual threading + offset management	Automatic consumer groups + No listeners
Step 5a	Buffer both responses	Manual in-memory maps	Built-in state stores
Step 5b	Join by order ID within 10 seconds	Complex correlation logic	join() with time windows
Step 5c	Handle late/missing responses	Manual timeout logic	Automatic window expiration
Step 6a	Persist final orders	Manual database inserts	Automatic KTable store
Step 6b	Recovery after crash	Manual changelog implementation	Automatic changelog topics
Step 7	Query persisted orders	Manual database queries	Query KTable directly

Added Advantages of Kafka Streams

Feature	Without Kafka Streams	With Kafka Streams
Join Logic	200+ lines of buffer management	3 lines with .join()
Time Windows	Manual timestamp tracking & expiration	Automatic window management
State Persistence	Build your own changelog system	Automatic changelog topics
Exactly-Once	Complex distributed transaction logic	Guaranteed by framework
Scaling	Manual partitioning coordination	Automatic dynamic scaling
Failure Recovery	Rebuild state from scratch	Replay from changelog topic
Production Ready	Months of testing & hardening	Battle-tested in thousands of companies

Key Kafka Streams Features in This Project

1. Stream Joins (Step 5): Automatically matches payment + stock responses by order ID with 10-second window.
2. KTable State Store (Step 6): Persists final orders in persistent state for querying.
3. Changelog Topics (Step 6 Recovery): Auto-created internal topics track all state changes for crash recovery.
4. Exactly-Once Semantics: Guarantees no duplicate order processing even if services crash.
5. Automatic Partitioning: Scales horizontally - add more instances without code changes.

格、status、source）。Topic：orders、payments、stock。消息 key 都是 orderId。

主流程（一次下单）

客户端 POST /orders

│

▼

ms-orders 生成 ID，status=NEW，写入 orders

│

├──────────────────┬──────────────────┐

▼ ▼ ▼

ms-payment 预留资金 ms-stock 预留库存 ms-orders 把 orders 物化成 KTable

结果写 payments 结果写 stock

│ │

└────────┬─────────┘

▼

Kafka Streams 按 orderId 窗口 Join（10 秒）

│

▼

confirm() 算出 CONFIRMED / REJECTED / ROLLBACK

再写回 orders

│

▼

支付 / 库存 第二次消费：确认扣款或回滚预留

1. 创建订单

POST /orders → createOrder()：用 UUID 高位生成正数 ID，设为 NEW，发到 orders。此时 HTTP 只表示「已发出」，最终成败要等 Join 之后。

2. 支付预留

ms-payment 用独立 consumer group 听 orders：

• NEW → reserve()：查客户余额。0 < price < amountAvailable 则把金额从 available 转到 reserved，状态 ACCEPT；否则 REJECT。结果发到 payments。
• 非 NEW（最终单）→ confirm()：CONFIRMED 只清 reserved；ROLLBACK 且失败源不是支付时，把 reserved 退回 available。

3. 库存预留

ms-stock 同样听 orders，逻辑对称：

• 库存够：available → reserved，ACCEPT
• 不够或商品不存在：REJECT
• 结果发到 stock
• 最终 CONFIRMED / ROLLBACK（失败源不是库存）时扣减或退回预留

两边失败都会带 source（PAYMENT 或 STOCK），方便另一边判断该不该回滚自己这边。

4. Join 出最终状态（核心）

ms-orders 的 Kafka Streams：

• 读 payments、stock
• 按 相同 key（orderId）、时间差 10 秒内 做 inner join
• 用 OrderService.confirm(payment, stock) 合成一单，再写回 orders

合成规则：

支付	库存	最终状态
ACCEPT + ACCEPT	CONFIRMED
REJECT + REJECT	REJECTED
一边 REJECT	ROLLBACK，并记下失败来源

这单再次进入 orders 后：

• KTable 更新，查询能看到最终状态
• 支付/库存再次消费，执行真正确认或补偿回滚

5. 查询

GET /orders / GET /orders/{id} 不查数据库，而是查 Streams 把 orders 物化出来的本地 Key-Value Store（KTable）。所以能查到的，是已经写进 orders 的消息：创建时的 NEW，以及 Join 后的最终状态。

为什么用 Kafka Streams

支付和库存是并行、到达时间不确定的。框架负责：按 key 对齐、10 秒窗口、join 状态 changelog、KTable 查询和崩溃恢复。没有 Streams 就要自己做相关 ID、缓冲和超时。

状态机（简图）

NEW ──reserve──► ACCEPT / REJECT （分别在 payments、stock）

│

Join confirm

▼

CONFIRMED / REJECTED / ROLLBACK

│

二次消费 confirm

▼

扣预留 或 退回 available

开发数据：支付侧预置几个客户余额，库存侧预置几种商品数量。余额或库存不够就会走 REJECT / ROLLBACK 那条补偿路径。