最终一致性方案详解
本章导读
最终一致性是分布式系统中平衡性能与一致性的重要策略,广泛应用于互联网大规模系统。本章将深入探讨事件驱动架构、CQRS、Saga模式等最终一致性实现方案,帮助你设计高可用、高性能的分布式系统。
学习目标:
- 目标1:理解最终一致性的核心概念和适用场景
- 目标2:掌握事件溯源、CQRS模式的设计与实现
- 目标3:能够运用Saga模式处理分布式事务
前置知识:熟悉强一致性方案(2PC、Paxos、Raft),了解消息队列基础
阅读时长:约 50 分钟
一、知识概述
最终一致性(Eventual Consistency)是分布式系统中一种重要的一致性模型,它允许系统在一段时间内处于不一致状态,但保证在没有新更新的情况下,最终所有副本都会达到一致状态。本文将深入探讨最终一致性的实现方案,包括消息队列、事件溯源、CQRS 等模式。
最终一致性的定义
最终一致性保证:
- 收敛性:所有副本最终会收敛到相同的状态
- 可用性:系统始终可读写
- 分区容错:网络分区时系统仍可工作
BASE 理论
BASE 是对 CAP 定理中 AP 方案的延伸:
- Basically Available(基本可用):系统出现故障时,允许损失部分可用性
- Soft State(软状态):允许系统存在中间状态
- Eventually Consistent(最终一致性):系统最终达到一致
二、核心实现方案
2.1 消息队列实现最终一致性
原理说明
通过消息队列实现异步数据同步:
- 主系统完成操作后发送消息
- 从系统异步消费消息并同步数据
- 通过重试机制保证消息最终被处理
Java 实现
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
/**
* 最终一致性消息队列
*/
public class EventualConsistencyMessageQueue {
private final String name;
private final BlockingQueue<Message> queue;
private final ExecutorService executor;
private final MessageStore messageStore;
private final List<MessageConsumer> consumers;
private final ScheduledExecutorService retryScheduler;
// 消息状态
public enum MessageStatus {
PENDING, PROCESSING, COMPLETED, FAILED, RETRYING
}
public EventualConsistencyMessageQueue(String name) {
this.name = name;
this.queue = new LinkedBlockingQueue<>(10000);
this.executor = Executors.newFixedThreadPool(4);
this.messageStore = new MessageStore();
this.consumers = new CopyOnWriteArrayList<>();
this.retryScheduler = Executors.newScheduledThreadPool(1);
startConsumers();
startRetryTask();
}
/**
* 发送消息
*/
public void send(String topic, Object payload) {
Message message = new Message(
UUID.randomUUID().toString(),
topic,
payload,
System.currentTimeMillis()
);
// 持久化消息
messageStore.save(message);
// 入队
try {
queue.put(message);
System.out.println("[Queue-" + name + "] Sent message: " + message.id);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
/**
* 注册消费者
*/
public void subscribe(MessageConsumer consumer) {
consumers.add(consumer);
}
/**
* 启动消费者
*/
private void startConsumers() {
for (int i = 0; i < 4; i++) {
executor.submit(() -> {
while (!Thread.currentThread().isInterrupted()) {
try {
Message message = queue.poll(1, TimeUnit.SECONDS);
if (message != null) {
processMessage(message);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
break;
}
}
});
}
}
/**
* 处理消息
*/
private void processMessage(Message message) {
messageStore.updateStatus(message.id, MessageStatus.PROCESSING);
boolean success = false;
Exception lastException = null;
for (MessageConsumer consumer : consumers) {
if (consumer.getTopic().equals(message.topic)) {
try {
consumer.consume(message);
success = true;
} catch (Exception e) {
lastException = e;
success = false;
break;
}
}
}
if (success) {
messageStore.updateStatus(message.id, MessageStatus.COMPLETED);
System.out.println("[Queue-" + name + "] Message completed: " + message.id);
} else {
handleFailedMessage(message, lastException);
}
}
/**
* 处理失败消息
*/
private void handleFailedMessage(Message message, Exception e) {
message.retryCount++;
if (message.retryCount >= message.maxRetries) {
// 超过最大重试次数,标记为失败
messageStore.updateStatus(message.id, MessageStatus.FAILED);
System.err.println("[Queue-" + name + "] Message failed after " +
message.retryCount + " retries: " + message.id);
} else {
// 加入重试队列
messageStore.updateStatus(message.id, MessageStatus.RETRYING);
System.out.println("[Queue-" + name + "] Message will retry: " + message.id +
" (attempt " + message.retryCount + ")");
}
}
/**
* 启动重试任务
*/
private void startRetryTask() {
retryScheduler.scheduleAtFixedRate(() -> {
List<Message> retryMessages = messageStore.findRetryMessages();
for (Message message : retryMessages) {
try {
queue.put(message);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}, 5, 5, TimeUnit.SECONDS);
}
/**
* 关闭队列
*/
public void shutdown() {
executor.shutdown();
retryScheduler.shutdown();
}
// 内部类
public static class Message {
final String id;
final String topic;
final Object payload;
final long timestamp;
volatile int retryCount;
final int maxRetries;
volatile MessageStatus status;
Message(String id, String topic, Object payload, long timestamp) {
this.id = id;
this.topic = topic;
this.payload = payload;
this.timestamp = timestamp;
this.retryCount = 0;
this.maxRetries = 3;
this.status = MessageStatus.PENDING;
}
}
public interface MessageConsumer {
String getTopic();
void consume(Message message) throws Exception;
}
/**
* 消息存储
*/
static class MessageStore {
private final Map<String, Message> messages = new ConcurrentHashMap<>();
void save(Message message) {
messages.put(message.id, message);
}
void updateStatus(String id, MessageStatus status) {
Message message = messages.get(id);
if (message != null) {
message.status = status;
}
}
List<Message> findRetryMessages() {
return messages.values().stream()
.filter(m -> m.status == MessageStatus.RETRYING)
.toList();
}
}
}
/**
* 基于消息队列的最终一致性数据同步
*/
public class EventualConsistencySync {
private final EventualConsistencyMessageQueue messageQueue;
public EventualConsistencySync() {
this.messageQueue = new EventualConsistencyMessageQueue("data-sync");
}
/**
* 同步数据变更
*/
public void syncDataChange(String table, String operation, Map<String, Object> data) {
DataChangeEvent event = new DataChangeEvent(table, operation, data);
messageQueue.send("data-change", event);
}
/**
* 注册数据同步处理器
*/
public void registerSyncHandler(DataSyncHandler handler) {
messageQueue.subscribe(new EventualConsistencyMessageQueue.MessageConsumer() {
@Override
public String getTopic() {
return "data-change";
}
@Override
public void consume(EventualConsistencyMessageQueue.Message message) throws Exception {
DataChangeEvent event = (DataChangeEvent) message.payload;
handler.handleSync(event);
}
});
}
static class DataChangeEvent {
final String table;
final String operation;
final Map<String, Object> data;
final long timestamp;
DataChangeEvent(String table, String operation, Map<String, Object> data) {
this.table = table;
this.operation = operation;
this.data = data;
this.timestamp = System.currentTimeMillis();
}
}
interface DataSyncHandler {
void handleSync(DataChangeEvent event) throws Exception;
}
}
// 使用示例
public class EventualConsistencyDemo {
public static void main(String[] args) throws InterruptedException {
EventualConsistencySync sync = new EventualConsistencySync();
// 注册同步处理器
sync.registerSyncHandler(event -> {
System.out.println("Syncing " + event.operation + " on " + event.table +
": " + event.data);
// 模拟同步到其他数据库
});
// 发送数据变更
Map<String, Object> userData = new HashMap<>();
userData.put("id", "user-001");
userData.put("name", "张三");
userData.put("email", "zhangsan@example.com");
sync.syncDataChange("users", "INSERT", userData);
// 更新操作
userData.put("name", "李四");
sync.syncDataChange("users", "UPDATE", userData);
Thread.sleep(2000);
}
}
2.2 事件溯源(Event Sourcing)
原理说明
事件溯源通过存储状态变更事件来记录数据:
- 不存储当前状态,而是存储所有变更事件
- 当前状态通过重放事件计算得出
- 保证事件的不可变性和顺序性
Java 实现
import java.util.*;
import java.util.concurrent.*;
import java.util.stream.*;
/**
* 事件存储
*/
public class EventStore {
private final Map<String, List<Event>> streams;
private final List<Event> allEvents;
private final List<EventSubscriber> subscribers;
public EventStore() {
this.streams = new ConcurrentHashMap<>();
this.allEvents = new CopyOnWriteArrayList<>();
this.subscribers = new CopyOnWriteArrayList<>();
}
/**
* 追加事件到流
*/
public void append(String streamId, Event event) {
// 获取当前版本
List<Event> stream = streams.computeIfAbsent(streamId, k -> new CopyOnWriteArrayList<>());
long expectedVersion = stream.size();
// 乐观锁检查
if (event.expectedVersion != expectedVersion) {
throw new ConcurrentModificationException(
"Expected version " + expectedVersion + " but got " + event.expectedVersion);
}
event.version = expectedVersion + 1;
stream.add(event);
allEvents.add(event);
// 通知订阅者
notifySubscribers(event);
System.out.println("[EventStore] Appended event to stream " + streamId +
": " + event.type + " (version " + event.version + ")");
}
/**
* 获取流的所有事件
*/
public List<Event> getEvents(String streamId) {
return new ArrayList<>(streams.getOrDefault(streamId, Collections.emptyList()));
}
/**
* 获取流的事件(从某个版本开始)
*/
public List<Event> getEvents(String streamId, long fromVersion) {
return streams.getOrDefault(streamId, Collections.emptyList())
.stream()
.filter(e -> e.version > fromVersion)
.collect(Collectors.toList());
}
/**
* 订阅事件
*/
public void subscribe(EventSubscriber subscriber) {
subscribers.add(subscriber);
}
/**
* 通知订阅者
*/
private void notifySubscribers(Event event) {
for (EventSubscriber subscriber : subscribers) {
try {
subscriber.onEvent(event);
} catch (Exception e) {
System.err.println("Subscriber error: " + e.getMessage());
}
}
}
/**
* 获取所有事件(用于重建)
*/
public List<Event> getAllEvents() {
return new ArrayList<>(allEvents);
}
}
/**
* 事件基类
*/
public abstract class Event {
public String id;
public String streamId;
public String type;
public long timestamp;
public long version;
public long expectedVersion;
public Event(String streamId, String type) {
this.id = UUID.randomUUID().toString();
this.streamId = streamId;
this.type = type;
this.timestamp = System.currentTimeMillis();
}
}
/**
* 事件订阅者
*/
interface EventSubscriber {
void onEvent(Event event);
}
/**
* 聚合基类
*/
public abstract class AggregateRoot {
protected String id;
protected long version;
protected List<Event> uncommittedChanges = new ArrayList<>();
/**
* 应用事件(用于重建状态)
*/
public abstract void apply(Event event);
/**
* 从事件流重建
*/
public void loadFromHistory(List<Event> history) {
for (Event event : history) {
apply(event);
version = event.version;
}
}
/**
* 获取未提交的变更
*/
public List<Event> getUncommittedChanges() {
return new ArrayList<>(uncommittedChanges);
}
/**
* 标记变更已提交
*/
public void markChangesAsCommitted() {
uncommittedChanges.clear();
}
}
/**
* 订单聚合(示例)
*/
public class Order extends AggregateRoot {
private String customerId;
private List<OrderItem> items = new ArrayList<>();
private OrderStatus status;
private Money totalAmount;
public enum OrderStatus {
CREATED, CONFIRMED, PAID, SHIPPED, DELIVERED, CANCELLED
}
// 创建订单
public static Order create(String orderId, String customerId) {
Order order = new Order();
order.raiseEvent(new OrderCreatedEvent(orderId, customerId));
return order;
}
// 添加商品
public void addItem(String productId, String productName, int quantity, Money price) {
if (status != OrderStatus.CREATED) {
throw new IllegalStateException("Cannot add items to non-pending order");
}
raiseEvent(new OrderItemAddedEvent(id, productId, productName, quantity, price));
}
// 确认订单
public void confirm() {
if (status != OrderStatus.CREATED) {
throw new IllegalStateException("Order already confirmed");
}
if (items.isEmpty()) {
throw new IllegalStateException("Cannot confirm empty order");
}
raiseEvent(new OrderConfirmedEvent(id));
}
// 支付
public void pay(Money paidAmount) {
if (status != OrderStatus.CONFIRMED) {
throw new IllegalStateException("Order not confirmed");
}
if (!paidAmount.equals(totalAmount)) {
throw new IllegalArgumentException("Payment amount mismatch");
}
raiseEvent(new OrderPaidEvent(id, paidAmount));
}
// 取消
public void cancel(String reason) {
if (status == OrderStatus.SHIPPED || status == OrderStatus.DELIVERED) {
throw new IllegalStateException("Cannot cancel shipped order");
}
raiseEvent(new OrderCancelledEvent(id, reason));
}
// 应用事件
@Override
public void apply(Event event) {
if (event instanceof OrderCreatedEvent e) {
this.id = e.streamId;
this.customerId = e.customerId;
this.status = OrderStatus.CREATED;
this.totalAmount = new Money(0, "CNY");
} else if (event instanceof OrderItemAddedEvent e) {
this.items.add(new OrderItem(e.productId, e.productName, e.quantity, e.price));
this.totalAmount = totalAmount.add(e.price.multiply(e.quantity));
} else if (event instanceof OrderConfirmedEvent e) {
this.status = OrderStatus.CONFIRMED;
} else if (event instanceof OrderPaidEvent e) {
this.status = OrderStatus.PAID;
} else if (event instanceof OrderCancelledEvent e) {
this.status = OrderStatus.CANCELLED;
}
}
// 触发事件
private void raiseEvent(Event event) {
event.expectedVersion = version;
apply(event);
uncommittedChanges.add(event);
}
// Getters
public String getCustomerId() { return customerId; }
public List<OrderItem> getItems() { return items; }
public OrderStatus getStatus() { return status; }
public Money getTotalAmount() { return totalAmount; }
}
// 领域事件
class OrderCreatedEvent extends Event {
final String customerId;
OrderCreatedEvent(String orderId, String customerId) {
super(orderId, "OrderCreated");
this.customerId = customerId;
}
}
class OrderItemAddedEvent extends Event {
final String productId;
final String productName;
final int quantity;
final Money price;
OrderItemAddedEvent(String orderId, String productId, String productName, int quantity, Money price) {
super(orderId, "OrderItemAdded");
this.productId = productId;
this.productName = productName;
this.quantity = quantity;
this.price = price;
}
}
class OrderConfirmedEvent extends Event {
OrderConfirmedEvent(String orderId) {
super(orderId, "OrderConfirmed");
}
}
class OrderPaidEvent extends Event {
final Money paidAmount;
OrderPaidEvent(String orderId, Money paidAmount) {
super(orderId, "OrderPaid");
this.paidAmount = paidAmount;
}
}
class OrderCancelledEvent extends Event {
final String reason;
OrderCancelledEvent(String orderId, String reason) {
super(orderId, "OrderCancelled");
this.reason = reason;
}
}
// 值对象
record Money(long amount, String currency) {
Money add(Money other) {
return new Money(amount + other.amount, currency);
}
Money multiply(int factor) {
return new Money(amount * factor, currency);
}
}
record OrderItem(String productId, String productName, int quantity, Money price) {}
/**
* 订单仓储
*/
public class OrderRepository {
private final EventStore eventStore;
public OrderRepository(EventStore eventStore) {
this.eventStore = eventStore;
}
/**
* 获取订单
*/
public Order getById(String orderId) {
List<Event> events = eventStore.getEvents(orderId);
if (events.isEmpty()) {
return null;
}
Order order = new Order();
order.loadFromHistory(events);
return order;
}
/**
* 保存订单
*/
public void save(Order order) {
List<Event> changes = order.getUncommittedChanges();
for (Event event : changes) {
eventStore.append(order.id, event);
}
order.markChangesAsCommitted();
}
}
// 使用示例
public class EventSourcingDemo {
public static void main(String[] args) {
EventStore eventStore = new EventStore();
OrderRepository repository = new OrderRepository(eventStore);
// 创建订单
Order order = Order.create("order-001", "customer-001");
order.addItem("prod-001", "iPhone 15", 1, new Money(699900, "CNY"));
order.addItem("prod-002", "AirPods Pro", 1, new Money(199900, "CNY"));
order.confirm();
order.pay(new Money(899800, "CNY"));
// 保存
repository.save(order);
// 从事件重建
Order loadedOrder = repository.getById("order-001");
System.out.println("\n=== Rebuilt Order ===");
System.out.println("Status: " + loadedOrder.getStatus());
System.out.println("Items: " + loadedOrder.getItems().size());
System.out.println("Total: " + loadedOrder.getTotalAmount().amount() / 100.0 +
" " + loadedOrder.getTotalAmount().currency());
}
}
2.3 CQRS(命令查询职责分离)
原理说明
CQRS 将读操作和写操作分离:
- Command Side:处理写操作,使用领域模型,保证一致性
- Query Side:处理读操作,使用优化的查询模型,追求性能
- Event Bus:通过事件同步读写模型
Java 实现
import java.util.*;
import java.util.concurrent.*;
/**
* CQRS 架构
*/
public class CQRS {
private final CommandBus commandBus;
private final QueryBus queryBus;
private final EventBus eventBus;
public CQRS() {
this.eventBus = new EventBus();
this.commandBus = new CommandBus(eventBus);
this.queryBus = new QueryBus();
}
public void registerCommandHandler(Class<?> commandType, CommandHandler handler) {
commandBus.registerHandler(commandType, handler);
}
public void registerQueryHandler(Class<?> queryType, QueryHandler handler) {
queryBus.registerHandler(queryType, handler);
}
public void registerEventHandler(Class<?> eventType, EventHandler handler) {
eventBus.subscribe(eventType, handler);
}
public <T> T send(Command command) {
return commandBus.send(command);
}
public <T> T query(Query query) {
return queryBus.query(query);
}
}
/**
* 命令总线
*/
public class CommandBus {
private final Map<Class<?>, CommandHandler> handlers;
private final EventBus eventBus;
public CommandBus(EventBus eventBus) {
this.handlers = new ConcurrentHashMap<>();
this.eventBus = eventBus;
}
public void registerHandler(Class<?> commandType, CommandHandler handler) {
handlers.put(commandType, handler);
}
@SuppressWarnings("unchecked")
public <T> T send(Command command) {
CommandHandler handler = handlers.get(command.getClass());
if (handler == null) {
throw new IllegalArgumentException("No handler for " + command.getClass());
}
CommandResult<T> result = handler.handle(command);
// 发布领域事件
for (Event event : result.events) {
eventBus.publish(event);
}
return result.result;
}
}
/**
* 查询总线
*/
public class QueryBus {
private final Map<Class<?>, QueryHandler> handlers;
public QueryBus() {
this.handlers = new ConcurrentHashMap<>();
}
public void registerHandler(Class<?> queryType, QueryHandler handler) {
handlers.put(queryType, handler);
}
@SuppressWarnings("unchecked")
public <T> T query(Query query) {
QueryHandler handler = handlers.get(query.getClass());
if (handler == null) {
throw new IllegalArgumentException("No handler for " + query.getClass());
}
return (T) handler.handle(query);
}
}
/**
* 事件总线
*/
public class EventBus {
private final Map<Class<?>, List<EventHandler>> subscribers;
private final ExecutorService executor;
public EventBus() {
this.subscribers = new ConcurrentHashMap<>();
this.executor = Executors.newFixedThreadPool(4);
}
public void subscribe(Class<?> eventType, EventHandler handler) {
subscribers.computeIfAbsent(eventType, k -> new CopyOnWriteArrayList<>())
.add(handler);
}
public void publish(Event event) {
List<EventHandler> handlers = subscribers.get(event.getClass());
if (handlers != null) {
for (EventHandler handler : handlers) {
executor.submit(() -> {
try {
handler.handle(event);
} catch (Exception e) {
System.err.println("Event handler error: " + e.getMessage());
}
});
}
}
}
}
// 基础接口
interface Command {}
interface Query {}
interface Event {}
interface CommandHandler<T> {
CommandResult<T> handle(Command command);
}
interface QueryHandler<T> {
T handle(Query query);
}
interface EventHandler {
void handle(Event event);
}
class CommandResult<T> {
final T result;
final List<Event> events;
CommandResult(T result, List<Event> events) {
this.result = result;
this.events = events;
}
static <T> CommandResult<T> of(T result, Event... events) {
return new CommandResult<>(result, Arrays.asList(events));
}
}
/**
* 产品命令和查询
*/
// 创建产品命令
class CreateProductCommand implements Command {
final String productId;
final String name;
final String description;
final Money price;
final int stock;
CreateProductCommand(String productId, String name, String description, Money price, int stock) {
this.productId = productId;
this.name = name;
this.description = description;
this.price = price;
this.stock = stock;
}
}
// 更新库存命令
class UpdateStockCommand implements Command {
final String productId;
final int delta;
UpdateStockCommand(String productId, int delta) {
this.productId = productId;
this.delta = delta;
}
}
// 查询产品
class GetProductQuery implements Query {
final String productId;
GetProductQuery(String productId) {
this.productId = productId;
}
}
// 搜索产品
class SearchProductsQuery implements Query {
final String keyword;
final int page;
final int size;
SearchProductsQuery(String keyword, int page, int size) {
this.keyword = keyword;
this.page = page;
this.size = size;
}
}
// 产品事件
class ProductCreatedEvent implements Event {
final String productId;
final String name;
final Money price;
final long timestamp;
ProductCreatedEvent(String productId, String name, Money price) {
this.productId = productId;
this.name = name;
this.price = price;
this.timestamp = System.currentTimeMillis();
}
}
class StockUpdatedEvent implements Event {
final String productId;
final int newStock;
final int delta;
StockUpdatedEvent(String productId, int newStock, int delta) {
this.productId = productId;
this.newStock = newStock;
this.delta = delta;
}
}
/**
* 写模型(命令侧)
*/
public class ProductWriteModel {
private final Map<String, ProductAggregate> products;
public ProductWriteModel() {
this.products = new ConcurrentHashMap<>();
}
public ProductAggregate getOrCreate(String productId) {
return products.computeIfAbsent(productId, id -> new ProductAggregate(id));
}
public ProductAggregate get(String productId) {
return products.get(productId);
}
public void applyEvent(Event event) {
// 同步事件到读模型
}
}
/**
* 产品聚合
*/
public class ProductAggregate {
private String id;
private String name;
private String description;
private Money price;
private int stock;
public ProductAggregate(String id) {
this.id = id;
}
public List<Event> create(String name, String description, Money price, int stock) {
this.name = name;
this.description = description;
this.price = price;
this.stock = stock;
return Collections.singletonList(
new ProductCreatedEvent(id, name, price)
);
}
public List<Event> updateStock(int delta) {
if (stock + delta < 0) {
throw new IllegalArgumentException("Insufficient stock");
}
stock += delta;
return Collections.singletonList(
new StockUpdatedEvent(id, stock, delta)
);
}
// Getters
public String getId() { return id; }
public String getName() { return name; }
public String getDescription() { return description; }
public Money getPrice() { return price; }
public int getStock() { return stock; }
}
/**
* 读模型(查询侧)
*/
public class ProductReadModel {
// 主数据存储
private final Map<String, ProductDto> products;
// 搜索索引
private final Map<String, Set<String>> nameIndex;
public ProductReadModel() {
this.products = new ConcurrentHashMap<>();
this.nameIndex = new ConcurrentHashMap<>();
}
/**
* 应用事件更新读模型
*/
public void apply(ProductCreatedEvent event) {
ProductDto dto = new ProductDto(
event.productId,
event.name,
event.price,
0,
event.timestamp
);
products.put(event.productId, dto);
// 更新索引
String nameLower = event.name.toLowerCase();
nameIndex.computeIfAbsent(nameLower, k -> ConcurrentHashMap.newKeySet())
.add(event.productId);
}
public void apply(StockUpdatedEvent event) {
ProductDto dto = products.get(event.productId);
if (dto != null) {
products.put(event.productId, dto.withStock(event.newStock));
}
}
/**
* 查询产品
*/
public ProductDto getById(String productId) {
return products.get(productId);
}
/**
* 搜索产品
*/
public List<ProductDto> search(String keyword, int page, int size) {
String keywordLower = keyword.toLowerCase();
return nameIndex.entrySet().stream()
.filter(e -> e.getKey().contains(keywordLower))
.flatMap(e -> e.getValue().stream())
.map(products::get)
.filter(Objects::nonNull)
.skip((long) page * size)
.limit(size)
.collect(Collectors.toList());
}
}
/**
* 产品 DTO
*/
record ProductDto(
String id,
String name,
Money price,
int stock,
long createdAt
) {
ProductDto withStock(int newStock) {
return new ProductDto(id, name, price, newStock, createdAt);
}
}
/**
* 产品命令处理器
*/
public class ProductCommandHandler implements CommandHandler<Void> {
private final ProductWriteModel writeModel;
public ProductCommandHandler(ProductWriteModel writeModel) {
this.writeModel = writeModel;
}
@Override
public CommandResult<Void> handle(Command command) {
List<Event> events = new ArrayList<>();
if (command instanceof CreateProductCommand cmd) {
ProductAggregate product = writeModel.getOrCreate(cmd.productId);
events = product.create(cmd.name, cmd.description, cmd.price, cmd.stock);
} else if (command instanceof UpdateStockCommand cmd) {
ProductAggregate product = writeModel.get(cmd.productId);
if (product == null) {
throw new IllegalArgumentException("Product not found: " + cmd.productId);
}
events = product.updateStock(cmd.delta);
}
return CommandResult.of(null, events.toArray(new Event[0]));
}
}
/**
* 产品查询处理器
*/
public class ProductQueryHandler {
private final ProductReadModel readModel;
public ProductQueryHandler(ProductReadModel readModel) {
this.readModel = readModel;
}
public ProductDto handle(GetProductQuery query) {
return readModel.getById(query.productId);
}
public List<ProductDto> handle(SearchProductsQuery query) {
return readModel.search(query.keyword, query.page, query.size);
}
}
// CQRS 完整演示
public class CQRSDemo {
public static void main(String[] args) throws InterruptedException {
// 初始化
CQRS cqrs = new CQRS();
ProductWriteModel writeModel = new ProductWriteModel();
ProductReadModel readModel = new ProductReadModel();
// 注册命令处理器
cqrs.registerCommandHandler(CreateProductCommand.class,
new ProductCommandHandler(writeModel));
cqrs.registerCommandHandler(UpdateStockCommand.class,
new ProductCommandHandler(writeModel));
// 注册事件处理器(同步到读模型)
cqrs.registerEventHandler(ProductCreatedEvent.class,
event -> readModel.apply((ProductCreatedEvent) event));
cqrs.registerEventHandler(StockUpdatedEvent.class,
event -> readModel.apply((StockUpdatedEvent) event));
// 注册查询处理器
ProductQueryHandler queryHandler = new ProductQueryHandler(readModel);
cqrs.registerQueryHandler(GetProductQuery.class,
query -> queryHandler.handle((GetProductQuery) query));
// 执行命令
System.out.println("=== Executing Commands ===");
cqrs.send(new CreateProductCommand(
"prod-001",
"iPhone 15 Pro",
"Apple flagship phone",
new Money(899900, "CNY"),
100
));
cqrs.send(new UpdateStockCommand("prod-001", -5));
cqrs.send(new UpdateStockCommand("prod-001", 10));
// 等待事件处理
Thread.sleep(500);
// 执行查询
System.out.println("\n=== Executing Queries ===");
ProductDto product = cqrs.query(new GetProductQuery("prod-001"));
System.out.println("Product: " + product.name());
System.out.println("Stock: " + product.stock());
System.out.println("Price: " + product.price().amount() / 100.0 + " " + product.price().currency());
}
}
2.4 补偿事务(Saga 模式)
原理说明
Saga 模式将长事务分解为一系列本地事务:
- 每个步骤有对应的补偿操作
- 某步骤失败时,反向执行补偿操作
- 保证最终一致性
Java 实现
import java.util.*;
import java.util.concurrent.*;
/**
* Saga 编排器
*/
public class SagaOrchestrator {
private final String sagaId;
private final List<SagaStep> steps;
private final List<SagaStep> completedSteps;
private SagaStatus status;
private final SagaLog sagaLog;
public enum SagaStatus {
RUNNING, COMPLETED, COMPENSATING, COMPENSATED, FAILED
}
public SagaOrchestrator(String sagaId, SagaLog sagaLog) {
this.sagaId = sagaId;
this.steps = new ArrayList<>();
this.completedSteps = new ArrayList<>();
this.status = SagaStatus.RUNNING;
this.sagaLog = sagaLog;
}
/**
* 添加步骤
*/
public SagaOrchestrator addStep(String name,
Callable<SagaResult> action,
Callable<SagaResult> compensation) {
steps.add(new SagaStep(name, action, compensation));
return this;
}
/**
* 执行 Saga
*/
public SagaResult execute() {
sagaLog.logStart(sagaId, steps.size());
for (int i = 0; i < steps.size(); i++) {
SagaStep step = steps.get(i);
try {
sagaLog.logStepStart(sagaId, step.name, i);
SagaResult result = step.action.call();
if (result.success) {
completedSteps.add(step);
sagaLog.logStepComplete(sagaId, step.name, i);
System.out.println("[Saga-" + sagaId + "] Step completed: " + step.name);
} else {
// 步骤失败,开始补偿
sagaLog.logStepFailed(sagaId, step.name, i, result.message);
compensate();
return SagaResult.failure("Step failed: " + step.name);
}
} catch (Exception e) {
sagaLog.logStepError(sagaId, step.name, i, e.getMessage());
compensate();
return SagaResult.failure("Step error: " + e.getMessage());
}
}
status = SagaStatus.COMPLETED;
sagaLog.logComplete(sagaId);
return SagaResult.success();
}
/**
* 执行补偿
*/
private void compensate() {
status = SagaStatus.COMPENSATING;
System.out.println("[Saga-" + sagaId + "] Starting compensation...");
// 逆序执行补偿
for (int i = completedSteps.size() - 1; i >= 0; i--) {
SagaStep step = completedSteps.get(i);
try {
sagaLog.logCompensationStart(sagaId, step.name, i);
SagaResult result = step.compensation.call();
if (result.success) {
sagaLog.logCompensationComplete(sagaId, step.name, i);
System.out.println("[Saga-" + sagaId + "] Compensated: " + step.name);
} else {
sagaLog.logCompensationFailed(sagaId, step.name, i, result.message);
System.err.println("[Saga-" + sagaId + "] Compensation failed: " + step.name);
}
} catch (Exception e) {
sagaLog.logCompensationError(sagaId, step.name, i, e.getMessage());
System.err.println("[Saga-" + sagaId + "] Compensation error: " + e.getMessage());
}
}
status = SagaStatus.COMPENSATED;
}
public SagaStatus getStatus() {
return status;
}
/**
* Saga 步骤
*/
static class SagaStep {
final String name;
final Callable<SagaResult> action;
final Callable<SagaResult> compensation;
SagaStep(String name, Callable<SagaResult> action, Callable<SagaResult> compensation) {
this.name = name;
this.action = action;
this.compensation = compensation;
}
}
}
/**
* Saga 结果
*/
class SagaResult {
final boolean success;
final String message;
final Object data;
SagaResult(boolean success, String message, Object data) {
this.success = success;
this.message = message;
this.data = data;
}
static SagaResult success() {
return new SagaResult(true, "Success", null);
}
static SagaResult success(Object data) {
return new SagaResult(true, "Success", data);
}
static SagaResult failure(String message) {
return new SagaResult(false, message, null);
}
}
/**
* Saga 日志
*/
class SagaLog {
private final List<SagaLogEntry> logs = new CopyOnWriteArrayList<>();
void logStart(String sagaId, int totalSteps) {
add(sagaId, "SAGA_START", "Total steps: " + totalSteps);
}
void logStepStart(String sagaId, String stepName, int stepIndex) {
add(sagaId, "STEP_START", "Step " + stepIndex + ": " + stepName);
}
void logStepComplete(String sagaId, String stepName, int stepIndex) {
add(sagaId, "STEP_COMPLETE", "Step " + stepIndex + ": " + stepName);
}
void logStepFailed(String sagaId, String stepName, int stepIndex, String reason) {
add(sagaId, "STEP_FAILED", "Step " + stepIndex + ": " + stepName + " - " + reason);
}
void logStepError(String sagaId, String stepName, int stepIndex, String error) {
add(sagaId, "STEP_ERROR", "Step " + stepIndex + ": " + stepName + " - " + error);
}
void logCompensationStart(String sagaId, String stepName, int stepIndex) {
add(sagaId, "COMPENSATION_START", "Step " + stepIndex + ": " + stepName);
}
void logCompensationComplete(String sagaId, String stepName, int stepIndex) {
add(sagaId, "COMPENSATION_COMPLETE", "Step " + stepIndex + ": " + stepName);
}
void logCompensationFailed(String sagaId, String stepName, int stepIndex, String reason) {
add(sagaId, "COMPENSATION_FAILED", "Step " + stepIndex + ": " + stepName + " - " + reason);
}
void logCompensationError(String sagaId, String stepName, int stepIndex, String error) {
add(sagaId, "COMPENSATION_ERROR", "Step " + stepIndex + ": " + stepName + " - " + error);
}
void logComplete(String sagaId) {
add(sagaId, "SAGA_COMPLETE", "Saga completed successfully");
}
private void add(String sagaId, String eventType, String message) {
logs.add(new SagaLogEntry(sagaId, eventType, message, System.currentTimeMillis()));
}
public List<SagaLogEntry> getLogs() {
return new ArrayList<>(logs);
}
}
record SagaLogEntry(String sagaId, String eventType, String message, long timestamp) {}
/**
* 订单处理 Saga 示例
*/
public class OrderProcessingSaga {
private final InventoryService inventoryService;
private final PaymentService paymentService;
private final ShippingService shippingService;
private final NotificationService notificationService;
public OrderProcessingSaga() {
this.inventoryService = new InventoryService();
this.paymentService = new PaymentService();
this.shippingService = new ShippingService();
this.notificationService = new NotificationService();
}
/**
* 创建订单处理 Saga
*/
public SagaOrchestrator createOrderSaga(String orderId, String customerId,
List<OrderItem> items, Money totalAmount) {
SagaLog sagaLog = new SagaLog();
SagaOrchestrator saga = new SagaOrchestrator("order-" + orderId, sagaLog);
// 步骤1:预留库存
saga.addStep(
"ReserveInventory",
() -> inventoryService.reserveInventory(orderId, items),
() -> inventoryService.releaseInventory(orderId, items)
);
// 步骤2:扣款
saga.addStep(
"ProcessPayment",
() -> paymentService.processPayment(orderId, customerId, totalAmount),
() -> paymentService.refundPayment(orderId, customerId, totalAmount)
);
// 步骤3:创建配送单
saga.addStep(
"CreateShipment",
() -> shippingService.createShipment(orderId, customerId, items),
() -> shippingService.cancelShipment(orderId)
);
// 步骤4:发送通知
saga.addStep(
"SendNotification",
() -> notificationService.sendOrderConfirmation(orderId, customerId),
() -> notificationService.sendOrderCancellation(orderId, customerId)
);
return saga;
}
}
// 服务接口
class InventoryService {
public SagaResult reserveInventory(String orderId, List<OrderItem> items) {
System.out.println("[Inventory] Reserved inventory for order: " + orderId);
return SagaResult.success();
}
public SagaResult releaseInventory(String orderId, List<OrderItem> items) {
System.out.println("[Inventory] Released inventory for order: " + orderId);
return SagaResult.success();
}
}
class PaymentService {
public SagaResult processPayment(String orderId, String customerId, Money amount) {
System.out.println("[Payment] Processed payment: " + amount.amount() / 100.0 +
" " + amount.currency());
return SagaResult.success();
}
public SagaResult refundPayment(String orderId, String customerId, Money amount) {
System.out.println("[Payment] Refunded payment: " + amount.amount() / 100.0 +
" " + amount.currency());
return SagaResult.success();
}
}
class ShippingService {
public SagaResult createShipment(String orderId, String customerId, List<OrderItem> items) {
System.out.println("[Shipping] Created shipment for order: " + orderId);
return SagaResult.success();
}
public SagaResult cancelShipment(String orderId) {
System.out.println("[Shipping] Cancelled shipment for order: " + orderId);
return SagaResult.success();
}
}
class NotificationService {
public SagaResult sendOrderConfirmation(String orderId, String customerId) {
System.out.println("[Notification] Sent confirmation for order: " + orderId);
return SagaResult.success();
}
public SagaResult sendOrderCancellation(String orderId, String customerId) {
System.out.println("[Notification] Sent cancellation for order: " + orderId);
return SagaResult.success();
}
}
// 使用示例
public class SagaDemo {
public static void main(String[] args) {
OrderProcessingSaga orderSaga = new OrderProcessingSaga();
List<OrderItem> items = List.of(
new OrderItem("prod-001", "iPhone 15", 1, new Money(699900, "CNY")),
new OrderItem("prod-002", "Case", 1, new Money(29900, "CNY"))
);
Money totalAmount = new Money(729800, "CNY");
SagaOrchestrator saga = orderSaga.createOrderSaga(
"order-001",
"customer-001",
items,
totalAmount
);
System.out.println("=== Executing Order Saga ===\n");
SagaResult result = saga.execute();
System.out.println("\n=== Saga Result ===");
System.out.println("Status: " + saga.getStatus());
System.out.println("Success: " + result.success);
if (!result.success) {
System.out.println("Message: " + result.message);
}
}
}
三、实战应用场景
3.1 跨系统数据同步
/**
* 跨系统数据同步管理器
*/
public class DataSynchronizationManager {
private final EventualConsistencyMessageQueue messageQueue;
private final SyncStateStore stateStore;
public DataSynchronizationManager() {
this.messageQueue = new EventualConsistencyMessageQueue("data-sync");
this.stateStore = new SyncStateStore();
setupSyncHandlers();
}
/**
* 同步数据
*/
public void sync(String sourceSystem, String targetSystem,
String entityType, String entityId, SyncOperation operation,
Map<String, Object> data) {
SyncMessage message = new SyncMessage(
UUID.randomUUID().toString(),
sourceSystem,
targetSystem,
entityType,
entityId,
operation,
data,
System.currentTimeMillis()
);
// 记录同步状态
stateStore.recordSync(message.id, SyncStatus.PENDING);
// 发送消息
messageQueue.send("data-sync", message);
}
/**
* 设置同步处理器
*/
private void setupSyncHandlers() {
messageQueue.subscribe(new EventualConsistencyMessageQueue.MessageConsumer() {
@Override
public String getTopic() {
return "data-sync";
}
@Override
public void consume(EventualConsistencyMessageQueue.Message msg) throws Exception {
SyncMessage syncMsg = (SyncMessage) msg.payload;
try {
// 执行同步
executeSync(syncMsg);
// 更新状态
stateStore.updateSync(syncMsg.id, SyncStatus.COMPLETED);
} catch (Exception e) {
stateStore.updateSync(syncMsg.id, SyncStatus.FAILED);
throw e;
}
}
});
}
private void executeSync(SyncMessage message) {
System.out.println("[Sync] " + message.operation + " " + message.entityType +
" " + message.entityId + " from " + message.sourceSystem +
" to " + message.targetSystem);
}
}
enum SyncOperation {
CREATE, UPDATE, DELETE
}
enum SyncStatus {
PENDING, PROCESSING, COMPLETED, FAILED, RETRYING
}
record SyncMessage(
String id,
String sourceSystem,
String targetSystem,
String entityType,
String entityId,
SyncOperation operation,
Map<String, Object> data,
long timestamp
) {}
class SyncStateStore {
private final Map<String, SyncState> states = new ConcurrentHashMap<>();
void recordSync(String id, SyncStatus status) {
states.put(id, new SyncState(id, status, System.currentTimeMillis()));
}
void updateSync(String id, SyncStatus status) {
SyncState state = states.get(id);
if (state != null) {
states.put(id, new SyncState(id, status, state.createdAt));
}
}
}
record SyncState(String id, SyncStatus status, long createdAt) {}
四、总结与最佳实践
最终一致性方案对比
| 方案 | 一致性保证 | 延迟 | 复杂度 | 适用场景 |
|---|---|---|---|---|
| 消息队列 | 最终一致 | 秒级 | 低 | 数据同步、异步处理 |
| 事件溯源 | 最终一致 | 毫秒级 | 高 | 审计日志、状态重建 |
| CQRS | 最终一致 | 毫秒级 | 中 | 读写分离、复杂查询 |
| Saga | 最终一致 | 秒级 | 中 | 长事务、跨服务操作 |
最佳实践
- 幂等设计:所有操作必须幂等,支持重试
- 消息持久化:消息必须持久化,防止丢失
- 监控与告警:监控同步延迟和失败率
- 死信处理:无法处理的消息进入死信队列
- 补偿机制:设计补偿操作处理异常情况
五、思考与练习
思考题
- 基础题:最终一致性与强一致性的核心区别是什么?在什么场景下应该选择最终一致性?
- 进阶题:Saga模式中的补偿事务如何保证执行成功?如果补偿事务也失败了怎么办?
- 实战题:设计一个电商订单系统的数据一致性方案,考虑订单创建、库存扣减、支付、物流等环节。
编程练习
练习:实现一个基于事件溯源的银行账户系统,要求:
- 支持开户、存款、取款、转账操作
- 所有操作以事件形式持久化
- 支持账户状态重建和事件回放
- 实现幂等性保证
章节关联
- 前置章节:《强一致性方案详解》
- 后续章节:《数据同步方案详解》
- 扩展阅读:Martin Fowler《Event Sourcing》、《Building Event-Driven Microservices》
📝 下一章预告
下一章将深入数据同步方案,学习主从复制、增量同步、双向同步等技术,掌握分布式系统中数据流转的核心机制。
本章完