分布式事务介绍

selrain001
291 阅读4分钟

分布式事务

介绍

随着业务的发展越来越快，数据库的压力越来越大，这时候就需要对数据库拆分（在数据库设计之初，出于未来业务的考量，也有可能提前拆分），在单库模式下，数据的一致性依赖JDBC的事务，分库之后，就需要依靠分布式事务来解决。

实现方式

1、最大努力送达

最大努力送达相信对于数据库的操作最终一定会成功，其原理就是创建一个事务库，对数据操作监听，记录事务日志，对失败的操作进行重试，最后清理事务日志。

enter image description here

执行过程有 四种 情况：

1、【红线】执行成功

2、【棕线】执行失败，同步重试成功

3、【粉线】执行失败，同步重试失败，异步重试成功

4、【绿线】执行失败，同步重试失败，异步重试失败，事务日志保留

整个过程如下：

  • SoftTransactionManager：事务管理器 getTransaction：实例化具体的事务实现（目前有2种：最大努力送达BEDSoftTransaction、TCCSoftTransaction） init：利用guava的EventBus注册监听数据库DML操作的监听器；创建事务日志表
  • BestEffortsDeliveryListener：事务监听器 listen：监听DML操作，维护事务表日志
  • RdbTransactionLogStorage 事务日志存储器 add：日志表添加一条数据 remove：删除数据
  • 异步送达作业

下面单独看看每个组件：

SoftTransactionManager

负责对柔性事务配置( SoftTransactionConfiguration ) 、柔性事务( AbstractSoftTransaction )的管理。

#调用init方法初始化管理器

public void init() throws SQLException {
    //guava注册监听器
    EventBusInstance.getInstance().register(new BestEffortsDeliveryListener());
    if (TransactionLogDataSourceType.RDB == transactionConfig.getStorageType()) {
        Preconditions.checkNotNull(transactionConfig.getTransactionLogDataSource());
        //创建事务表
        createTable();
    }
}
private void createTable() throws SQLException {
    String dbSchema = "CREATE TABLE IF NOT EXISTS `transaction_log` ("
            + "`id` VARCHAR(40) NOT NULL, "
            + "`transaction_type` VARCHAR(30) NOT NULL, "
            + "`data_source` VARCHAR(255) NOT NULL, "
            + "`sql` TEXT NOT NULL, "
            + "`parameters` TEXT NOT NULL, "
            + "`creation_time` LONG NOT NULL, "
            + "`async_delivery_try_times` INT NOT NULL DEFAULT 0, "
            + "PRIMARY KEY (`id`));";
    try (
            Connection conn = transactionConfig.getTransactionLogDataSource().getConnection();
            PreparedStatement preparedStatement = conn.prepareStatement(dbSchema)) {
        preparedStatement.executeUpdate();
    }
}

#调用getTransaction方法获取事务实现

public AbstractSoftTransaction getTransaction(final SoftTransactionType type) {
    AbstractSoftTransaction result;
    switch (type) {
            //最大努力送达
        case BestEffortsDelivery: 
            result = new BEDSoftTransaction();
            break;
            //TCC 目前尚未实现
        case TryConfirmCancel:
            result = new TCCSoftTransaction();
            break;
        default: 
            throw new UnsupportedOperationException(type.toString());
    }
    // TODO don't support nested transaction, should configurable in future
    if (getCurrentTransaction().isPresent()) {
        throw new UnsupportedOperationException("Cannot support nested transaction.");
    }
    ExecutorDataMap.getDataMap().put(TRANSACTION, result);
    ExecutorDataMap.getDataMap().put(TRANSACTION_CONFIG, transactionConfig);
    return result;
}

BestEffortsDeliveryListener

最大努力事务监听器，通过监听数据库操作，同步重试失败日志

@Subscribe
@AllowConcurrentEvents
public void listen(final DMLExecutionEvent event) {
    if (!isProcessContinuously()) {
        return;
    }
    SoftTransactionConfiguration transactionConfig = SoftTransactionManager.getCurrentTransactionConfiguration().get();
    TransactionLogStorage transactionLogStorage = TransactionLogStorageFactory.createTransactionLogStorage(transactionConfig.buildTransactionLogDataSource());
    BEDSoftTransaction bedSoftTransaction = (BEDSoftTransaction) SoftTransactionManager.getCurrentTransaction().get();
    switch (event.getEventExecutionType()) {
        case BEFORE_EXECUTE:
            //TODO for batch SQL need split to 2-level records
            transactionLogStorage.add(new TransactionLog(event.getId(), bedSoftTransaction.getTransactionId(), bedSoftTransaction.getTransactionType(), 
                    event.getDataSource(), event.getSqlUnit().getSql(), event.getParameters(), System.currentTimeMillis(), 0));
            return;
        case EXECUTE_SUCCESS: 
            transactionLogStorage.remove(event.getId());
            return;
        case EXECUTE_FAILURE: 
            boolean deliverySuccess = false;
            for (int i = 0; i < transactionConfig.getSyncMaxDeliveryTryTimes(); i++) {
                if (deliverySuccess) {
                    return;
                }
                boolean isNewConnection = false;
                Connection conn = null;
                PreparedStatement preparedStatement = null;
                try {
                    conn = bedSoftTransaction.getConnection().getConnection(event.getDataSource());
                    if (!isValidConnection(conn)) {
                        bedSoftTransaction.getConnection().release(conn);
                        conn = bedSoftTransaction.getConnection().getConnection(event.getDataSource());
                        isNewConnection = true;
                    }
                    preparedStatement = conn.prepareStatement(event.getSqlUnit().getSql());
                    //TODO for batch event need split to 2-level records
                    for (int parameterIndex = 0; parameterIndex < event.getParameters().size(); parameterIndex++) {
                        preparedStatement.setObject(parameterIndex + 1, event.getParameters().get(parameterIndex));
                    }
                    preparedStatement.executeUpdate();
                    deliverySuccess = true;
                    transactionLogStorage.remove(event.getId());
                } catch (final SQLException ex) {
                    log.error(String.format("Delivery times %s error, max try times is %s", i + 1, transactionConfig.getSyncMaxDeliveryTryTimes()), ex);
                } finally {
                    close(isNewConnection, conn, preparedStatement);
                }
            }
            return;
        default: 
            throw new UnsupportedOperationException(event.getEventExecutionType().toString());
    }
}
  • BestEffortsDeliveryListener 通过 EventBus 实现监听 SQL 的执行。Sharding-JDBC 如何实现 EventBus 的，请参考芋艿大神的《Sharding-JDBC 源码分析 —— SQL 执行》分析
  • 调用 #isProcessContinuously() 方法判断是否处于最大努力送达型事务中，当且仅当处于该状态才进行监听事件处理
  • SQL 执行前，插入事务日志
  • SQL 执行成功，移除事务日志
  • SQL 执行失败，根据柔性事务配置( SoftTransactionConfiguration )同步的事务送达的最大尝试次数( syncMaxDeliveryTryTimes )进行多次重试直到成功。

TransactionLogStorage

柔性事务执行过程中，会通过事务日志( TransactionLog ) 记录每条 SQL 执行状态： 目前有2中实现：

  • MemoryTransactionLogStorage：基于内存的事务日志存储器。主要用于开发测试，生产环境下不要使用。
  • RdbTransactionLogStorage ：基于数据库的事务日志存储器。

这里主要讲讲RdbTransactionLogStorage的实现：

#add
public void add(TransactionLog transactionLog) {
    String sql = "INSERT INTO `transaction_log` (`id`, `transaction_type`, `data_source`, `sql`, `parameters`, `creation_time`) VALUES (?, ?, ?, ?, ?, ?);";

    try {
        Connection conn = this.dataSource.getConnection();
        Throwable var4 = null;

        try {
            PreparedStatement preparedStatement = conn.prepareStatement(sql);
            Throwable var6 = null;

            try {
                preparedStatement.setString(1, transactionLog.getId());
                preparedStatement.setString(2, SoftTransactionType.BestEffortsDelivery.name());
                preparedStatement.setString(3, transactionLog.getDataSource());
                preparedStatement.setString(4, transactionLog.getSql());
                preparedStatement.setString(5, (new Gson()).toJson(transactionLog.getParameters()));
                preparedStatement.setLong(6, transactionLog.getCreationTime());
                preparedStatement.executeUpdate();
            } catch (Throwable var31) {
                var6 = var31;
                throw var31;
            } finally {
                if (preparedStatement != null) {
                    if (var6 != null) {
                        try {
                            preparedStatement.close();
                        } catch (Throwable var30) {
                            var6.addSuppressed(var30);
                        }
                    } else {
                        preparedStatement.close();
                    }
                }

            }
        } catch (Throwable var33) {
            var4 = var33;
            throw var33;
        } finally {
            conn->close
        }

    } catch (SQLException var35) {
        throw new TransactionLogStorageException(var35);
    }
}
#remove
public void remove(String id) {
    String sql = "DELETE FROM `transaction_log` WHERE `id`=?;";

    try {
        Connection conn = this.dataSource.getConnection();
        Throwable var4 = null;

        try {
            PreparedStatement preparedStatement = conn.prepareStatement(sql);
            Throwable var6 = null;

            try {
                preparedStatement.setString(1, id);
                preparedStatement.executeUpdate();
            } catch (Throwable var31) {
                var6 = var31;
                throw var31;
            } finally {
                if (preparedStatement != null) {
                    if (var6 != null) {
                        try {
                            preparedStatement.close();
                        } catch (Throwable var30) {
                            var6.addSuppressed(var30);
                        }
                    } else {
                        preparedStatement.close();
                    }
                }

            }
        } catch (Throwable var33) {
            var4 = var33;
            throw var33;
        } finally {
            conn->close
        }

    } catch (SQLException var35) {
        throw new TransactionLogStorageException(var35);
    }
}

BestEffortsDeliveryJob

BestEffortsDeliveryJob 所在 Maven 项目为 sharding-jdbc-transaction-async-job，基于当当开源的 Elastic-Job 实现。如下是官方对该 Maven 项目的简要说明： 由于柔性事务采用异步尝试，需要部署独立的作业和Zookeeper。sharding-jdbc-transaction采用elastic-job实现的sharding-jdbc-transaction-async-job，通过简单配置即可启动高可用作业异步送达柔性事务，启动脚本为start.sh。

最后，欢迎follow我的个人订阅号：

avatar
文章
阅读
粉丝