摘要:本文主要介绍SpringBoot+ShardingSphere融合的快速使用指南,主要包括读写分离、数据分片模式。
简介
Apache ShardingSphere 是一款开源的分布式数据库生态项目,由 JDBC 和 Proxy 两款产品组成。其核心采用微内核+可插拔架构,通过插件开放扩展功能。它提供多源异构数据库增强平台,进而围绕其上层构建生态。
Apache ShardingSphere 设计哲学为 Database Plus,旨在构建异构数据库上层的标准和生态。它关注如何充分合理地利用数据库的计算和存储能力,而并非实现一个全新的数据库。它站在数据库的上层视角,关注它们之间的协作多于数据库自身。
基于SpringBoot的入门指南
基础准备
mysql数据库准备
我创建了一个
user库,准备了一张address表
CREATE TABLE `address` (
`id` bigint NOT NULL,
`code` varchar(255) DEFAULT NULL,
`name` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
pom.xml
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<shardingsphere.version>5.2.0</shardingsphere.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<dependency>
<groupId>com.baomidou</groupId>
<artifactId>mybatis-plus-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc-core-spring-boot-starter</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
</dependencies>
ShardingSphereApplication启动类
@SpringBootApplication
@MapperScan("com.github.huzhihui.shardingsphere.mapper")
public class ShardingSphereApplication {
public static void main(String[] args) {
SpringApplication.run(ShardingSphereApplication.class, args);
}
}
AddressEntity实体类
@Data
@TableName(value = "address")
public class AddressEntity {
private Long id;
private String code;
private String name;
}
AddressMapper mybatis数据库操作接口
public interface AddressMapper extends BaseMapper<AddressEntity> {
}
AddressController 入口访问类
@RestController
@RequestMapping(value = "address")
public class AddressController {
@Autowired
private AddressMapper addressMapper;
@GetMapping(value = "queryById")
public AddressEntity queryById(Long id){
return addressMapper.selectById(id);
}
}
读写分离
案例读库与写库,使用同一个数据库来演示,我这里没有做
mysql主从了,如果想自己测试看看读写是不是不同的数据库,则可以自己创建2个数据库,里面有不同的数据来演示。
application.yml配置
本案例使用的是静态读写分离配置,一般这样的配置就够用了
spring:
shardingsphere:
datasource:
names: user-master01,user-slave01
user-master01:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://192.168.137.180:3306/user?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=false&rewriteBatchedStatements=true
username: root
password: root
maximum-pool-size: 10
minimum-idle: 4
idle-timeout: 30000
max-lifetime: 1800000
connection-timeout: 30000
user-slave01:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://192.168.137.180:3306/user?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=false&rewriteBatchedStatements=true
username: root
password: root
maximum-pool-size: 10
minimum-idle: 4
idle-timeout: 30000
max-lifetime: 1800000
connection-timeout: 30000
rules:
readwrite-splitting:
data-sources:
user-datasource:
static-strategy:
write-data-source-name: user-master01
read-data-source-names: user-slave01
load-balancer-name: round_robin
load-balancers:
round_robin:
type: ROUND_ROBIN
props:
sql-show: true
mybatis-plus:
configuration:
log-impl: org.apache.ibatis.logging.stdout.StdOutImpl
配置注意事项
- 读写分离的数据定义
user-datasource是自己随便定义的,因为我上面是user库,如果我上面配置了多库,配置了user和order库,那么这里要增加读写分离配置,就复制出来一个order-datasource,来配置order库的读写分离配置。
- 负载算法配置
使用上面的
round_robin即可,如果想使用其他配置,自己翻阅文档,按理说这样配置足够日常使用。
访问结果
访问
http://localhost:8080/address/queryById?id=1,会显示是从user-slave01的数据源配置读取的
2024-04-10 08:40:22.149 INFO 13428 --- [nio-8080-exec-4] ShardingSphere-SQL : Logic SQL: SELECT id,code,name FROM address WHERE id=?
2024-04-10 08:40:22.149 INFO 13428 --- [nio-8080-exec-4] ShardingSphere-SQL : SQLStatement: MySQLSelectStatement(super=SelectStatement(super=AbstractSQLStatement(parameterCount=1, parameterMarkerSegments=[ParameterMarkerExpressionSegment(startIndex=42, stopIndex=42, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty)], commentSegments=[]), projections=ProjectionsSegment(startIndex=7, stopIndex=18, projections=[ColumnProjectionSegment(column=ColumnSegment(startIndex=7, stopIndex=8, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), alias=Optional.empty), ColumnProjectionSegment(column=ColumnSegment(startIndex=10, stopIndex=13, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), alias=Optional.empty), ColumnProjectionSegment(column=ColumnSegment(startIndex=15, stopIndex=18, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty), alias=Optional.empty)], distinctRow=false), from=SimpleTableSegment(tableName=TableNameSegment(startIndex=25, stopIndex=31, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), where=Optional[WhereSegment(startIndex=33, stopIndex=42, expr=BinaryOperationExpression(startIndex=39, stopIndex=42, left=ColumnSegment(startIndex=39, stopIndex=40, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), right=ParameterMarkerExpressionSegment(startIndex=42, stopIndex=42, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), operator==, text=id=?))], groupBy=Optional.empty, having=Optional.empty, orderBy=Optional.empty, combines=[]), table=Optional.empty, limit=Optional.empty, lock=Optional.empty, window=Optional.empty)
2024-04-10 08:40:22.149 INFO 13428 --- [nio-8080-exec-4] ShardingSphere-SQL : Actual SQL: user-slave01 ::: SELECT id,code,name FROM address WHERE id=? ::: [1]
<== Columns: id, code, name
<== Row: 1, A, A
<== Total: 1
数据分片
广播表
用于在多个库中需要进行联表操作的基础数据表,读多写少的情况
- 提前创建2个数据库
user-02,user-02 - 创建
dict表
CREATE TABLE `dict` (
`id` bigint NOT NULL,
`code` varchar(255) DEFAULT NULL,
`name` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
Entity实体类和Mapper接口按照上面的照抄下来
@Data
@TableName(value = "dict")
public class DictEntity {
private Long id;
private String code;
private String name;
}
public interface DictMapper extends BaseMapper<DictEntity> {
}
application.yml配置
spring:
shardingsphere:
datasource:
names: user-01,user-02
user-01:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://192.168.137.180:3306/user-01?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=false&rewriteBatchedStatements=true
username: root
password: root
maximum-pool-size: 10
minimum-idle: 4
idle-timeout: 30000
max-lifetime: 1800000
connection-timeout: 30000
user-02:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://192.168.137.180:3306/user-02?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=false&rewriteBatchedStatements=true
username: root
password: root
maximum-pool-size: 10
minimum-idle: 4
idle-timeout: 30000
max-lifetime: 1800000
connection-timeout: 30000
rules:
sharding:
broadcast-tables:
- dict
props:
sql-show: true
mybatis-plus:
configuration:
log-impl: org.apache.ibatis.logging.stdout.StdOutImpl
DictController
作为流量入口
@RestController
@RequestMapping(value = "dict")
public class DictController {
@Autowired
private DictMapper dictMapper;
@Transactional(rollbackFor = Exception.class)
@PostMapping(value = "add")
public Object add(@RequestBody DictEntity dictEntity){
dictMapper.insert(dictEntity);
return dictEntity;
}
@GetMapping(value = "getById")
public Object getById(Long id){
return dictMapper.selectById(id);
}
}
结果
2024-04-12 15:43:17.698 INFO 22236 --- [nio-8080-exec-2] ShardingSphere-SQL : Actual SQL: user-01 ::: INSERT INTO dict ( id,
code,
name ) VALUES (?, ?, ?) ::: [4, C-04, C-04名称]
2024-04-12 15:43:17.698 INFO 22236 --- [nio-8080-exec-2] ShardingSphere-SQL : Actual SQL: user-02 ::: INSERT INTO dict ( id,
code,
name ) VALUES (?, ?, ?) ::: [4, C-04, C-04名称]
注意事项
- 一定要加上事务注解
@Transactional(rollbackFor = Exception.class),否则会导致部分库写入成功部分库写入失败
单库分表
该案例的场景是单库模式下的某一种业务单据数量爆炸,突破了
500W数据量,则需要进行分表处理,加快查询效率。
application.yml配置
spring:
shardingsphere:
datasource:
names: user-01 #数据源名称,与下面定义的数据源相对应
user-01: #数据源名称,自己随便定义
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://192.168.137.180:3306/user-01?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=false&rewriteBatchedStatements=true
username: root
password: root
maximum-pool-size: 10
minimum-idle: 4
idle-timeout: 30000
max-lifetime: 1800000
connection-timeout: 30000
rules:
sharding:
broadcast-tables: #广播表
- dict
key-generators: #id生产算法
snowflake:
type: SNOWFLAKE
tables:
address: #规则名称,一般写具体的表
actual-data-nodes: user-01.address_$->{0..1} #处于真实的节点
table-strategy:
standard: #标准分片模式
sharding-column: id #分表列
sharding-algorithm-name: address-inline #对应下面定义的分片算法
sharding-algorithms:
address-inline: #算法名称,自己定义的
type: INLINE #行内表达式
props:
algorithm-expression: address_$->{id % 2} #分表取模2
props:
sql-show: true
mybatis-plus:
configuration:
log-impl: org.apache.ibatis.logging.stdout.StdOutImpl
AddressController入口类
@RestController
@RequestMapping(value = "address")
public class AddressController {
@Autowired
private AddressMapper addressMapper;
@PostMapping(value = "add")
public AddressEntity add(@RequestBody AddressEntity addressEntity){
addressMapper.insert(addressEntity);
return addressEntity;
}
@GetMapping(value = "queryById")
public AddressEntity queryById(Long id){
return addressMapper.selectById(id);
}
@GetMapping(value = "page")
public Page<AddressEntity> page(long pages, long size){
Page<AddressEntity> page = new Page<>(pages, size);
addressMapper.selectPage(page, Wrappers.lambdaQuery());
return page;
}
}
案例结果
- 数据新增案例
按照
id插入到了不同的表中
Creating a new SqlSession
SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@7b50270a] was not registered for synchronization because synchronization is not active
JDBC Connection [org.apache.shardingsphere.driver.jdbc.core.connection.ShardingSphereConnection@42cacf1c] will not be managed by Spring
==> Preparing: INSERT INTO address ( id, code, name ) VALUES ( ?, ?, ? )
==> Parameters: 1779113652820451329(Long), 0826(String), 四川-武胜-胜利镇(String)
2024-04-13 19:45:24.118 INFO 14960 --- [nio-8080-exec-8] ShardingSphere-SQL : Logic SQL: INSERT INTO address ( id,
code,
name ) VALUES ( ?,
?,
? )
2024-04-13 19:45:24.118 INFO 14960 --- [nio-8080-exec-8] ShardingSphere-SQL : SQLStatement: MySQLInsertStatement(super=InsertStatement(super=AbstractSQLStatement(parameterCount=3, parameterMarkerSegments=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)], commentSegments=[]), table=SimpleTableSegment(tableName=TableNameSegment(startIndex=12, stopIndex=18, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), insertColumns=Optional[InsertColumnsSegment(startIndex=21, stopIndex=38, columns=[ColumnSegment(startIndex=23, stopIndex=24, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=27, stopIndex=30, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=33, stopIndex=36, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty)])], insertSelect=Optional.empty, values=[InsertValuesSegment(startIndex=49, stopIndex=59, values=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)])]), setAssignment=Optional.empty, onDuplicateKeyColumns=Optional.empty)
2024-04-13 19:45:24.118 INFO 14960 --- [nio-8080-exec-8] ShardingSphere-SQL : Actual SQL: user-01 ::: INSERT INTO address_1 ( id,
code,
name ) VALUES (?, ?, ?) ::: [1779113652820451329, 0826, 四川-武胜-胜利镇]
<== Updates: 1
Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@7b50270a]
Creating a new SqlSession
SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@19b2cd26] was not registered for synchronization because synchronization is not active
JDBC Connection [org.apache.shardingsphere.driver.jdbc.core.connection.ShardingSphereConnection@17753060] will not be managed by Spring
==> Preparing: INSERT INTO address ( id, code, name ) VALUES ( ?, ?, ? )
==> Parameters: 1779113699846987778(Long), 0826(String), 四川-武胜-胜利镇(String)
2024-04-13 19:45:35.072 INFO 14960 --- [nio-8080-exec-9] ShardingSphere-SQL : Logic SQL: INSERT INTO address ( id,
code,
name ) VALUES ( ?,
?,
? )
2024-04-13 19:45:35.072 INFO 14960 --- [nio-8080-exec-9] ShardingSphere-SQL : SQLStatement: MySQLInsertStatement(super=InsertStatement(super=AbstractSQLStatement(parameterCount=3, parameterMarkerSegments=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)], commentSegments=[]), table=SimpleTableSegment(tableName=TableNameSegment(startIndex=12, stopIndex=18, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), insertColumns=Optional[InsertColumnsSegment(startIndex=21, stopIndex=38, columns=[ColumnSegment(startIndex=23, stopIndex=24, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=27, stopIndex=30, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=33, stopIndex=36, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty)])], insertSelect=Optional.empty, values=[InsertValuesSegment(startIndex=49, stopIndex=59, values=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)])]), setAssignment=Optional.empty, onDuplicateKeyColumns=Optional.empty)
2024-04-13 19:45:35.072 INFO 14960 --- [nio-8080-exec-9] ShardingSphere-SQL : Actual SQL: user-01 ::: INSERT INTO address_0 ( id,
code,
name ) VALUES (?, ?, ?) ::: [1779113699846987778, 0826, 四川-武胜-胜利镇]
<== Updates: 1
Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@19b2cd26]
- 数据查询
Creating a new SqlSession
SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@7b83b7c7] was not registered for synchronization because synchronization is not active
JDBC Connection [org.apache.shardingsphere.driver.jdbc.core.connection.ShardingSphereConnection@49608520] will not be managed by Spring
==> Preparing: SELECT COUNT(*) FROM address
==> Parameters:
2024-04-13 19:47:45.904 INFO 14960 --- [nio-8080-exec-2] ShardingSphere-SQL : Logic SQL: SELECT COUNT(*) FROM address
2024-04-13 19:47:45.904 INFO 14960 --- [nio-8080-exec-2] ShardingSphere-SQL : SQLStatement: MySQLSelectStatement(super=SelectStatement(super=AbstractSQLStatement(parameterCount=0, parameterMarkerSegments=[], commentSegments=[]), projections=ProjectionsSegment(startIndex=7, stopIndex=14, projections=[AggregationProjectionSegment(startIndex=7, stopIndex=14, type=COUNT, innerExpression=(*), parameters=[], alias=Optional.empty)], distinctRow=false), from=SimpleTableSegment(tableName=TableNameSegment(startIndex=21, stopIndex=27, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), where=Optional.empty, groupBy=Optional.empty, having=Optional.empty, orderBy=Optional.empty, combines=[]), table=Optional.empty, limit=Optional.empty, lock=Optional.empty, window=Optional.empty)
2024-04-13 19:47:45.904 INFO 14960 --- [nio-8080-exec-2] ShardingSphere-SQL : Actual SQL: user-01 ::: SELECT COUNT(*) FROM address_0 UNION ALL SELECT COUNT(*) FROM address_1
<== Columns: COUNT(*)
<== Row: 8
<== Total: 1
==> Preparing: SELECT id,code,name FROM address LIMIT ?
==> Parameters: 2(Long)
2024-04-13 19:47:45.943 INFO 14960 --- [nio-8080-exec-2] ShardingSphere-SQL : Logic SQL: SELECT id,code,name FROM address LIMIT ?
2024-04-13 19:47:45.943 INFO 14960 --- [nio-8080-exec-2] ShardingSphere-SQL : SQLStatement: MySQLSelectStatement(super=SelectStatement(super=AbstractSQLStatement(parameterCount=1, parameterMarkerSegments=[ParameterMarkerLimitValueSegment(super=LimitValueSegment(startIndex=41, stopIndex=41), parameterIndex=0)], commentSegments=[]), projections=ProjectionsSegment(startIndex=8, stopIndex=19, projections=[ColumnProjectionSegment(column=ColumnSegment(startIndex=8, stopIndex=9, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), alias=Optional.empty), ColumnProjectionSegment(column=ColumnSegment(startIndex=11, stopIndex=14, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), alias=Optional.empty), ColumnProjectionSegment(column=ColumnSegment(startIndex=16, stopIndex=19, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty), alias=Optional.empty)], distinctRow=false), from=SimpleTableSegment(tableName=TableNameSegment(startIndex=27, stopIndex=33, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), where=Optional.empty, groupBy=Optional.empty, having=Optional.empty, orderBy=Optional.empty, combines=[]), table=Optional.empty, limit=Optional[LimitSegment(startIndex=35, stopIndex=41, offset=Optional.empty, rowCount=Optional[ParameterMarkerLimitValueSegment(super=LimitValueSegment(startIndex=41, stopIndex=41), parameterIndex=0)])], lock=Optional.empty, window=Optional.empty)
2024-04-13 19:47:45.943 INFO 14960 --- [nio-8080-exec-2] ShardingSphere-SQL : Actual SQL: user-01 ::: SELECT id,code,name FROM address_0 LIMIT ? ::: [2]
2024-04-13 19:47:45.944 INFO 14960 --- [nio-8080-exec-2] ShardingSphere-SQL : Actual SQL: user-01 ::: SELECT id,code,name FROM address_1 LIMIT ? ::: [2]
<== Columns: id, code, name
<== Row: 1779107432797605890, 0826, 四川-武胜-胜利镇
<== Row: 1779107453500690434, 0826, 四川-武胜-胜利镇
<== Total: 2
Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@7b83b7c7]
分库分表案例
一般大型项目用这种模式。下面的案例使用的是
user-01,user-02数据库
user-01库,拥有address_0,address_1表user-02库,拥有address_2,address_3表
application.yml
spring:
shardingsphere:
datasource:
names: user-01,user-02
user-01:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://192.168.137.180:3306/user-01?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=false&rewriteBatchedStatements=true
username: root
password: root
maximum-pool-size: 10
minimum-idle: 4
idle-timeout: 30000
max-lifetime: 1800000
connection-timeout: 30000
user-02:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbc-url: jdbc:mysql://192.168.137.180:3306/user-02?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=false&rewriteBatchedStatements=true
username: root
password: root
maximum-pool-size: 10
minimum-idle: 4
idle-timeout: 30000
max-lifetime: 1800000
connection-timeout: 30000
rules:
sharding:
key-generators:
snowflake:
type: SNOWFLAKE
binding-tables:
- address
tables:
address:
actual-data-nodes: user-01.address_$->{0..1},user-02.address_$->{2..3},
database-strategy:
standard:
sharding-column: id
sharding-algorithm-name: database-user-inline
table-strategy:
standard:
sharding-column: id
sharding-algorithm-name: address-inline
sharding-algorithms:
database-user-inline:
type: INLINE #这种表达式需要groovy语法的
props:
algorithm-expression: user-0$->{id % 4 / 2 as int + 1} #这里写法是让取模为0和1的落入1数据库,2和3的落入2数据库
address-inline:
type: INLINE
props:
algorithm-expression: address_$->{id % 4}
props:
sql-show: true
访问结果
- 插入数据
2024-04-13 20:35:23.987 INFO 28024 --- [io-8080-exec-10] ShardingSphere-SQL : SQLStatement: MySQLInsertStatement(super=InsertStatement(super=AbstractSQLStatement(parameterCount=3, parameterMarkerSegments=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)], commentSegments=[]), table=SimpleTableSegment(tableName=TableNameSegment(startIndex=12, stopIndex=18, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), insertColumns=Optional[InsertColumnsSegment(startIndex=21, stopIndex=38, columns=[ColumnSegment(startIndex=23, stopIndex=24, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=27, stopIndex=30, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=33, stopIndex=36, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty)])], insertSelect=Optional.empty, values=[InsertValuesSegment(startIndex=49, stopIndex=59, values=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)])]), setAssignment=Optional.empty, onDuplicateKeyColumns=Optional.empty)
2024-04-13 20:35:23.988 INFO 28024 --- [io-8080-exec-10] ShardingSphere-SQL : Actual SQL: user-01 ::: INSERT INTO address_0 ( id,
code,
name ) VALUES (?, ?, ?) ::: [0, 0826, 四川-武胜-胜利镇]
2024-04-13 20:35:29.999 INFO 28024 --- [nio-8080-exec-1] ShardingSphere-SQL : Logic SQL: INSERT INTO address ( id,
code,
name ) VALUES ( ?,
?,
? )
2024-04-13 20:35:29.999 INFO 28024 --- [nio-8080-exec-1] ShardingSphere-SQL : SQLStatement: MySQLInsertStatement(super=InsertStatement(super=AbstractSQLStatement(parameterCount=3, parameterMarkerSegments=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)], commentSegments=[]), table=SimpleTableSegment(tableName=TableNameSegment(startIndex=12, stopIndex=18, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), insertColumns=Optional[InsertColumnsSegment(startIndex=21, stopIndex=38, columns=[ColumnSegment(startIndex=23, stopIndex=24, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=27, stopIndex=30, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=33, stopIndex=36, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty)])], insertSelect=Optional.empty, values=[InsertValuesSegment(startIndex=49, stopIndex=59, values=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)])]), setAssignment=Optional.empty, onDuplicateKeyColumns=Optional.empty)
2024-04-13 20:35:29.999 INFO 28024 --- [nio-8080-exec-1] ShardingSphere-SQL : Actual SQL: user-01 ::: INSERT INTO address_1 ( id,
code,
name ) VALUES (?, ?, ?) ::: [1, 0826, 四川-武胜-胜利镇]
2024-04-13 20:35:35.205 INFO 28024 --- [nio-8080-exec-2] ShardingSphere-SQL : Logic SQL: INSERT INTO address ( id,
code,
name ) VALUES ( ?,
?,
? )
2024-04-13 20:35:35.205 INFO 28024 --- [nio-8080-exec-2] ShardingSphere-SQL : SQLStatement: MySQLInsertStatement(super=InsertStatement(super=AbstractSQLStatement(parameterCount=3, parameterMarkerSegments=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)], commentSegments=[]), table=SimpleTableSegment(tableName=TableNameSegment(startIndex=12, stopIndex=18, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), insertColumns=Optional[InsertColumnsSegment(startIndex=21, stopIndex=38, columns=[ColumnSegment(startIndex=23, stopIndex=24, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=27, stopIndex=30, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=33, stopIndex=36, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty)])], insertSelect=Optional.empty, values=[InsertValuesSegment(startIndex=49, stopIndex=59, values=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)])]), setAssignment=Optional.empty, onDuplicateKeyColumns=Optional.empty)
2024-04-13 20:35:35.205 INFO 28024 --- [nio-8080-exec-2] ShardingSphere-SQL : Actual SQL: user-02 ::: INSERT INTO address_2 ( id,
code,
name ) VALUES (?, ?, ?) ::: [2, 0826, 四川-武胜-胜利镇]
2024-04-13 20:35:45.941 INFO 28024 --- [nio-8080-exec-3] ShardingSphere-SQL : Logic SQL: INSERT INTO address ( id,
code,
name ) VALUES ( ?,
?,
? )
2024-04-13 20:35:45.941 INFO 28024 --- [nio-8080-exec-3] ShardingSphere-SQL : SQLStatement: MySQLInsertStatement(super=InsertStatement(super=AbstractSQLStatement(parameterCount=3, parameterMarkerSegments=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)], commentSegments=[]), table=SimpleTableSegment(tableName=TableNameSegment(startIndex=12, stopIndex=18, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), insertColumns=Optional[InsertColumnsSegment(startIndex=21, stopIndex=38, columns=[ColumnSegment(startIndex=23, stopIndex=24, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=27, stopIndex=30, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=33, stopIndex=36, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty)])], insertSelect=Optional.empty, values=[InsertValuesSegment(startIndex=49, stopIndex=59, values=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)])]), setAssignment=Optional.empty, onDuplicateKeyColumns=Optional.empty)
2024-04-13 20:35:45.941 INFO 28024 --- [nio-8080-exec-3] ShardingSphere-SQL : Actual SQL: user-02 ::: INSERT INTO address_2 ( id,
code,
name ) VALUES (?, ?, ?) ::: [2, 0826, 四川-武胜-胜利镇]
2024-04-13 20:35:57.756 INFO 28024 --- [nio-8080-exec-4] ShardingSphere-SQL : Logic SQL: INSERT INTO address ( id,
code,
name ) VALUES ( ?,
?,
? )
2024-04-13 20:35:57.756 INFO 28024 --- [nio-8080-exec-4] ShardingSphere-SQL : SQLStatement: MySQLInsertStatement(super=InsertStatement(super=AbstractSQLStatement(parameterCount=3, parameterMarkerSegments=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)], commentSegments=[]), table=SimpleTableSegment(tableName=TableNameSegment(startIndex=12, stopIndex=18, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), insertColumns=Optional[InsertColumnsSegment(startIndex=21, stopIndex=38, columns=[ColumnSegment(startIndex=23, stopIndex=24, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=27, stopIndex=30, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=33, stopIndex=36, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty)])], insertSelect=Optional.empty, values=[InsertValuesSegment(startIndex=49, stopIndex=59, values=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)])]), setAssignment=Optional.empty, onDuplicateKeyColumns=Optional.empty)
2024-04-13 20:35:57.756 INFO 28024 --- [nio-8080-exec-4] ShardingSphere-SQL : Actual SQL: user-02 ::: INSERT INTO address_3 ( id,
code,
name ) VALUES (?, ?, ?) ::: [3, 0826, 四川-武胜-胜利镇]
2024-04-13 20:36:04.925 INFO 28024 --- [nio-8080-exec-5] ShardingSphere-SQL : Logic SQL: INSERT INTO address ( id,
code,
name ) VALUES ( ?,
?,
? )
2024-04-13 20:36:04.925 INFO 28024 --- [nio-8080-exec-5] ShardingSphere-SQL : SQLStatement: MySQLInsertStatement(super=InsertStatement(super=AbstractSQLStatement(parameterCount=3, parameterMarkerSegments=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)], commentSegments=[]), table=SimpleTableSegment(tableName=TableNameSegment(startIndex=12, stopIndex=18, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), insertColumns=Optional[InsertColumnsSegment(startIndex=21, stopIndex=38, columns=[ColumnSegment(startIndex=23, stopIndex=24, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=27, stopIndex=30, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), ColumnSegment(startIndex=33, stopIndex=36, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty)])], insertSelect=Optional.empty, values=[InsertValuesSegment(startIndex=49, stopIndex=59, values=[ParameterMarkerExpressionSegment(startIndex=51, stopIndex=51, parameterMarkerIndex=0, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=54, stopIndex=54, parameterMarkerIndex=1, parameterMarkerType=QUESTION, alias=Optional.empty), ParameterMarkerExpressionSegment(startIndex=57, stopIndex=57, parameterMarkerIndex=2, parameterMarkerType=QUESTION, alias=Optional.empty)])]), setAssignment=Optional.empty, onDuplicateKeyColumns=Optional.empty)
2024-04-13 20:36:04.925 INFO 28024 --- [nio-8080-exec-5] ShardingSphere-SQL : Actual SQL: user-01 ::: INSERT INTO address_0 ( id,
code,
name ) VALUES (?, ?, ?) ::: [4, 0826, 四川-武胜-胜利镇]
- 分页查询
2024-04-13 21:08:38.354 INFO 29060 --- [nio-8080-exec-5] ShardingSphere-SQL : Logic SQL: SELECT COUNT(*) FROM address
2024-04-13 21:08:38.354 INFO 29060 --- [nio-8080-exec-5] ShardingSphere-SQL : SQLStatement: MySQLSelectStatement(super=SelectStatement(super=AbstractSQLStatement(parameterCount=0, parameterMarkerSegments=[], commentSegments=[]), projections=ProjectionsSegment(startIndex=7, stopIndex=14, projections=[AggregationProjectionSegment(startIndex=7, stopIndex=14, type=COUNT, innerExpression=(*), parameters=[], alias=Optional.empty)], distinctRow=false), from=SimpleTableSegment(tableName=TableNameSegment(startIndex=21, stopIndex=27, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), where=Optional.empty, groupBy=Optional.empty, having=Optional.empty, orderBy=Optional.empty, combines=[]), table=Optional.empty, limit=Optional.empty, lock=Optional.empty, window=Optional.empty)
2024-04-13 21:08:38.354 INFO 29060 --- [nio-8080-exec-5] ShardingSphere-SQL : Actual SQL: user-01 ::: SELECT COUNT(*) FROM address_0 UNION ALL SELECT COUNT(*) FROM address_1
2024-04-13 21:08:38.354 INFO 29060 --- [nio-8080-exec-5] ShardingSphere-SQL : Actual SQL: user-02 ::: SELECT COUNT(*) FROM address_2 UNION ALL SELECT COUNT(*) FROM address_3
2024-04-13 21:08:38.699 INFO 29060 --- [nio-8080-exec-5] ShardingSphere-SQL : Logic SQL: SELECT id,code,name FROM address
ORDER BY id ASC LIMIT ?
2024-04-13 21:08:38.699 INFO 29060 --- [nio-8080-exec-5] ShardingSphere-SQL : SQLStatement: MySQLSelectStatement(super=SelectStatement(super=AbstractSQLStatement(parameterCount=1, parameterMarkerSegments=[ParameterMarkerLimitValueSegment(super=LimitValueSegment(startIndex=67, stopIndex=67), parameterIndex=0)], commentSegments=[]), projections=ProjectionsSegment(startIndex=8, stopIndex=19, projections=[ColumnProjectionSegment(column=ColumnSegment(startIndex=8, stopIndex=9, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty), alias=Optional.empty), ColumnProjectionSegment(column=ColumnSegment(startIndex=11, stopIndex=14, identifier=IdentifierValue(value=code, quoteCharacter=NONE), owner=Optional.empty), alias=Optional.empty), ColumnProjectionSegment(column=ColumnSegment(startIndex=16, stopIndex=19, identifier=IdentifierValue(value=name, quoteCharacter=NONE), owner=Optional.empty), alias=Optional.empty)], distinctRow=false), from=SimpleTableSegment(tableName=TableNameSegment(startIndex=27, stopIndex=33, identifier=IdentifierValue(value=address, quoteCharacter=NONE)), owner=Optional.empty, alias=Optional.empty), where=Optional.empty, groupBy=Optional.empty, having=Optional.empty, orderBy=Optional[OrderBySegment(startIndex=45, stopIndex=59, orderByItems=[ColumnOrderByItemSegment(super=TextOrderByItemSegment(), column=ColumnSegment(startIndex=54, stopIndex=55, identifier=IdentifierValue(value=id, quoteCharacter=NONE), owner=Optional.empty))])], combines=[]), table=Optional.empty, limit=Optional[LimitSegment(startIndex=61, stopIndex=67, offset=Optional.empty, rowCount=Optional[ParameterMarkerLimitValueSegment(super=LimitValueSegment(startIndex=67, stopIndex=67), parameterIndex=0)])], lock=Optional.empty, window=Optional.empty)
2024-04-13 21:08:38.699 INFO 29060 --- [nio-8080-exec-5] ShardingSphere-SQL : Actual SQL: user-01 ::: SELECT id,code,name FROM address_0
ORDER BY id ASC LIMIT ? ::: [4]
2024-04-13 21:08:38.699 INFO 29060 --- [nio-8080-exec-5] ShardingSphere-SQL : Actual SQL: user-01 ::: SELECT id,code,name FROM address_1
ORDER BY id ASC LIMIT ? ::: [4]
2024-04-13 21:08:38.699 INFO 29060 --- [nio-8080-exec-5] ShardingSphere-SQL : Actual SQL: user-02 ::: SELECT id,code,name FROM address_2
ORDER BY id ASC LIMIT ? ::: [4]
2024-04-13 21:08:38.699 INFO 29060 --- [nio-8080-exec-5] ShardingSphere-SQL : Actual SQL: user-02 ::: SELECT id,code,name FROM address_3
ORDER BY id ASC LIMIT ? ::: [4]
备注说明
- 为什么没有单独的分库案例,我觉得没有这种场景,单独的分库一般是多租户的场景,不同的租户不同的库,对于已经有的业务扩展,一般分库分表一起用的,所以没有单独的分库案例。
多数据源下liquibase+ShardingSphere使用指南
为什么要单独抽出来说这个问题,因为我工作中遇到了,可能也会让采用这种方案的同学遇到相同的问题,在此给出解决方案
必须在数据源初始化之前执行了所有数据库的liquibase
下面给出实际代码,需要个人结合自己的实际功能调整
@Configuration
@Import({LiquibaseDependsOnPostConfiguration.LiquibaseDependsOnPostProcessor.class})
public class LiquibaseDependsOnPostConfiguration {
@Bean
public UserLiquibase userLiquibase(){
return new UserLiquibase();
}
static class LiquibaseDependsOnPostProcessor extends AbstractDependsOnBeanFactoryPostProcessor {
// 让DataSource依赖于UserLiquibase,暨让DataSource在UserLiquibase之后加载
LiquibaseDependsOnPostProcessor(){
super(DataSource.class, UserLiquibase.class);
}
}
public static class UserLiquibase {
// 自己执行liquibase的数据库逻辑
}
}