【资源】
【本文内容】
- 结合HikariCP官方文档和实例,介绍了常用的参数配置的作用。
- 本文源码版本:HikariCP-5.1.0
1. 必须要配置的信息
这里列的参数名是基于Hikari官方文档上的。必填的配置有4个,都是比较熟悉的配置:
dataSourceClassNamejdbcUrlusernamepassword
2. 高频使用的配置
autoCommit:连接池中的connection是否为自动提交,默认值为trueconnectionTimeout:客户端(即我们的应用)等待连接池中返回connection的最长时间(毫秒),如果超过该时间(即没有在声明的时间内拿到有用的connection),那么会抛SQLException异常。默认值为30000,即30秒。最短允许被设置为250ms。idleTimeout:一个idle connection最多被允许在连接池中存活的时间。这个配置生效需要满足以下两个条件:1)idleTimeout值需要大于0(即等于0表示idle connection不会从连接池中被移除)。2)minimumIdle<maximumPoolSize(默认这两个参数都为10)。- 在连接池中的connection数量达到
minimumIdle前,Idle connection不会被移除(即需要被移除的数量=没有在使用的connection数量-minimumIdle)。 - 默认值为600000,即10分钟。最短允许被设置为10000ms。
- 在连接池中的connection数量达到
HikariPool.java中关于idleTimeout的代码:
if (idleTimeout > 0L && config.getMinimumIdle() < config.getMaximumPoolSize()) {
logPoolState("Before cleanup ");
final var notInUse = connectionBag.values(STATE_NOT_IN_USE);
var maxToRemove = notInUse.size() - config.getMinimumIdle();
for (PoolEntry entry : notInUse) {
if (maxToRemove > 0 && elapsedMillis(entry.lastAccessed, now) > idleTimeout && connectionBag.reserve(entry)) {
closeConnection(entry, "(connection has passed idleTimeout)");
maxToRemove--;
}
}
logPoolState("After cleanup ");
}
keepaliveTime:connection存活时间,HikariCP会尝试在一定时间内保持一个connection活着的状态,即阻止因为db server端超时或网络超时等问题导致的connection失效。(如何保持?通过定时器的方式使这个connection活着)。- 这个配置只有小于
maxLifetime值才生效(因为如果超过了connection的最大生命周期也就没有意义了。) - 这个配置只针对idle connection(只有闲置连接才需要类似的心跳机制)。
- 当connection达到了需要检查的时间,则会从连接池中先移除,再做检查,检查完后再放回连接池中。
- 如何保持alive:通过调用JDBC4
isValid()方法或是执行connectionTestQuery。 - 默认值为0(即关闭该功能),最短允许被设置为30000ms(即30秒),但最理想的值是在分钟范围内。
- 这个配置只有小于
以下摘自HikariPool.java:心跳机制相关,从以下代码中可以看到,variance为:0到keepaliveTime的10分之1的随机数。心跳间隔时间为keepaliveTime - variance。
举列来说的话就是如果设置keepaliveTime=60000,即60秒,那么大概每隔50-60秒之间有一次心跳检查。
final long keepaliveTime = config.getKeepaliveTime();
if (keepaliveTime > 0) {
// variance up to 10% of the heartbeat time
final var variance = ThreadLocalRandom.current().nextLong(keepaliveTime / 10);
final var heartbeatTime = keepaliveTime - variance;
poolEntry.setKeepalive(houseKeepingExecutorService.scheduleWithFixedDelay(new KeepaliveTask(poolEntry), heartbeatTime, heartbeatTime, MILLISECONDS));
}
-
maxLifetime:connection在连接池中最大的存活时间。- 如果这个connection正在被使用中,那么就不会被淘汰,即connection被close后才会被移除。
- Hikari官方建议需要设置maxLifetime,通常情况下需要比相应的database端(如oracle, mysql)默认的connection最大限制时间短几秒。
- 如果设为0,表示永久存活。
- 默认值为1800000(即30分钟)。
-
connectionTestQuery:如果我们应用的db驱动支持JDBC4,那么推荐不要配置该值。即如果JDBC4的Connection.isValid()可以使用,推荐使用JDBC4的API,而不是connectionTestQuery。- 当connection从连接池中返回时,需要检查是否活着,则需要用到该方法来校验。
- 何时会被调用?连接池中新创建connection对象时 / keepalive发生心跳时 / 从线程池中返回已有connection时。
- 默认值为空。
PoolBase.java中关于isValid和connectionTestQuery的选择,可以看到如果支持jdbc4Validation,那么则会优先选择jdbc4的isValid()方式:
private void checkValidationSupport(Connection connection) throws SQLException {
try {
if (this.isUseJdbc4Validation) {
connection.isValid(1);
} else {
this.executeSql(connection, this.config.getConnectionTestQuery(), false);
}
} catch (AbstractMethodError | Exception var3) {
this.logger.error("{} - Failed to execute{} connection test query ({}).", new Object[]{this.poolName, this.isUseJdbc4Validation ? " isValid() for connection, configure" : "", var3.getMessage()});
throw var3;
}
}
minimumIdle:这个配置表示连接池中最小保持的idle connection数量。当连接池中的idle connection数量小于该值,同时还没有到达maximumPoolSize数量时,HikariCP会尝试创建出新的connections,直到到达该数值。- 官方建议,为了获得最大的性能和对峰值需求的响应能力,建议不要配置该值,即使得HikariCP创建出固定大小的连接池。
- 默认值和
maximumPoolSize相等,即为10个。
HikariPool.java中关于是否需要增加连接池中的connection的判断:
private synchronized void fillPool(final boolean isAfterAdd)
{
final var idle = getIdleConnections();
final var shouldAdd = getTotalConnections() < config.getMaximumPoolSize() && idle < config.getMinimumIdle();
if (shouldAdd) {
final var countToAdd = config.getMinimumIdle() - idle;
for (int i = 0; i < countToAdd; i++)
addConnectionExecutor.submit(isAfterAdd ? postFillPoolEntryCreator : poolEntryCreator);
}
else if (isAfterAdd) {
logger.debug("{} - Fill pool skipped, pool has sufficient level or currently being filled.", poolName);
}
}
-
maximumPoolSize:线程池中的最大connection连接数,包括了idle状态和正在使用状态的connection。通常情况下这个值关系到数据库后端的最大实际连接数。当线程池中的connection数量到达该值后,并且池中没有idle connection,那么调用getConnection()方法会被阻塞,直到等待时间到达connectionTimeout(超时后会抛出SQLException异常)。- 默认值为10个。
- 关于maximumpoolSize的设置,Hikari官网有专门的文章,详细参考:github.com/brettwooldr…,关于最大连接数的设置,可能是反直觉的(即并不是设置越多越好),这篇文章中还有计算公式,国内也有比较好的翻译文章,参考:blog.csdn.net/m0_38048955…
-
metricRegistry:metrics相关配置。 -
healthCheckRegistry:health check相关配置。 -
poolName:连接池的名字,在日志或是JMX中被会显示,默认自动生成。
3. 通过测试代码来测试上述参数表现
3.1 case-1 配置minimumIdle和maximumPool不同时,HikariCP会保持connection数量=minimumIdle
首先是spring-boot项目中的设置:
- 打开Hikari Debug日志。
- 最大线程数为4个,最小idle connection数为2个,idle connection time out时间为30秒。
logging.level.com.zaxxer.hikari=DEBUG
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=4
spring.datasource.hikari.pool-name=test001
启动项目后,通过日志可以看到,在连接池启动时就会先创建出一个connection(JDBC4Connection@52b06bef),在连接池创建好后,会立刻启动housekeeper线程,用来管理空闲连接,此时housekeeper发现配置的minimumIdle=2,所以它会试图再创建一个connection,即JDBC4Connection@7c94dad2。
会立刻启动housekeeper线程每隔30秒运行一次,发现已经达到最小的idle connection数量了(即2个),所以不会再创建新的connection了。log会显示Fill pool skipped, pool has sufficient level or currently being filled。
此后,会一直保持:(total=2, active=0, idle=2, waiting=0)
直到waiting大于2时,才会创建新的connection。
16:39:54.499 INFO 28326 --- [ main] c.z.h.HikariDataSource : test001 - Starting...
16:39:54.655 INFO 28326 --- [ main] c.z.h.pool.HikariPool : test001 - Added connection com.mysql.jdbc.JDBC4Connection@52b06bef
16:39:54.657 INFO 28326 --- [ main] c.z.h.HikariDataSource : test001 - Start completed.
16:39:54.757 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Before cleanup stats (total=1, active=0, idle=1, waiting=0)
16:39:54.758 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - After cleanup stats (total=1, active=0, idle=1, waiting=0)
16:39:54.770 DEBUG 28326 --- [onnection adder] c.z.h.p.HikariPool : test001 - Added connection com.mysql.jdbc.JDBC4Connection@7c94dad2
16:39:54.804 DEBUG 28326 --- [onnection adder] c.z.h.p.HikariPool : test001 - After adding stats (total=2, active=0, idle=2, waiting=0)
16:39:55.569 INFO 28326 --- [ main] com.HikariApplication : Started HikariApplication in 3.183 seconds (JVM running for 3.758)
16:40:24.761 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
16:40:24.761 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - After cleanup stats (total=2, active=0, idle=2, waiting=0)
16:40:24.761 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
16:40:54.763 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
16:40:54.764 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - After cleanup stats (total=2, active=0, idle=2, waiting=0)
16:40:54.764 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
16:41:24.768 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
16:41:24.769 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - After cleanup stats (total=2, active=0, idle=2, waiting=0)
16:41:24.769 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
3.2 case-2 更为复杂的测试
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=4
spring.datasource.hikari.idle-timeout=30000
写了一个test方法,用来摸拟同时请求6个connection,等待2秒后再close掉这个connection:
@Autowired
private HikariDataSource dataSource;
public void test() {
ExecutorService executorService = Executors.newFixedThreadPool(6);
for (int i = 0; i < 6; i ++) {
executorService.submit(() -> {
Connection connection = null;
try {
connection = dataSource.getConnection();
// use the connection
log.info("get connection: [{}]", connection);
Thread.sleep(2000L);
} catch (SQLException | InterruptedException e) {
throw new RuntimeException(e);
} finally {
try {
connection.close();
log.info("connection [{}] closed.", connection);
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
});
}
}
日志,一开始没有请求上述test()方法前,和#3.1一样,会保持2个idle connection:
JDBC4Connection@6af91cc8JDBC4Connection@37584497
17:33:40.831 INFO 29477 --- [ main] c.z.h.p.HikariPool : test001 - Added connection com.mysql.jdbc.JDBC4Connection@6af91cc8
17:33:40.948 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool : test001 - Added connection com.mysql.jdbc.JDBC4Connection@37584497
17:34:10.937 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
17:34:10.938 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - After cleanup stats (total=2, active=0, idle=2, waiting=0)
17:34:10.938 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
开始请求test()方法后,需要拿到6个connection,但我们配置的最大线程数为4个,所以会有两个线程处于等待中。
第1行第2行返回连接池中的connection,正是上述创建的两个实例。即:
JDBC4Connection@6af91cc8--> test()方法中connection对象为HikariProxyConnection@247742127JDBC4Connection@37584497--> test()方法中connection对象为HikariProxyConnection@1779361118
然后再继续创建了2个(此时到达了配置的maximumPool4个):
- 创建第3个connection:
JDBC4Connection@170bcf77--> test()方法中connection对象为:HikariProxyConnection@1880798211- 此时housekeeper打印:
(total=3, active=3, idle=0, waiting=3),表明还需要再创建3个connection。
- 此时housekeeper打印:
- 创建第4个:
JDBC4Connection@42862c42--> test()方法中connection对象为:HikariProxyConnection@2121888672- 此时housekeeper打印:
(total=4, active=4, idle=0, waiting=2)
- 此时housekeeper打印:
17:34:18.601 INFO 29477 --- [pool-1-thread-2] com.TestController : get connection: [HikariProxyConnection@1779361118 wrapping com.mysql.jdbc.JDBC4Connection@37584497]
17:34:18.601 INFO 29477 --- [pool-1-thread-1] com.TestController : get connection: [HikariProxyConnection@247742127 wrapping com.mysql.jdbc.JDBC4Connection@6af91cc8]
17:34:18.611 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool : test001 - Added connection com.mysql.jdbc.JDBC4Connection@170bcf77
17:34:18.611 INFO 29477 --- [pool-1-thread-4] com.TestController : get connection: [HikariProxyConnection@1880798211 wrapping com.mysql.jdbc.JDBC4Connection@170bcf77]
17:34:18.643 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool : test001 - Connection not added, stats (total=3, active=3, idle=0, waiting=3)
17:34:18.649 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool : test001 - Added connection com.mysql.jdbc.JDBC4Connection@42862c42
17:34:18.649 INFO 29477 --- [pool-1-thread-5] com.TestController : get connection: [HikariProxyConnection@2121888672 wrapping com.mysql.jdbc.JDBC4Connection@42862c42]
17:34:18.682 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool : test001 - Connection not added, stats (total=4, active=4, idle=0, waiting=2)
17:34:18.683 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool : test001 - Connection not added, stats (total=4, active=4, idle=0, waiting=2)
17:34:18.683 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool : test001 - Connection not added, stats (total=4, active=4, idle=0, waiting=2)
test()方法中的connection在2秒后开始close,即:
- close了
HikariProxyConnection@1779361118对象(即JDBC4Connection@37584497) - close了
HikariProxyConnection@247742127对象(即JDBC4Connection@6af91cc8)
17:34:20.606 INFO 29477 --- [pool-1-thread-2] com.TestController : connection [HikariProxyConnection@1779361118 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
17:34:20.606 INFO 29477 --- [pool-1-thread-1] com.TestController : connection [HikariProxyConnection@247742127 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
因为2个connection被close了,所以另外两个waiting的线程在此刻拿到了这两个线程,同样是JDBC4Connection@6af91cc8和JDBC4Connection@37584497。
17:34:20.606 INFO 29477 --- [pool-1-thread-3] com.TestController : get connection: [HikariProxyConnection@1266310434 wrapping com.mysql.jdbc.JDBC4Connection@6af91cc8]
17:34:20.606 INFO 29477 --- [pool-1-thread-6] com.TestController : get connection: [HikariProxyConnection@1274808071 wrapping com.mysql.jdbc.JDBC4Connection@37584497]
test()方法开始陆续关闭connection,housekeeper打印total=4, active=0, idle=4, waiting=0,表明4个connection都处于idle状态。
分析idleTimeout=30秒的逻辑:
- 在17:34:22 test()方法开始陆续关闭connection
- 在17:34:40 housekeeper打印
total=4, active=0, idle=4, waiting=0,此时没有触发idle connection的回收。 - 在17:35:10 开始回收多余的idle connection,回收后状态
total=2, active=0, idle=2, waiting=0。
17:34:20.612 INFO 29477 --- [pool-1-thread-4] com.TestController : connection [HikariProxyConnection@1880798211 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
17:34:20.654 INFO 29477 --- [pool-1-thread-5] com.TestController : connection [HikariProxyConnection@2121888672 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
17:34:22.607 INFO 29477 --- [pool-1-thread-6] com.TestController : connection [HikariProxyConnection@1274808071 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
17:34:22.607 INFO 29477 --- [pool-1-thread-3] com.TestController : connection [HikariProxyConnection@1266310434 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
17:34:40.940 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Before cleanup stats (total=4, active=0, idle=4, waiting=0)
17:34:40.941 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - After cleanup stats (total=4, active=0, idle=4, waiting=0)
17:34:40.941 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
17:35:10.940 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Before cleanup stats (total=4, active=0, idle=4, waiting=0)
17:35:10.942 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - After cleanup stats (total=2, active=0, idle=2, waiting=0)
17:35:10.942 DEBUG 29477 --- [nnection closer] c.z.h.p.PoolBase : test001 - Closing connection com.mysql.jdbc.JDBC4Connection@42862c42: (connection has passed idleTimeout)
17:35:10.942 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
17:35:10.946 DEBUG 29477 --- [nnection closer] c.z.h.p.PoolBase : test001 - Closing connection com.mysql.jdbc.JDBC4Connection@170bcf77: (connection has passed idleTimeout)
17:35:40.946 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
17:35:40.947 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - After cleanup stats (total=2, active=0, idle=2, waiting=0)
17:35:40.947 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
3.3 case-3 如果不配置idleTimeout,那么idle connection不会被回收
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=4
配置和#3.1一样,只是在#3.1的基础上,调用了test()方法后,connection 在2秒后close了,然后housekeeper打印:(total=4, active=0, idle=4, waiting=0),和#3.2不同的是,#3.2配置了idleTimeout=30s,而#3.3没有配置,即默认为10分钟,所以在10分钟内,total都会等于4个,而不是一开始的2个。
3.4 测试keepaliveTime配置
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=4
spring.datasource.hikari.keep-alive-time=60000
启动项目后,调试后看到取得的随机数为4787,所以心跳间隔时间则为60000-4787=55213ms,即大约每隔55秒进行一次心跳检查。
注:这里的心跳检查指的是调用JDBC4的isValid()方法:
日志,创建了两个connection:
JDBC4Connection@3d1f558a和JDBC4Connection@51569e5b。
21:06:07.996 INFO 35275 --- [ main] c.z.h.p.HikariPool : test001 - Added connection com.mysql.jdbc.JDBC4Connection@3d1f558a
21:06:08.707 DEBUG 35275 --- [onnection adder] c.z.h.p.HikariPool : test001 - Added connection com.mysql.jdbc.JDBC4Connection@51569e5b
项目启动后进行keepalive的心跳,对于JDBC4Connection@3d1f558a,可以看到第一次心跳是在21:07:03,第二次心跳是在21:07:58,差不多刚好53秒。
注:为什么两个connection的心跳检查时间差了4秒?因为我在创建的时候有打了断点,耽误了第2个connection创建的时间。
21:07:03.207 DEBUG 35275 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - keepalive: connection com.mysql.jdbc.JDBC4Connection@3d1f558a is alive
21:07:07.944 DEBUG 35275 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - keepalive: connection com.mysql.jdbc.JDBC4Connection@51569e5b is alive
21:07:58.421 DEBUG 35275 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - keepalive: connection com.mysql.jdbc.JDBC4Connection@3d1f558a is alive
21:08:07.184 DEBUG 35275 --- [001 housekeeper] c.z.h.p.HikariPool : test001 - keepalive: connection com.mysql.jdbc.JDBC4Connection@51569e5b is alive
4. 其它一些不常用的配置
initializationFailTimeout:isolateInternalQueries:allowPoolSuspension:readOnly:registerMbeans:catalog:connectionInitSql:driverClassName:transactionIsolation:validationTimeout:leakDetectionThreshold:dataSource:schema:threadFactory:scheduledExecutor: