【Java】HikariCP配置学习(有例子,非常详细)

690 阅读14分钟

【资源】

【本文内容】

  • 结合HikariCP官方文档和实例,介绍了常用的参数配置的作用。
  • 本文源码版本:HikariCP-5.1.0

1. 必须要配置的信息

这里列的参数名是基于Hikari官方文档上的。必填的配置有4个,都是比较熟悉的配置:

  • dataSourceClassName
  • jdbcUrl
  • username
  • password

2. 高频使用的配置

  • autoCommit:连接池中的connection是否为自动提交,默认值为true
  • connectionTimeout:客户端(即我们的应用)等待连接池中返回connection的最长时间(毫秒),如果超过该时间(即没有在声明的时间内拿到有用的connection),那么会抛SQLException异常。默认值为30000,即30秒。最短允许被设置为250ms。
  • idleTimeout:一个idle connection最多被允许在连接池中存活的时间。这个配置生效需要满足以下两个条件:1)idleTimeout值需要大于0(即等于0表示idle connection不会从连接池中被移除)。2) minimumIdle < maximumPoolSize(默认这两个参数都为10)。
    • 在连接池中的connection数量达到minimumIdle前,Idle connection不会被移除(即需要被移除的数量=没有在使用的connection数量-minimumIdle)。
    • 默认值为600000,即10分钟。最短允许被设置为10000ms。

HikariPool.java中关于idleTimeout的代码:

if (idleTimeout > 0L && config.getMinimumIdle() < config.getMaximumPoolSize()) {
   logPoolState("Before cleanup ");
   final var notInUse = connectionBag.values(STATE_NOT_IN_USE);
   var maxToRemove = notInUse.size() - config.getMinimumIdle();
   for (PoolEntry entry : notInUse) {
      if (maxToRemove > 0 && elapsedMillis(entry.lastAccessed, now) > idleTimeout && connectionBag.reserve(entry)) {
         closeConnection(entry, "(connection has passed idleTimeout)");
         maxToRemove--;
      }
   }
   logPoolState("After cleanup  ");
}
  • keepaliveTime:connection存活时间,HikariCP会尝试在一定时间内保持一个connection活着的状态,即阻止因为db server端超时或网络超时等问题导致的connection失效。(如何保持?通过定时器的方式使这个connection活着)。
    • 这个配置只有小于maxLifetime值才生效(因为如果超过了connection的最大生命周期也就没有意义了。)
    • 这个配置只针对idle connection(只有闲置连接才需要类似的心跳机制)。
    • 当connection达到了需要检查的时间,则会从连接池中先移除,再做检查,检查完后再放回连接池中。
    • 如何保持alive:通过调用JDBC4 isValid()方法或是执行connectionTestQuery
    • 默认值为0(即关闭该功能),最短允许被设置为30000ms(即30秒),但最理想的值是在分钟范围内。

以下摘自HikariPool.java:心跳机制相关,从以下代码中可以看到,variance为:0到keepaliveTime的10分之1的随机数。心跳间隔时间为keepaliveTime - variance。

举列来说的话就是如果设置keepaliveTime=60000,即60秒,那么大概每隔50-60秒之间有一次心跳检查。

final long keepaliveTime = config.getKeepaliveTime();
if (keepaliveTime > 0) {
   // variance up to 10% of the heartbeat time
   final var variance = ThreadLocalRandom.current().nextLong(keepaliveTime / 10);
   final var heartbeatTime = keepaliveTime - variance;
   poolEntry.setKeepalive(houseKeepingExecutorService.scheduleWithFixedDelay(new KeepaliveTask(poolEntry), heartbeatTime, heartbeatTime, MILLISECONDS));
}
  • maxLifetime:connection在连接池中最大的存活时间。

    • 如果这个connection正在被使用中,那么就不会被淘汰,即connection被close后才会被移除。
    • Hikari官方建议需要设置maxLifetime,通常情况下需要比相应的database端(如oracle, mysql)默认的connection最大限制时间短几秒。
    • 如果设为0,表示永久存活。
    • 默认值为1800000(即30分钟)。
  • connectionTestQuery如果我们应用的db驱动支持JDBC4,那么推荐不要配置该值。即如果JDBC4的Connection.isValid()可以使用,推荐使用JDBC4的API,而不是connectionTestQuery。

    • 当connection从连接池中返回时,需要检查是否活着,则需要用到该方法来校验。
    • 何时会被调用?连接池中新创建connection对象时 / keepalive发生心跳时 / 从线程池中返回已有connection时。
    • 默认值为空。

PoolBase.java中关于isValid和connectionTestQuery的选择,可以看到如果支持jdbc4Validation,那么则会优先选择jdbc4的isValid()方式:

private void checkValidationSupport(Connection connection) throws SQLException {
    try {
        if (this.isUseJdbc4Validation) {
            connection.isValid(1);
        } else {
            this.executeSql(connection, this.config.getConnectionTestQuery(), false);
        }

    } catch (AbstractMethodError | Exception var3) {
        this.logger.error("{} - Failed to execute{} connection test query ({}).", new Object[]{this.poolName, this.isUseJdbc4Validation ? " isValid() for connection, configure" : "", var3.getMessage()});
        throw var3;
    }
}
  • minimumIdle:这个配置表示连接池中最小保持的idle connection数量。当连接池中的idle connection数量小于该值,同时还没有到达maximumPoolSize数量时,HikariCP会尝试创建出新的connections,直到到达该数值。
    • 官方建议,为了获得最大的性能和对峰值需求的响应能力,建议不要配置该值,即使得HikariCP创建出固定大小的连接池。
    • 默认值和maximumPoolSize相等,即为10个。

HikariPool.java中关于是否需要增加连接池中的connection的判断:

private synchronized void fillPool(final boolean isAfterAdd)
{
   final var idle = getIdleConnections();
   final var shouldAdd = getTotalConnections() < config.getMaximumPoolSize() && idle < config.getMinimumIdle();

   if (shouldAdd) {
      final var countToAdd = config.getMinimumIdle() - idle;
      for (int i = 0; i < countToAdd; i++)
         addConnectionExecutor.submit(isAfterAdd ? postFillPoolEntryCreator : poolEntryCreator);
   }
   else if (isAfterAdd) {
      logger.debug("{} - Fill pool skipped, pool has sufficient level or currently being filled.", poolName);
   }
}
  • maximumPoolSize:线程池中的最大connection连接数,包括了idle状态和正在使用状态的connection。通常情况下这个值关系到数据库后端的最大实际连接数。当线程池中的connection数量到达该值后,并且池中没有idle connection,那么调用getConnection()方法会被阻塞,直到等待时间到达connectionTimeout(超时后会抛出SQLException异常)。

    • 默认值为10个。
    • 关于maximumpoolSize的设置,Hikari官网有专门的文章,详细参考github.com/brettwooldr…,关于最大连接数的设置,可能是反直觉的(即并不是设置越多越好),这篇文章中还有计算公式,国内也有比较好的翻译文章,参考:blog.csdn.net/m0_38048955…
  • metricRegistry:metrics相关配置。

  • healthCheckRegistry:health check相关配置。

  • poolName:连接池的名字,在日志或是JMX中被会显示,默认自动生成。

3. 通过测试代码来测试上述参数表现

3.1 case-1 配置minimumIdle和maximumPool不同时,HikariCP会保持connection数量=minimumIdle

首先是spring-boot项目中的设置:

  • 打开Hikari Debug日志。
  • 最大线程数为4个,最小idle connection数为2个,idle connection time out时间为30秒。
logging.level.com.zaxxer.hikari=DEBUG
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=4
spring.datasource.hikari.pool-name=test001

启动项目后,通过日志可以看到,在连接池启动时就会先创建出一个connection(JDBC4Connection@52b06bef),在连接池创建好后,会立刻启动housekeeper线程,用来管理空闲连接,此时housekeeper发现配置的minimumIdle=2,所以它会试图再创建一个connection,即JDBC4Connection@7c94dad2

会立刻启动housekeeper线程每隔30秒运行一次,发现已经达到最小的idle connection数量了(即2个),所以不会再创建新的connection了。log会显示Fill pool skipped, pool has sufficient level or currently being filled

此后,会一直保持:(total=2, active=0, idle=2, waiting=0)

直到waiting大于2时,才会创建新的connection。

16:39:54.499  INFO 28326 --- [           main] c.z.h.HikariDataSource    : test001 - Starting...
16:39:54.655  INFO 28326 --- [           main] c.z.h.pool.HikariPool     : test001 - Added connection com.mysql.jdbc.JDBC4Connection@52b06bef
16:39:54.657  INFO 28326 --- [           main] c.z.h.HikariDataSource    : test001 - Start completed.
16:39:54.757 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Before cleanup stats (total=1, active=0, idle=1, waiting=0)
16:39:54.758 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - After cleanup  stats (total=1, active=0, idle=1, waiting=0)
16:39:54.770 DEBUG 28326 --- [onnection adder] c.z.h.p.HikariPool        : test001 - Added connection com.mysql.jdbc.JDBC4Connection@7c94dad2
16:39:54.804 DEBUG 28326 --- [onnection adder] c.z.h.p.HikariPool        : test001 - After adding stats (total=2, active=0, idle=2, waiting=0)
16:39:55.569  INFO 28326 --- [           main] com.HikariApplication     : Started HikariApplication in 3.183 seconds (JVM running for 3.758)
16:40:24.761 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
16:40:24.761 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - After cleanup  stats (total=2, active=0, idle=2, waiting=0)
16:40:24.761 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
16:40:54.763 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
16:40:54.764 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - After cleanup  stats (total=2, active=0, idle=2, waiting=0)
16:40:54.764 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
16:41:24.768 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
16:41:24.769 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - After cleanup  stats (total=2, active=0, idle=2, waiting=0)
16:41:24.769 DEBUG 28326 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
3.2 case-2 更为复杂的测试
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=4
spring.datasource.hikari.idle-timeout=30000

写了一个test方法,用来摸拟同时请求6个connection,等待2秒后再close掉这个connection:

@Autowired
private HikariDataSource dataSource;

public void test() {
    ExecutorService executorService = Executors.newFixedThreadPool(6);
    for (int i = 0; i < 6; i ++) {
        executorService.submit(() -> {
            Connection connection = null;
            try {
                connection = dataSource.getConnection();
                // use the connection
                log.info("get connection: [{}]", connection);
                Thread.sleep(2000L);
            } catch (SQLException | InterruptedException e) {
                throw new RuntimeException(e);
            } finally {
                try {
                    connection.close();
                    log.info("connection [{}] closed.", connection);
                } catch (SQLException e) {
                    throw new RuntimeException(e);
                }
            }
        });
    }
}

日志,一开始没有请求上述test()方法前,和#3.1一样,会保持2个idle connection:

  • JDBC4Connection@6af91cc8
  • JDBC4Connection@37584497
17:33:40.831  INFO 29477 --- [           main] c.z.h.p.HikariPool        : test001 - Added connection com.mysql.jdbc.JDBC4Connection@6af91cc8
17:33:40.948 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool        : test001 - Added connection com.mysql.jdbc.JDBC4Connection@37584497
17:34:10.937 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
17:34:10.938 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - After cleanup  stats (total=2, active=0, idle=2, waiting=0)
17:34:10.938 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Fill pool skipped, pool has sufficient level or currently being filled.

开始请求test()方法后,需要拿到6个connection,但我们配置的最大线程数为4个,所以会有两个线程处于等待中。

第1行第2行返回连接池中的connection,正是上述创建的两个实例。即:

  • JDBC4Connection@6af91cc8 --> test()方法中connection对象为HikariProxyConnection@247742127
  • JDBC4Connection@37584497 --> test()方法中connection对象为HikariProxyConnection@1779361118

然后再继续创建了2个(此时到达了配置的maximumPool4个):

  • 创建第3个connection:JDBC4Connection@170bcf77 --> test()方法中connection对象为:HikariProxyConnection@1880798211
    • 此时housekeeper打印:(total=3, active=3, idle=0, waiting=3),表明还需要再创建3个connection。
  • 创建第4个:JDBC4Connection@42862c42 --> test()方法中connection对象为:HikariProxyConnection@2121888672
    • 此时housekeeper打印:(total=4, active=4, idle=0, waiting=2)
17:34:18.601  INFO 29477 --- [pool-1-thread-2] com.TestController        : get connection: [HikariProxyConnection@1779361118 wrapping com.mysql.jdbc.JDBC4Connection@37584497]
17:34:18.601  INFO 29477 --- [pool-1-thread-1] com.TestController        : get connection: [HikariProxyConnection@247742127 wrapping com.mysql.jdbc.JDBC4Connection@6af91cc8]
17:34:18.611 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool        : test001 - Added connection com.mysql.jdbc.JDBC4Connection@170bcf77
17:34:18.611  INFO 29477 --- [pool-1-thread-4] com.TestController        : get connection: [HikariProxyConnection@1880798211 wrapping com.mysql.jdbc.JDBC4Connection@170bcf77]
17:34:18.643 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool        : test001 - Connection not added, stats (total=3, active=3, idle=0, waiting=3)
17:34:18.649 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool        : test001 - Added connection com.mysql.jdbc.JDBC4Connection@42862c42
17:34:18.649  INFO 29477 --- [pool-1-thread-5] com.TestController        : get connection: [HikariProxyConnection@2121888672 wrapping com.mysql.jdbc.JDBC4Connection@42862c42]
17:34:18.682 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool        : test001 - Connection not added, stats (total=4, active=4, idle=0, waiting=2)
17:34:18.683 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool        : test001 - Connection not added, stats (total=4, active=4, idle=0, waiting=2)
17:34:18.683 DEBUG 29477 --- [onnection adder] c.z.h.p.HikariPool        : test001 - Connection not added, stats (total=4, active=4, idle=0, waiting=2)

test()方法中的connection在2秒后开始close,即:

  • close了HikariProxyConnection@1779361118对象(即JDBC4Connection@37584497
  • close了HikariProxyConnection@247742127对象(即JDBC4Connection@6af91cc8
17:34:20.606  INFO 29477 --- [pool-1-thread-2] com.TestController        : connection [HikariProxyConnection@1779361118 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
17:34:20.606  INFO 29477 --- [pool-1-thread-1] com.TestController        : connection [HikariProxyConnection@247742127 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.

因为2个connection被close了,所以另外两个waiting的线程在此刻拿到了这两个线程,同样是JDBC4Connection@6af91cc8JDBC4Connection@37584497

17:34:20.606  INFO 29477 --- [pool-1-thread-3] com.TestController        : get connection: [HikariProxyConnection@1266310434 wrapping com.mysql.jdbc.JDBC4Connection@6af91cc8]
17:34:20.606  INFO 29477 --- [pool-1-thread-6] com.TestController        : get connection: [HikariProxyConnection@1274808071 wrapping com.mysql.jdbc.JDBC4Connection@37584497]

test()方法开始陆续关闭connection,housekeeper打印total=4, active=0, idle=4, waiting=0,表明4个connection都处于idle状态。

分析idleTimeout=30秒的逻辑:

  • 在17:34:22 test()方法开始陆续关闭connection
  • 在17:34:40 housekeeper打印total=4, active=0, idle=4, waiting=0,此时没有触发idle connection的回收。
  • 在17:35:10 开始回收多余的idle connection,回收后状态total=2, active=0, idle=2, waiting=0
17:34:20.612  INFO 29477 --- [pool-1-thread-4] com.TestController        : connection [HikariProxyConnection@1880798211 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
17:34:20.654  INFO 29477 --- [pool-1-thread-5] com.TestController        : connection [HikariProxyConnection@2121888672 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
17:34:22.607  INFO 29477 --- [pool-1-thread-6] com.TestController        : connection [HikariProxyConnection@1274808071 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
17:34:22.607  INFO 29477 --- [pool-1-thread-3] com.TestController        : connection [HikariProxyConnection@1266310434 wrapping com.zaxxer.hikari.pool.ProxyConnection.ClosedConnection] closed.
17:34:40.940 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Before cleanup stats (total=4, active=0, idle=4, waiting=0)
17:34:40.941 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - After cleanup  stats (total=4, active=0, idle=4, waiting=0)
17:34:40.941 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
17:35:10.940 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Before cleanup stats (total=4, active=0, idle=4, waiting=0)
17:35:10.942 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - After cleanup  stats (total=2, active=0, idle=2, waiting=0)
17:35:10.942 DEBUG 29477 --- [nnection closer] c.z.h.p.PoolBase          : test001 - Closing connection com.mysql.jdbc.JDBC4Connection@42862c42: (connection has passed idleTimeout)
17:35:10.942 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
17:35:10.946 DEBUG 29477 --- [nnection closer] c.z.h.p.PoolBase          : test001 - Closing connection com.mysql.jdbc.JDBC4Connection@170bcf77: (connection has passed idleTimeout)
17:35:40.946 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
17:35:40.947 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - After cleanup  stats (total=2, active=0, idle=2, waiting=0)
17:35:40.947 DEBUG 29477 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - Fill pool skipped, pool has sufficient level or currently being filled.
3.3 case-3 如果不配置idleTimeout,那么idle connection不会被回收
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=4

配置和#3.1一样,只是在#3.1的基础上,调用了test()方法后,connection 在2秒后close了,然后housekeeper打印:(total=4, active=0, idle=4, waiting=0),和#3.2不同的是,#3.2配置了idleTimeout=30s,而#3.3没有配置,即默认为10分钟,所以在10分钟内,total都会等于4个,而不是一开始的2个。

3.4 测试keepaliveTime配置
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=4
spring.datasource.hikari.keep-alive-time=60000

启动项目后,调试后看到取得的随机数为4787,所以心跳间隔时间则为60000-4787=55213ms,即大约每隔55秒进行一次心跳检查。 注:这里的心跳检查指的是调用JDBC4的isValid()方法:image.png 日志,创建了两个connection:JDBC4Connection@3d1f558aJDBC4Connection@51569e5b

21:06:07.996  INFO 35275 --- [           main] c.z.h.p.HikariPool        : test001 - Added connection com.mysql.jdbc.JDBC4Connection@3d1f558a
21:06:08.707 DEBUG 35275 --- [onnection adder] c.z.h.p.HikariPool        : test001 - Added connection com.mysql.jdbc.JDBC4Connection@51569e5b

项目启动后进行keepalive的心跳,对于JDBC4Connection@3d1f558a,可以看到第一次心跳是在21:07:03,第二次心跳是在21:07:58,差不多刚好53秒。

注:为什么两个connection的心跳检查时间差了4秒?因为我在创建的时候有打了断点,耽误了第2个connection创建的时间。

21:07:03.207 DEBUG 35275 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - keepalive: connection com.mysql.jdbc.JDBC4Connection@3d1f558a is alive
21:07:07.944 DEBUG 35275 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - keepalive: connection com.mysql.jdbc.JDBC4Connection@51569e5b is alive
21:07:58.421 DEBUG 35275 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - keepalive: connection com.mysql.jdbc.JDBC4Connection@3d1f558a is alive
21:08:07.184 DEBUG 35275 --- [001 housekeeper] c.z.h.p.HikariPool        : test001 - keepalive: connection com.mysql.jdbc.JDBC4Connection@51569e5b is alive

4. 其它一些不常用的配置

  • initializationFailTimeout
  • isolateInternalQueries
  • allowPoolSuspension
  • readOnly
  • registerMbeans
  • catalog
  • connectionInitSql
  • driverClassName
  • transactionIsolation
  • validationTimeout
  • leakDetectionThreshold
  • dataSource
  • schema
  • threadFactory
  • scheduledExecutor