如何实现一个简易的动态线程池的spring boot starter

进阶的Farmer
462 阅读6分钟

前言

在开发中，我们项目许多功能都依赖多线程，所以线程池使用很频繁，我们计算核心线程数和最大线程数采用的公式为

image.png

但是这往往没有考虑到一个应用中会使用多个线程池，且一台机器也可能会部署多个应用。

并发任务的执行情况和任务类型相关，IO密集型和CPU密集型的任务运行起来的情况差异非常大，但这种占比是较难合理预估的，这导致很难有一个简单有效的通用公式帮我们直接计算出结果。

虽然不能很好的提前估计线程数量，但是可以将修改线程的成本降下来，组长把这个任务分配给了我，功能也不要太复杂，只要可以支持通过接口修改核心线程数和。刚好网上资料很多，我简单编写了一个适合我们项目的spring boot starter。

参考资料:

[1] Java线程池实现原理及其在美团业务中的实践 [2] 线程池合适的线程数量是多少? [3] 美团一面：Spring Cloud 如何构建动态线程池？ [4] Define a Spring RestController via Java configuration [5] 如何创建自己的Spring Boot Starter并为其编写单元测试

实现流程

依赖

首先需要创建一个spring boot, 删除启动器，下面是我用到的依赖

 <dependencies>

	<!-- 基础依赖 -->
	<dependency>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter</artifactId>
	</dependency>
	<!-- 编译时依赖 -->
	<dependency>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-configuration-processor</artifactId>
		<optional>true</optional>
	</dependency>

	<!-- 需要提供修改接口 -->
	<dependency>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-web</artifactId>
	</dependency>

	<!-- 日志打印对象json信息 -->
	<dependency>
		<groupId>com.alibaba</groupId>
		<artifactId>fastjson</artifactId>
		<version>2.0.49</version>
	</dependency>

	<!-- 工具类 -->
	<dependency>
		<groupId>commons-lang</groupId>
		<artifactId>commons-lang</artifactId>
		<version>2.6</version>
	</dependency>
</dependencies>

配置服务

线程池参数配置实体

/**
 * 线程池配置实体对象
 */
public class ThreadPoolConfigEntity {

    /**
     * 应用名称
     */
    private String appName;

    /**
     * 线程池名称
     */
    private String threadPoolName;

    /**
     * 核心线程数
     */
    private Integer corePoolSize;

    /**
     * 最大线程数
     */
    private Integer maximumPoolSize;

    /**
     * 当前活跃线程数
     */
    private Integer activeCount;

    /**
     * 当前池中线程数
     */
    private Integer poolSize;

    /**
     * 队列类型
     */
    private String queueType;

    /**
     * 当前队列任务数
     */
    private Integer queueSize;

    /**
     * 当前队列剩余任容量
     */
    private Integer remainingCapacity;

    public ThreadPoolConfigEntity() {
    }

    public ThreadPoolConfigEntity(String appName, String threadPoolName) {
        this.appName = appName;
        this.threadPoolName = threadPoolName;
    }
    // 省略getter和setter
}

线程池配置Service

接口和实现类, 这里省略查询单个线程池的信息功能（下班后在自己电脑上写的，省略了一些内容）。

public interface IDynamicThreadPoolService {

    /**
     * 应用下的所有线程池的信息
     *
     * @return 应用下的所有线程池的信息
     */
    List<ThreadPoolConfigEntity> queryThreadPoolList();

    /**
     * 更新线程池配置，主要用到的参数有
     * appName，threadPoolName，corePoolSize，maximumPoolSize
     *
     * @param threadPoolConfigEntity 修改请求参数
     * @return 成功
     */
    boolean updateThreadPoolConfig(ThreadPoolConfigEntity threadPoolConfigEntity);
}

public class DynamicThreadPoolService implements IDynamicThreadPoolService {

    private final Logger logger = LoggerFactory.getLogger(DynamicThreadPoolService.class);

    private final Map<String, ThreadPoolExecutor> threadPoolExecutorMap;

    private final String applicationName;

    public DynamicThreadPoolService(String applicationName, Map<String, ThreadPoolExecutor> threadPoolExecutorMap) {
        this.applicationName = applicationName;
        this.threadPoolExecutorMap = threadPoolExecutorMap;
    }

    @Override
    public List<ThreadPoolConfigEntity> queryThreadPoolList() {
        Set<String> threadPoolBeanNames = threadPoolExecutorMap.keySet();
        List<ThreadPoolConfigEntity> threadPoolVOS = new ArrayList<>(threadPoolBeanNames.size());
        for (Map.Entry<String, ThreadPoolExecutor> entry : threadPoolExecutorMap.entrySet()) {
            String beanName = entry.getKey();
            ThreadPoolExecutor threadPoolExecutor = entry.getValue();
            ThreadPoolConfigEntity threadPoolConfigVO = new ThreadPoolConfigEntity(applicationName, beanName);
            threadPoolConfigVO.setCorePoolSize(threadPoolExecutor.getCorePoolSize());
            threadPoolConfigVO.setMaximumPoolSize(threadPoolExecutor.getMaximumPoolSize());
            threadPoolConfigVO.setActiveCount(threadPoolExecutor.getActiveCount());
            threadPoolConfigVO.setPoolSize(threadPoolExecutor.getPoolSize());
            threadPoolConfigVO.setQueueType(threadPoolExecutor.getQueue().getClass().getSimpleName());
            threadPoolConfigVO.setQueueSize(threadPoolExecutor.getQueue().size());
            threadPoolConfigVO.setRemainingCapacity(threadPoolExecutor.getQueue().remainingCapacity());
            threadPoolVOS.add(threadPoolConfigVO);
        }
        return threadPoolVOS;
    }

    @Override
    public boolean updateThreadPoolConfig(ThreadPoolConfigEntity threadPoolConfigEntity) {
        if (null == threadPoolConfigEntity || !applicationName.equals(threadPoolConfigEntity.getAppName())) {
            logger.error("修改线程池配置时：{}-{}线程池不属于应用：{}",
                    threadPoolConfigEntity.getAppName(),
                    threadPoolConfigEntity.getThreadPoolName(),
                    applicationName);
            return false;
        }

        ThreadPoolExecutor threadPoolExecutor = threadPoolExecutorMap.get(threadPoolConfigEntity.getThreadPoolName());
        if (null == threadPoolExecutor) {
            logger.error("当前应用:{}不存在线程池:{}", applicationName, threadPoolConfigEntity.getThreadPoolName());
            return false;
        }

        // 设置参数 「调整核心线程数和最大线程数」
        if (threadPoolConfigEntity.getCorePoolSize() != null) {
            threadPoolConfigEntity.setCorePoolSize(threadPoolConfigEntity.getCorePoolSize());
        }
        if (threadPoolConfigEntity.getMaximumPoolSize() != null) {
            threadPoolExecutor.setMaximumPoolSize(threadPoolConfigEntity.getMaximumPoolSize());
        }
        return true;
    }
}

查询和修改接口

省略了查询单个线程池信息的接口。

@RestController
@RequestMapping("/dynamic_thread_pool/")
public class DynamicThreadPoolController {

    private final Logger logger = LoggerFactory.getLogger(DynamicThreadPoolService.class);

    private final IDynamicThreadPoolService dynamicThreadPoolService;

    public DynamicThreadPoolController(IDynamicThreadPoolService dynamicThreadPoolService) {
        this.dynamicThreadPoolService = dynamicThreadPoolService;
    }

    /**
     * 查询线程池数据
     */
    @RequestMapping(value = "query_thread_pool_list", method = RequestMethod.GET)
    public Response<List<ThreadPoolConfigEntity>> queryThreadPoolList() {
        try {
            List<ThreadPoolConfigEntity> threadPoolConfigEntities = dynamicThreadPoolService.queryThreadPoolList();
            return Response.success(threadPoolConfigEntities);
        } catch (Exception e) {
            logger.error("查询线程池数据异常", e);
            return Response.error();
        }
    }


    /**
     * 修改线程池配置
     */
    @RequestMapping(value = "update_thread_pool_config", method = RequestMethod.POST)
    public Response<Boolean> updateThreadPoolConfig(@RequestBody ThreadPoolConfigEntity request) {
        try {
            boolean result = dynamicThreadPoolService.updateThreadPoolConfig(request);
            if (result) {
                return Response.success(true);
            } else {
                return Response.success(false);
            }
        } catch (Exception e) {
            logger.error("修改线程池配置异常 {}", request, e);
            return Response.error();
        }
    }
}

自动配置

/**
 * 动态配置入口
 */
@Configuration
@ComponentScan(basePackages = "com.jframe.dynamic.thread.pool.controller")
@ConditionalOnProperty(name = "dynamic.threadpool.enabled", havingValue = "true")
public class DynamicThreadPoolAutoConfig {

    private final Logger logger = LoggerFactory.getLogger(DynamicThreadPoolAutoConfig.class);

    @Bean("dynamicThreadPollService")
    public DynamicThreadPoolService dynamicThreadPollService(ApplicationContext applicationContext, Map<String, ThreadPoolExecutor> threadPoolExecutorMap) {
        String applicationName = applicationContext.getEnvironment().getProperty("spring.application.name");
        if (StringUtils.isBlank(applicationName)) {
            applicationName = "缺省的";
            logger.warn("动态线程池，启动提示。SpringBoot 应用未配置 spring.application.name 无法获取到应用名称！");
        }
        return new DynamicThreadPoolService(applicationName, threadPoolExecutorMap);
    }
}

建议spring starter的Bean都通过自动配置类去创建，这样就可以用@ConditionalOnProperty, @ConditionalOnBean, @ConditionalOnMissingBean等注解条件化加载加载Bean，使自动配置更加灵活和可定制，根据应用环境或上下文的不同选择是否加载某些Bean。

@ComponentScan(basePackages = "com.jframe.dynamic.thread.pool.controller")
@ConditionalOnProperty(name = "dynamic.threadpool.enabled", havingValue = "true")
  1. @ConditionalOnProperty注解表示存在配置dynamic.threadpool.enabled且值为true时就加载该配置类中的Bean。
  2. @ComponentScan: @ConditionalOnProperty条件判断通过就扫描控制器包，以提供接口给业务软件使用。

测试

创建spring boot项目来进行测试，引入组件
<dependency>
	<groupId>com.jframe</groupId>
	<artifactId>dynamic-thread-pool-spring-boot-starter</artifactId>
	<version>1.0.0</version>
</dependency>
创建线程池Bean

这里就贴一个线程池实例的代码

@Bean("threadPoolExecutor01")
public ThreadPoolExecutor threadPoolExecutor01(ThreadPoolConfigProperties properties) {
	// 实例化策略
	RejectedExecutionHandler handler;
	switch (properties.getPolicy()){
		case "AbortPolicy":
			handler = new ThreadPoolExecutor.AbortPolicy();
			break;
		case "DiscardPolicy":
			handler = new ThreadPoolExecutor.DiscardPolicy();
			break;
		case "DiscardOldestPolicy":
			handler = new ThreadPoolExecutor.DiscardOldestPolicy();
			break;
		case "CallerRunsPolicy":
			handler = new ThreadPoolExecutor.CallerRunsPolicy();
			break;
		default:
			handler = new ThreadPoolExecutor.AbortPolicy();
			break;
	}

	// 创建线程池
	return new ThreadPoolExecutor(properties.getCorePoolSize(),
			properties.getMaxPoolSize(),
			properties.getKeepAliveTime(),
			TimeUnit.SECONDS,
			new LinkedBlockingQueue<>(properties.getBlockQueueSize()),
			Executors.defaultThreadFactory(),
			handler);
}
application.yml
spring:
  application:
    name: dynamic-thread-pool-test-app
# 开启动态线程池配置
dynamic:
  threadpool:
    enabled: true
模拟给线程池提交任务
@SpringBootApplication
@Configurable
public class Application implements ApplicationRunner {
    private final ExecutorService threadPoolExecutor01;

    public Application(ExecutorService threadPoolExecutor01) {
        this.threadPoolExecutor01 = threadPoolExecutor01;
    }

    public static void main(String[] args) {
        SpringApplication.run(Application.class);
    }

    @Override
    public void run(ApplicationArguments args) throws Exception {
        new Thread(
                () -> {
                    while (true) {
                        // 创建一个随机时间生成器
                        Random random = new Random();
                        // 随机时间，用于模拟任务启动延迟
                        int initialDelay = random.nextInt(10) + 1; // 1到10秒之间
                        // 随机休眠时间，用于模拟任务执行时间
                        int sleepTime = random.nextInt(10) + 1; // 1到10秒之间

                        // 提交任务到线程池
                        threadPoolExecutor01.submit(
                                () -> {
                                    try {
                                        // 模拟任务启动延迟
                                        TimeUnit.SECONDS.sleep(initialDelay);
                                        // System.out.println("Task started after " + initialDelay + " seconds.");

                                        // 模拟任务执行
                                        TimeUnit.SECONDS.sleep(sleepTime);
                                        // System.out.println("Task executed for " + sleepTime + " seconds.");
                                    } catch (InterruptedException e) {
                                        Thread.currentThread().interrupt();
                                    }
                                });

                        try {
                            Thread.sleep(random.nextInt(50) + 1);
                        } catch (InterruptedException e) {
                            throw new RuntimeException(e);
                        }
                    }
                })
                .start();
    }
}

src/main/resource/META-INF/spring.factories目录配置自动配置类

org.springframework.boot.autoconfigure.EnableAutoConfiguration=com.jframe.dynamic.thread.pool.config.DynamicThreadPoolAutoConfig

注意：Spring Boot 2.7开始，不再推荐使用spring.factories，而是改用/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports，文件内容直接放需要自动加载配置类路径即可。这个变更具体可见之前的这篇文章：《Spring Boot 2.7开始spring.factories不推荐使用了》

访问接口测试

http://localhost:8093/dynamic_thread_pool/query_thread_pool_list

image.png

可以看到threadPoolExecutor01存在任务，而threadPoolExecutor02中不存在。 post http://localhost:8093/dynamic_thread_pool/update_thread_pool_config

{
    "appName": "dynamic-thread-pool-test-app",
    "threadPoolName": "threadPoolExecutor01",
    "corePoolSize": 15,
    "maximumPoolSize": 30
}

咋此查询线程状态

image.png

可见已经修改成功，且activeCount(当前活跃线程数)和poolSize(池中线程数)已经变化。

注意

因为还存在@ConditionalOnProperty(name = "dynamic.threadpool.enabled", havingValue = "true")注解，还需要判断配置不存在或为false是否能正常访问接口等。

心得和思考

心得

通过此次经历:

  1. 了解了线程池运行时的一些可配置和修改参数以及可线程池可查询的信息。
  2. 学会了如何创建一个Spring Boot Start。

思考

还存在一些可以优化的地方，比如可以结和数据库、nacos等工具让配置持久化；加入一些监控组件。

avatar
文章
阅读
粉丝