动态线程池-DynamicTp源码浅析动态线程池的架构设计。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。。

技术架构

启动流程

核心组件

spring中的配置


package org.dromara.dynamictp.spring;


import org.dromara.dynamictp.common.properties.DtpProperties;
import org.dromara.dynamictp.core.DtpRegistry;
import org.dromara.dynamictp.core.monitor.DtpMonitor;
import org.dromara.dynamictp.core.lifecycle.DtpLifecycle;
import org.dromara.dynamictp.core.lifecycle.LifeCycleManagement;
import org.dromara.dynamictp.core.support.DtpBannerPrinter;
import org.dromara.dynamictp.spring.lifecycle.DtpLifecycleSpringAdapter;
import org.dromara.dynamictp.spring.listener.DtpApplicationListener;
import org.springframework.beans.factory.config.BeanDefinition;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Role;




/**







DtpBaseBeanConfiguration related







@author yanhom






@since 1.0.0
**/
@Configuration
@Role(BeanDefinition.ROLE_INFRASTRUCTURE)
public class DtpBaseBeanConfiguration {




@Bean
public DtpProperties dtpProperties() {
//        使用了单例，因为这些配置在spring的生命周期中进行了解析录入
return DtpProperties.getInstance();
}




@Bean
public DtpLifecycle dtpLifecycle() {
return new DtpLifecycle();
}




@Bean
public DtpRegistry dtpRegistry(DtpProperties dtpProperties) {
return new DtpRegistry(dtpProperties);
}




@Bean
public DtpMonitor dtpMonitor(DtpProperties dtpProperties) {
return new DtpMonitor(dtpProperties);
}




@Bean
public DtpBannerPrinter dtpBannerPrinter() {
return DtpBannerPrinter.getInstance();
}




@Bean
public DtpLifecycleSpringAdapter dtpLifecycleSpringAdapter(LifeCycleManagement lifeCycleManagement) {
return new DtpLifecycleSpringAdapter(lifeCycleManagement);
}




@Bean
public DtpApplicationListener dtpApplicationListener() {
return new DtpApplicationListener();
}
}

DtpBaseBeanConfiguration related



@author yanhom


@since 1.0.0
**/
@Configuration
@Role(BeanDefinition.ROLE_INFRASTRUCTURE)
public class DtpBaseBeanConfiguration {
@Bean
public DtpProperties dtpProperties() {
//        使用了单例，因为这些配置在spring的生命周期中进行了解析录入
return DtpProperties.getInstance();
}
@Bean
public DtpLifecycle dtpLifecycle() {
return new DtpLifecycle();
}
@Bean
public DtpRegistry dtpRegistry(DtpProperties dtpProperties) {
return new DtpRegistry(dtpProperties);
}
@Bean
public DtpMonitor dtpMonitor(DtpProperties dtpProperties) {
return new DtpMonitor(dtpProperties);
}
@Bean
public DtpBannerPrinter dtpBannerPrinter() {
return DtpBannerPrinter.getInstance();
}
@Bean
public DtpLifecycleSpringAdapter dtpLifecycleSpringAdapter(LifeCycleManagement lifeCycleManagement) {
return new DtpLifecycleSpringAdapter(lifeCycleManagement);
}
@Bean
public DtpApplicationListener dtpApplicationListener() {
return new DtpApplicationListener();
}
}

在spring包

配置文件解析

在注册之前，通过ConfigParser接口解析出配置

默认支持json，properties，yaml

配置文件

DtpProperties是配置类

private static class Holder {
        private static final DtpProperties INSTANCE = new DtpProperties();
    }

这里后面为了解耦spring，使用单例模式，创建出来后在spring的后置处理器中对配置文件进行了解析注入

创建线程池到spring中

 @Override
    public void registerBeanDefinitions(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry) {
        DtpProperties dtpProperties = DtpProperties.getInstance();
        BinderHelper.bindDtpProperties(environment, dtpProperties);
        val executors = dtpProperties.getExecutors();
        if (CollectionUtils.isEmpty(executors)) {
            log.info("DynamicTp registrar, no executors are configured.");
            return;
        }
// 将每个线程池的配置都放进spring注册中心
        executors.forEach(e -> {
            if (!e.isAutoCreate()) {
                return;
            }
            Class<?> executorTypeClass = ExecutorType.getClass(e.getExecutorType());
            Map<String, Object> propertyValues = buildPropertyValues(e);
            Object[] args = buildConstructorArgs(executorTypeClass, e);
            BeanRegistrationUtil.register(registry, e.getThreadPoolName(), executorTypeClass, propertyValues, args);
        });
    }

将配置文件中的所有文件都注册进beandefinition

线程池类型：

EagerDtpExecutor 使用TaskQueue
PriorityDtpExecutor使用PriorityBlockingQueue
普通的使用配置的队列

DtpRegistry

在DtpPostProcessor中，对bean进行后置处理，将线程池放进DtpRegistry


//    后置处理，获取到bean后，对bean进行增强，并注册到DtpRegistry中
    @Override
    public Object postProcessAfterInitialization(@NonNull Object bean, @NonNull String beanName) throws BeansException {
//      对是 ThreadPoolExecutor或者 ThreadPoolTaskExecutor类型的bean进行处理，如果不是直接退出
        if (!(bean instanceof ThreadPoolExecutor) && !(bean instanceof ThreadPoolTaskExecutor)) {
            return bean;
        }
        if (bean instanceof DtpExecutor) {
            return registerAndReturnDtp(bean);
        }
        // register juc ThreadPoolExecutor or ThreadPoolTaskExecutor
        return registerAndReturnCommon(bean, beanName);
    }

动态刷新

refresfer使用了模板方法。

nacos：

nacos会自己发布刷新事件，所以直接注册进spring就可以了

别的组件刷新

在DtpAdapterListener中

有Tomcat等

线程池类型

在新版本的源码中，好像是换成了ExecutorWrapper包装线程池的参数，配置等

在wrapper中，有ExecutorAdapter

抽象了两层差不多

核心DtpExecutor

io密集型`EagerDtpExecutor`

为什么是io：

当核心线程都处于繁忙状态时，创建新的线程，而不是放入阻塞队列

核心是**TaskQueue**


    @Override
    public boolean offer(@NonNull Runnable runnable) {
        if (executor == null) {
            throw new RejectedExecutionException("The task queue does not have executor.");
        }
        if (executor.getPoolSize() == executor.getMaximumPoolSize()) {
            return super.offer(runnable);
        }
        // have free worker. put task into queue to let the worker deal with task.
        if (executor.getSubmittedTaskCount() <= executor.getPoolSize()) {
            return super.offer(runnable);
        }
        // return false to let executor create new worker.
        //   还没到最大的线程,返回false让父创建线程
        if (executor.getPoolSize() < executor.getMaximumPoolSize()) {
            return false;
        }
        // currentPoolThreadSize >= max
        return super.offer(runnable);
    }

报警通知

工厂模式-枚举

 private static final ExecutorService ALARM_EXECUTOR = ThreadPoolBuilder.newBuilder()
            .threadFactory("dtp-alarm")
            .corePoolSize(1)
            .maximumPoolSize(1)
            .workQueue(LINKED_BLOCKING_QUEUE.getName(), 2000)
            .rejectedExecutionHandler(RejectedTypeEnum.DISCARD_OLDEST_POLICY.getName())
            .rejectEnhanced(false)
            .taskWrappers(TaskWrappers.getInstance().getByNames(Sets.newHashSet("mdc")))
            .buildDynamic();

    private static final InvokerChain<BaseNotifyCtx> ALARM_INVOKER_CHAIN;

    static {
//        构建责任链
        ALARM_INVOKER_CHAIN = NotifyFilterBuilder.getAlarmInvokerChain();
    }

然后调用到下面


//    构造？
    public static InvokerChain<BaseNotifyCtx> getAlarmInvokerChain() {
//        spi机制加载的spring容器（解耦了）
        val filters = ContextManagerHelper.getBeansOfType(NotifyFilter.class);
        Collection<NotifyFilter> alarmFilters = Lists.newArrayList(filters.values());
//        责任链？
        alarmFilters.add(new AlarmBaseFilter());
        alarmFilters = alarmFilters.stream()
                .filter(x -> x.supports(NotifyTypeEnum.ALARM))
                .sorted(Comparator.comparing(Filter::getOrder))
                .collect(Collectors.toList());
        return InvokerChainFactory.buildInvokerChain(new AlarmInvoker(), alarmFilters.toArray(new NotifyFilter[0]));
    }

InvokerChain是核心，然后proceed方法调用Invoker

最结尾的是AlarmInvoker然后中间是一个包装了Filter（AlarmBaseFilter）的Invoker

核心就是AlarmInvoker 然后进行遍历通知

监控

在DtpMonitor中的run方法中进行监控

//    核心方法
    private void run() {
        Set<String> executorNames = DtpRegistry.getAllExecutorNames();
        try {
//            通知？
            checkAlarm(executorNames);
//            核心方法
            collectMetrics(executorNames);
        } catch (Exception e) {
            log.error("DynamicTp monitor, run error", e);
        }
    }

核心就是将线程池的线程的数据转为Bean，然后调用被监控端

@Slf4j
public class LogCollector extends AbstractCollector {

    @Override
    public void collect(ThreadPoolStats threadPoolStats) {
        String metrics = JsonUtil.toJson(threadPoolStats);
        if (LogHelper.getMonitorLogger() == null) {
            log.error("Cannot find monitor logger...");
            return;
        }
        LogHelper.getMonitorLogger().info("{}", metrics);
    }

    @Override
    public String type() {
        return CollectorTypeEnum.LOGGING.name().toLowerCase();
    }
}

待处理

HashedWheelTimer

tomcat的自动刷新TomcatDtpAdapter

Handler进行单例模式的设计

public final class CollectorHandler {


    private CollectorHandler() {
     
    }

    public static CollectorHandler getInstance() {
        return CollectorHandlerHolder.INSTANCE;
    }

    private static class CollectorHandlerHolder {
        private static final CollectorHandler INSTANCE = new CollectorHandler();
    }
}

使用到的类库

Equator
EventBus
mdc

动态线程池-DynamicTp源码浅析