xxl-job 使用与优化

用户3561325137693
501 阅读7分钟

概述

XXL-JOB是一个分布式任务调度平台，其核心设计目标是开发迅速、学习简单、轻量级、易扩展。 采用中心式调度室设计，提供丰富的触发策略和调度策略， 支持弹性扩容缩容、查看任务运行报表、以及邮件报警等众多实用功能。

中文文档： www.xuxueli.com/xxl-job/ Git 地址： github.com/xuxueli/xxl… Gitee 地址： gitee.com/xuxueli0323…

环境要求

  • Maven 3+
  • Mysql 8.0+
  • xxl-job 2.x 要求JDK 8
  • xxl-job 3.x 要求 JDK 17+

下载

由于目前使用的JDK 版本为 JDK8, 所以这里以 xxl-job-2.5.0 ， maven-3.9.8 为例

方式一

去 github 下载, 根据使用的宿主机选择对应的 zip 文件或 tar.gz 文件

在这里插入图片描述

方式二

下载源码自行编译， 切到 2.5.0-release 分支打出对应jar包， 在 xxl-job-admin 目录下的 xxl-jov-admin-2.5.0.jar 即为任务调度中心（注册中心）jar包

在这里插入图片描述

部署任务调度中心

任务调度中心其实就是一个spring boot 项目， 用 java -jar 启动即可， 这里推荐使用外部配置启动， 改一下占用端口（默认为8080）

将 jar 里面的 application.properties 文件拷贝出来放在在同级目录下， 将 web 和 datasource 配置改为自己的配置， 其他配置属性按需修改。 在这里插入图片描述 启动命令

java -jar xxl-job-admin-2.5.0.jar

启动成功后访问 http://localhost:8080/xxl-job-admin , 默认用户名和密码为 admin/123456

在这里插入图片描述

接入xxl-job

Spring Boot 项目

引入依赖

<dependency>
	<groupId>com.xuxueli</groupId>
	<artifactId>xxl-job-core</artifactId>
	<version>2.5.0</version>
</dependency>

在 resources/application.properties 配置文件中增加xxl-job 配置

# web port
server.port=8081
# no web
#spring.main.web-environment=false

# log config
logging.config=classpath:logback.xml


### xxl-job admin address list, such as "http://address" or "http://address01,http://address02"
xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin
### xxl-job, access token
xxl.job.admin.accessToken=default_token
### xxl-job timeout by second, default 3s
xxl.job.admin.timeout=3

### xxl-job executor appname
xxl.job.executor.appname=xxl-job-executor-sample
### xxl-job executor registry-address: default use address to registry , otherwise use ip:port if address is null
xxl.job.executor.address=
### xxl-job executor server-info
xxl.job.executor.ip=
xxl.job.executor.port=9999
### xxl-job executor log-path
xxl.job.executor.logpath=/data/applogs/xxl-job/jobhandler
### xxl-job executor log-retention-days
xxl.job.executor.logretentiondays=30

新建配置类


import com.xxl.job.core.executor.impl.XxlJobSpringExecutor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class XxlJobConfig {
    private Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);

    @Value("${xxl.job.admin.addresses}")
    private String adminAddresses;

    @Value("${xxl.job.admin.accessToken}")
    private String accessToken;

    @Value("${xxl.job.admin.timeout}")
    private int timeout;

    @Value("${xxl.job.executor.appname}")
    private String appname;

    @Value("${xxl.job.executor.address}")
    private String address;

    @Value("${xxl.job.executor.ip}")
    private String ip;

    @Value("${xxl.job.executor.port}")
    private int port;

    @Value("${xxl.job.executor.logpath}")
    private String logPath;

    @Value("${xxl.job.executor.logretentiondays}")
    private int logRetentionDays;
    
    @Bean
    public XxlJobSpringExecutor xxlJobExecutor() {
        logger.info(">>>>>>>>>>> xxl-job config init.");
        XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
        xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
        xxlJobSpringExecutor.setAppname(appname);
        xxlJobSpringExecutor.setAddress(address);
        xxlJobSpringExecutor.setIp(ip);
        xxlJobSpringExecutor.setPort(port);
        xxlJobSpringExecutor.setAccessToken(accessToken);
        xxlJobSpringExecutor.setTimeout(timeout);
        xxlJobSpringExecutor.setLogPath(logPath);
        xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);

        return xxlJobSpringExecutor;
    }
}

新建任务执行类, 通过注解标注 JobHandler， 通过 XxlJobHelper.log 打印执行日志

import com.xxl.job.core.context.XxlJobHelper;
import com.xxl.job.core.handler.annotation.XxlJob;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Component
public class SampleXxlJob {
    private static Logger logger = LoggerFactory.getLogger(SampleXxlJob.class);

    /**
     * 1、简单任务示例（Bean模式）
     */
    @XxlJob("demoJobHandler")
    public void demoJobHandler() throws Exception {
        XxlJobHelper.log("XXL-JOB, Hello World.");
    }
    
}

非框架项目

这里以简单的 web 项目为例

增加 xxl-job 属性配置文件 xxl-job-executor.properties

### xxl-job admin address list, such as "http://address" or "http://address01,http://address02"
xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin
### xxl-job access-token
xxl.job.admin.accessToken=default_token
### xxl-job timeout by second, default 3s
xxl.job.admin.timeout=3

### xxl-job executor appname
xxl.job.executor.appname=sublet-task
### xxl-job executor registry-address: default use address to registry , otherwise use ip:port if address is null
xxl.job.executor.address=
### xxl-job executor server-info
xxl.job.executor.ip=
xxl.job.executor.port=9997
### xxl-job executor log-path
xxl.job.executor.logpath=/data/applogs/xxl-job/jobhandler
### xxl-job executor log-retention-days
xxl.job.executor.logretentiondays=30

新建一个配置类, 读取 xxl-job-executor.properties 配置文件

public class XxlJobConfig {
	
    private static Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);

    private static XxlJobConfig instance = new XxlJobConfig();
    public static XxlJobConfig getInstance() {
        return instance;
    }

    private XxlJobSimpleExecutor xxlJobExecutor = null;

    /**
     * init
     */
    public void initXxlJobExecutor() {
        // load executor prop
        Properties xxlJobProp = loadProperties("xxl-job-executor.properties");

        // init executor
        xxlJobExecutor = new XxlJobSimpleExecutor();
        xxlJobExecutor.setAdminAddresses(xxlJobProp.getProperty("xxl.job.admin.addresses"));
        xxlJobExecutor.setAccessToken(xxlJobProp.getProperty("xxl.job.admin.accessToken"));
        xxlJobExecutor.setTimeout(Integer.valueOf(xxlJobProp.getProperty("xxl.job.admin.timeout")));
        xxlJobExecutor.setAppname(xxlJobProp.getProperty("xxl.job.executor.appname"));
        xxlJobExecutor.setAddress(xxlJobProp.getProperty("xxl.job.executor.address"));
        xxlJobExecutor.setIp(xxlJobProp.getProperty("xxl.job.executor.ip"));
        xxlJobExecutor.setPort(Integer.valueOf(xxlJobProp.getProperty("xxl.job.executor.port")));
        xxlJobExecutor.setLogPath(xxlJobProp.getProperty("xxl.job.executor.logpath"));
        xxlJobExecutor.setLogRetentionDays(Integer.valueOf(xxlJobProp.getProperty("xxl.job.executor.logretentiondays")));

        // registry job bean
        xxlJobExecutor.setXxlJobBeanList(Arrays.asList(new SampleXxlJob()));

        // start executor
        try {
            xxlJobExecutor.start();
        } catch (Exception e) {
            logger.error(e.getMessage(), e);
        }
    }

    /**
     * destroy
     */
    public void destroyXxlJobExecutor() {
        if (xxlJobExecutor != null) {
            xxlJobExecutor.destroy();
        }
    }

    public static Properties loadProperties(String propertyFileName) {
        InputStreamReader in = null;
        try {
            ClassLoader loder = Thread.currentThread().getContextClassLoader();

            in = new InputStreamReader(loder.getResourceAsStream(propertyFileName), "UTF-8");;
            if (in != null) {
                Properties prop = new Properties();
                prop.load(in);
                return prop;
            }
        } catch (IOException e) {
            logger.error("load {} error!", propertyFileName);
        } finally {
            if (in != null) {
                try {
                    in.close();
                } catch (IOException e) {
                    logger.error("close {} error!", propertyFileName);
                }
            }
        }
        return null;
    }

}

编写监听类， 需要注册到 tomcat 中, 伴随 tomcat 服务的启动和销毁一起进行

import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;

public class XxlJobListener implements ServletContextListener {

    @Override
    public void contextInitialized(ServletContextEvent sce) {
        XxlJobConfig.getInstance().initXxlJobExecutor();
    }

    @Override
    public void contextDestroyed(ServletContextEvent sce) {
        XxlJobConfig.getInstance().destroyXxlJobExecutor();
    }
}

在 web.xml 中注册监听器 XxlJobListener

<listener>
	<listener-class>com..XxlJobListener</listener-class>
</listener>

新建任务执行类， 需要在 XxlJobConfig 中通过 xxlJobExecutor.setXxlJobBeanList(Arrays.asList(new SampleXxlJob()))注册

public class SampleXxlJob {
    private static Logger logger = LoggerFactory.getLogger(SampleXxlJob.class);

    @XxlJob("demoJobHandler")
    public void demoJobHandler() throws Exception {
        XxlJobHelper.log("XXL-JOB, Hello World.");
    }
}

自动注册JobBeanList(可选)

创建接口类， 仅用于标识是一个任务类

public interface Job {
    //Implement this interface to indicate that this is a task class.
}

创建任务类， 实现 Job 接口类

public class MyXxlJob implements Job {

    private static Logger logger = LoggerFactory.getLogger(MyXxlJob.class);

    @XxlJob("demoJobHandler")
    public void demoJobHandler() throws Exception {
        logger.info("执行 xxljob");
    }
}

在 xxl-job-executor.properties 文件中新增属性， 配置 Job 扫描路径

# Configure your scanning path
xxl.job.executor.package=com.test.job

创建 XxlJobHelper 类， 根据配置的扫描路径获取符合条件的 JobBeanList,

public class XxlJobHelper {

    private static List<Class<?>> getImplementations(Class<?> interfaceClass, String packageNames) throws IOException, ClassNotFoundException {
        List<Class<?>> implementations = new ArrayList<>();
        for (String packageName : packageNames.split(",")) {
            if(packageName == null || packageName.isEmpty()){
                continue;
            }
            ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
            String path = packageName.replace('.', '/');
            Enumeration<URL> resources = classLoader.getResources(path);

            while (resources.hasMoreElements()) {
                URL resource = resources.nextElement();
                File directory = new File(resource.getFile());
                if (directory.exists()) {
                    for (File file : directory.listFiles()) {
                        dealFile(implementations, interfaceClass, packageName, file);
                    }
                }
            }
        }
        return implementations;
    }

    private static void dealFile(List<Class<?>> implementations, Class<?> interfaceClass, String packageName, File file){
        if(file.isDirectory()){
            for(File child : file.listFiles()){
                dealFile(implementations, interfaceClass, packageName + "." + file.getName(), child);
            }
        }
        if (file.getName().endsWith(".class")) {
            String className = packageName + '.' + file.getName().substring(0, file.getName().length() - 6);
            try {
                Class<?> clazz = Class.forName(className);
                if (interfaceClass.isAssignableFrom(clazz) && !clazz.isInterface()) {
                    implementations.add(clazz);
                }
            } catch (ClassNotFoundException e) {
                // Class could not be loaded
            }
        }
    }

    public static List<Object> getXxlJobBeanList(String packageName){
        if(packageName == null || packageName.isEmpty()){
            throw new IllegalArgumentException("packageName is null or empty");
        }
        List<Object> list = new ArrayList<>();
        try {
            List<Class<?>> implementations = getImplementations(Job.class, packageName);
            for (Class<?> implementation : implementations) {
                list.add(implementation.newInstance());
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        return list;
    }
}

修改 XxlJobConfig.java , 配置JobBeanList 注册方式， 改成通过 XxlJobHelper 扫描出符合的 JobBeanList 注册。

// registry job bean
//xxlJobExecutor.setXxlJobBeanList(Arrays.asList(new SampleXxlJob()));
xxlJobExecutor.setXxlJobBeanList(XxlJobHelper.getXxlJobBeanList(xxlJobProp.getProperty("xxl.job.executor.package")));

这样后续增加其他的任务类， 我们就只需要实现指定的 Job 接口即可， 无需去修改 XxlJobConfig ， 方便其他业务代码的接入。

调度任务

添加执行器

项目接入 xxl-job 后， 会主动注册到 xxl_job_registry 表中， 其中， registry_key 是配置文件中配置的 xxl.job.executor.appname 属性值， registry_value 为服务地址

在这里插入图片描述

服务会自动注册， 但是执行器 （xxl_job_group）并不会主动注册，我们需要主动添加执行器

在执行器管理中， 点击新增

  • AppName : xxl.job.executor.appname 属性值
  • 名称： 执行器名称， 创建执行任务时下拉框显示的值， 长度不超过12
  • 注册方式： 如果是自动注册， xxl-job 有一个后台进程会去扫描 xxl_job_registry 表，将符合的地址注册到执行器中；如果是手动录入， 则机器地址必填
  • 机器地址 ：服务地址， 注册方式为手动录入时必填

在这里插入图片描述

添加任务

在任务管理， 点击新增，

  • JobHandler ： 注解中对应的值， @XxlJob("demoJobHandler")
  • Cron： 执行时间表达式 在这里插入图片描述

点击执行一次，然后查看日志， 测试配置是否成功， 如果没有配置机器地址，则会报错【执行器地址为空】

在这里插入图片描述

在这里插入图片描述

自动注册执行器

机器注册原理

项目引入 xxl-job 启动后， 会将自身信息注册到注册中心（xxl-job-admin）的 xxl_job_registry 表中。

xxl-job-admin 项目的 JobRegistryHelper 类中， 有一个start() 方法, 里面开了几个线程去扫描已注册的执行器表（xxl_job_group ）， 发现执行器后再去扫描 xxl_job_registry 表， 将各执行器的机器地址更新。

public void start(){

	// for registry or remove
	registryOrRemoveThreadPool = new ThreadPoolExecutor(
			2,
			10,
			30L,
			TimeUnit.SECONDS,
			new LinkedBlockingQueue<Runnable>(2000),
			new ThreadFactory() {
				@Override
				public Thread newThread(Runnable r) {
					return new Thread(r, "xxl-job, admin JobRegistryMonitorHelper-registryOrRemoveThreadPool-" + r.hashCode());
				}
			},
			new RejectedExecutionHandler() {
				@Override
				public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
					r.run();
					logger.warn(">>>>>>>>>>> xxl-job, registry or remove too fast, match threadpool rejected handler(run now).");
				}
			});

	// for monitor
	registryMonitorThread = new Thread(new Runnable() {
		@Override
		public void run() {
			while (!toStop) {
				try {
					// auto registry group
					List<XxlJobGroup> groupList = XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().findByAddressType(0);
					if (groupList!=null && !groupList.isEmpty()) {

						// remove dead address (admin/executor)
						List<Integer> ids = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().findDead(RegistryConfig.DEAD_TIMEOUT, new Date());
						if (ids!=null && ids.size()>0) {
							XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().removeDead(ids);
						}

						// fresh online address (admin/executor)
						HashMap<String, List<String>> appAddressMap = new HashMap<String, List<String>>();
						List<XxlJobRegistry> list = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().findAll(RegistryConfig.DEAD_TIMEOUT, new Date());
						if (list != null) {
							for (XxlJobRegistry item: list) {
								if (RegistryConfig.RegistType.EXECUTOR.name().equals(item.getRegistryGroup())) {
									String appname = item.getRegistryKey();
									List<String> registryList = appAddressMap.get(appname);
									if (registryList == null) {
										registryList = new ArrayList<String>();
									}

									if (!registryList.contains(item.getRegistryValue())) {
										registryList.add(item.getRegistryValue());
									}
									appAddressMap.put(appname, registryList);
								}
							}
						}

						// fresh group address
						for (XxlJobGroup group: groupList) {
							List<String> registryList = appAddressMap.get(group.getAppname());
							String addressListStr = null;
							if (registryList!=null && !registryList.isEmpty()) {
								Collections.sort(registryList);
								StringBuilder addressListSB = new StringBuilder();
								for (String item:registryList) {
									addressListSB.append(item).append(",");
								}
								addressListStr = addressListSB.toString();
								addressListStr = addressListStr.substring(0, addressListStr.length()-1);
							}
							group.setAddressList(addressListStr);
							group.setUpdateTime(new Date());

							XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().update(group);
						}
					}
				} catch (Throwable e) {
					if (!toStop) {
						logger.error(">>>>>>>>>>> xxl-job, job registry monitor thread error:{}", e);
					}
				}
				try {
					TimeUnit.SECONDS.sleep(RegistryConfig.BEAT_TIMEOUT);
				} catch (Throwable e) {
					if (!toStop) {
						logger.error(">>>>>>>>>>> xxl-job, job registry monitor thread error:{}", e);
					}
				}
			}
			logger.info(">>>>>>>>>>> xxl-job, job registry monitor thread stop");
		}
	});
	registryMonitorThread.setDaemon(true);
	registryMonitorThread.setName("xxl-job, admin JobRegistryMonitorHelper-registryMonitorThread");
	registryMonitorThread.start();
}

执行器自动注册

作者预留了一个 JobRegistryHelper.freshGroupRegistryInfo() 方法 , 标注在考虑中， 但还未实现。

private void freshGroupRegistryInfo(RegistryParam registryParam){
	// Under consideration, prevent affecting core tables
}

我们目的想要将列表中没有的执行器自动注册到列表中， 那么我们只需要将预留方法完善即可， 查询执行器是否存在， 不存在时将其注册到 xxl_job_group 中

private void freshGroupRegistryInfo(RegistryParam registryParam) {
	// Under consideration, prevent affecting core tables

	// auto register group
	if (!StringUtils.hasText(registryParam.getRegistryGroup())) {
		return;
	}
	int exist = XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().findByAppName(registryParam.getRegistryKey());
	if (exist > 0) {
		return;
	}
	XxlJobGroup xxlJobGroup = new XxlJobGroup();
	xxlJobGroup.setAppname(registryParam.getRegistryKey());
	xxlJobGroup.setTitle(registryParam.getRegistryKey().substring(0, Math.min(registryParam.getRegistryKey().length(), 12)));
	xxlJobGroup.setAddressType(0);
	xxlJobGroup.setAddressList(registryParam.getRegistryValue());
	xxlJobGroup.setUpdateTime(new Date());
	XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().save(xxlJobGroup);
}

这里由于表结构限制， title 字段只允许十二个字符， 因此需要对超出长度的数据进行截断， 如果是从任务调度中心创建的执行器， 在输入时就会被截断。

avatar
文章
阅读
粉丝