随记-xxlJob安装与调试

阡陌Qianmo
303 阅读6分钟

xxl-job

XXL-JOB是一个分布式任务调度平台，其核心设计目标是开发迅速、学习简单、轻量级、易扩展。现已开放源代码并接入多家公司线上产品线，开箱即用。

引入

<!-- java项目引入 -->
<dependency>
   <groupId>com.xuxueli</groupId>
   <artifactId>xxl-job-core</artifactId>
   <version>2.3.1</version>
</dependency>

源码下载地址: github.com/xuxueli/xxl…

客户端配置文件与配置类详解

配置文件

### 调度中心部署根地址 [选填]：如调度中心集群部署存在多个地址则用逗号分隔。执行器将会使用该地址进行"执行器心跳注册"和"任务结果回调"；为空则关闭自动注册；
xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin
### 执行器通讯TOKEN [选填]：非空时启用；
xxl.job.accessToken=
### 执行器AppName [选填]：执行器心跳注册分组依据；为空则关闭自动注册
xxl.job.executor.appname=xxl-job-executor-sample
### 执行器注册 [选填]：优先使用该配置作为注册地址，为空时使用内嵌服务 ”IP:PORT“ 作为注册地址。从而更灵活的支持容器类型执行器动态IP和动态映射端口问题。
xxl.job.executor.address=
### 执行器IP [选填]：默认为空表示自动获取IP，多网卡时可手动设置指定IP，该IP不会绑定Host仅作为通讯实用；地址信息用于 "执行器注册" 和 "调度中心请求并触发任务"；
xxl.job.executor.ip=
### 执行器端口号 [选填]：小于等于0则自动获取；默认端口为9999，单机部署多个执行器时，注意要配置不同执行器端口；`
xxl.job.executor.port=9999
### 执行器运行日志文件存储磁盘路径 [选填] ：需要对该路径拥有读写权限；为空则使用默认路径；
xxl.job.executor.logpath=/data/applogs/xxl-job/jobhandler
### 执行器日志文件保存天数 [选填] ： 过期日志自动清理, 限制值大于等于3时生效; 否则, 如-1, 关闭自动清理功能；
xxl.job.executor.logretentiondays=30

配置类


import com.frogshealth.legou.common.config.CommonConfig;
import com.xxl.job.core.executor.impl.XxlJobSpringExecutor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * xxl-job配置类
 */
@Configuration
public class XxlJobConfig {
    private Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);

    @Value("${xxl.job.admin.addresses}")
    private String adminAddresses;

    @Value("${xxl.job.accessToken}")
    private String accessToken;

    @Value("${xxl.job.executor.appname}")
    private String appname;

    @Value("${xxl.job.executor.logpath}")
    private String logPath;

    @Value("${xxl.job.executor.logretentiondays}")
    private int logRetentionDays;
    @Value("${xxl.job.executor.port}")
    private int port;

    @Value("${xxl.job.executor.address}")
    private String address;

    @Bean
    public XxlJobSpringExecutor xxlJobExecutor() {
        XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
        xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
        xxlJobSpringExecutor.setAppname(appname);
        xxlJobSpringExecutor.setAddress(address);
        xxlJobSpringExecutor.setAccessToken(accessToken);
        xxlJobSpringExecutor.setLogPath(logPath);
        xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);
        xxlJobSpringExecutor.setPort(port);
        return xxlJobSpringExecutor;
    }



}

服务端配置文件详解

### web
server.port=8080
server.servlet.context-path=/xxl-job-admin

### actuator
management.server.servlet.context-path=/actuator
management.health.mail.enabled=false

### resources
spring.mvc.servlet.load-on-startup=0
spring.mvc.static-path-pattern=/static/**
spring.resources.static-locations=classpath:/static/

### freemarker
spring.freemarker.templateLoaderPath=classpath:/templates/
spring.freemarker.suffix=.ftl
spring.freemarker.charset=UTF-8
spring.freemarker.request-context-attribute=request
spring.freemarker.settings.number_format=0.##########

### mybatis
mybatis.mapper-locations=classpath:/mybatis-mapper/*Mapper.xml
#mybatis.type-aliases-package=com.xxl.job.admin.core.model

### xxl-job, datasource
spring.datasource.url=jdbc:mysql://docker-mysql-mongo-redis_mysql_1:3306/xxl_job?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=Asia/Shanghai
spring.datasource.username=root
spring.datasource.password=root
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

### datasource-pool
spring.datasource.type=com.zaxxer.hikari.HikariDataSource
spring.datasource.hikari.minimum-idle=10
spring.datasource.hikari.maximum-pool-size=30
spring.datasource.hikari.auto-commit=true
spring.datasource.hikari.idle-timeout=30000
spring.datasource.hikari.pool-name=HikariCP
spring.datasource.hikari.max-lifetime=900000
spring.datasource.hikari.connection-timeout=10000
spring.datasource.hikari.connection-test-query=SELECT 1
spring.datasource.hikari.validation-timeout=1000

### xxl-job, email
spring.mail.host=smtp.qq.com
spring.mail.port=25
spring.mail.username=xxxx
spring.mail.from=xxxxx@qq.com
spring.mail.password=xxx
spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true
spring.mail.properties.mail.smtp.socketFactory.class=javax.net.ssl.SSLSocketFactory

### 调度中心通讯TOKEN [选填]：非空时启用；
xxl.job.accessToken=default_token

### 调度中心国际化配置 [必填]： 默认为 "zh_CN"/中文简体, 可选范围为 "zh_CN"/中文简体, "zh_TC"/中文繁体 and "en"/英文；
xxl.job.i18n=zh_CN

## 调度线程池最大线程配置【必填】
xxl.job.triggerpool.fast.max=200
xxl.job.triggerpool.slow.max=100

### 调度中心日志表数据保存天数 [必填]：过期日志自动清理；限制大于等于7时生效，否则, 如-1，关闭自动清理功能；
xxl.job.logretentiondays=30

服务端改造

由于原本的xxl-job不支持动态注入任务，需要自定义一个controller类完成该需求。

在模块xxl-job-admin中找到com.xxl.job.admin.controller包中,添加DynamicApiController类


package com.xxl.job.admin.controller;

import com.xxl.job.admin.controller.annotation.PermissionLimit;
import com.xxl.job.admin.controller.request.XxlJobQuery;
import com.xxl.job.admin.core.model.XxlJobGroup;
import com.xxl.job.admin.core.model.XxlJobInfo;
import com.xxl.job.admin.core.util.I18nUtil;
import com.xxl.job.admin.dao.XxlJobGroupDao;
import com.xxl.job.admin.service.LoginService;
import com.xxl.job.admin.service.XxlJobService;
import com.xxl.job.core.biz.model.ReturnT;

import java.util.Date;
import java.util.List;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping({"/ext/jobinfo"})
public class DynamicApiController {
    private static Logger logger = LoggerFactory.getLogger(DynamicApiController.class);
    @Autowired
    private XxlJobService xxlJobService;
    @Autowired
    private XxlJobGroupDao xxlJobGroupDao;

    public DynamicApiController() {
    }

    @RequestMapping(
            value = {"/pageList"},
            method = {RequestMethod.POST}
    )
    @PermissionLimit(
            limit = false
    )
    public Map<String, Object> pageList(@RequestBody XxlJobQuery xxlJobQuery) {
        return this.xxlJobService.pageList(xxlJobQuery.getStart(), xxlJobQuery.getLength(), xxlJobQuery.getJobGroup(), xxlJobQuery.getTriggerStatus(), xxlJobQuery.getJobDesc(), xxlJobQuery.getExecutorHandler(), xxlJobQuery.getAuthor());
    }

    @PostMapping({"/add"})
    @PermissionLimit(
            limit = false
    )
    public ReturnT<String> add(@RequestBody(required = true) XxlJobInfo jobInfo) {
        logger.info("add req is {}", jobInfo);
        ReturnT<String> add = this.xxlJobService.add(jobInfo);
        logger.info("add resp is {}", add);
        return add;
    }

    @RequestMapping(
            value = {"/delete"},
            method = {RequestMethod.POST}
    )
    @PermissionLimit(
            limit = false
    )
    public ReturnT<String> delete(@RequestBody(required = true) XxlJobInfo jobInfo) {
        logger.info("remove req is {}", jobInfo);
        ReturnT<String> remove = this.xxlJobService.remove(jobInfo.getId());
        logger.info("delete resp is {}", remove);
        return remove;
    }

    @RequestMapping(
            value = {"/start"},
            method = {RequestMethod.POST}
    )
    @PermissionLimit(
            limit = false
    )
    public ReturnT<String> start(@RequestBody(required = true) XxlJobInfo jobInfo) {
        logger.info("start req is {}", jobInfo);
        ReturnT<String> start = this.xxlJobService.start(jobInfo.getId());
        logger.info("start resp is {}", start);
        return start;
    }

    @RequestMapping(
            value = {"/stop"},
            method = {RequestMethod.POST}
    )
    @PermissionLimit(
            limit = false
    )
    public ReturnT<String> stop(@RequestBody(required = true) XxlJobInfo jobInfo) {
        logger.info("stop req is {}", jobInfo);
        ReturnT<String> stop = this.xxlJobService.stop(jobInfo.getId());
        logger.info("stop resp is {}", stop);
        return stop;
    }

    @RequestMapping({"/update"})
    @ResponseBody
    public ReturnT<String> update(@RequestBody(required = true) XxlJobInfo jobInfo) {
        logger.info("update req is {}", jobInfo);
        ReturnT<String> update = this.xxlJobService.update(jobInfo);
        logger.info("update resp is {}", update);
        return update;
    }

    @PostMapping({"/getGroupId"})
    @PermissionLimit(
            limit = false
    )
    public ReturnT<List<Integer>> getGroupIds(@RequestBody XxlJobGroup xxlJobGroup) {
        logger.info("getGroupIds req is {}", xxlJobGroup);
        List<Integer> idByAppName = this.xxlJobGroupDao.findIdByAppName(xxlJobGroup.getAppname());
        ReturnT<List<Integer>> resp = new ReturnT(idByAppName);
        logger.info("getGroupIds resp is {}", resp);
        return resp;
    }

    @PostMapping("/saveGroup")
    @PermissionLimit(
            limit = false
    )
    @ResponseBody
    public ReturnT<String> save(@RequestBody XxlJobGroup xxlJobGroup){
        logger.info("saveGroupIds req is {}", xxlJobGroup);
        // valid
        if (xxlJobGroup.getAppname()==null || xxlJobGroup.getAppname().trim().length()==0) {
            return new ReturnT<String>(500, (I18nUtil.getString("system_please_input")+"AppName") );
        }
        if (xxlJobGroup.getAppname().length()<4 || xxlJobGroup.getAppname().length()>64) {
            return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_appname_length") );
        }
        if (xxlJobGroup.getAppname().contains(">") || xxlJobGroup.getAppname().contains("<")) {
            return new ReturnT<String>(500, "AppName"+I18nUtil.getString("system_unvalid") );
        }
        if (xxlJobGroup.getTitle()==null || xxlJobGroup.getTitle().trim().length()==0) {
            return new ReturnT<String>(500, (I18nUtil.getString("system_please_input") + I18nUtil.getString("jobgroup_field_title")) );
        }
        if (xxlJobGroup.getTitle().contains(">") || xxlJobGroup.getTitle().contains("<")) {
            return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_title")+I18nUtil.getString("system_unvalid") );
        }
        if (xxlJobGroup.getAddressType()!=0) {
            if (xxlJobGroup.getAddressList()==null || xxlJobGroup.getAddressList().trim().length()==0) {
                return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_addressType_limit") );
            }
            if (xxlJobGroup.getAddressList().contains(">") || xxlJobGroup.getAddressList().contains("<")) {
                return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_registryList")+I18nUtil.getString("system_unvalid") );
            }

            String[] addresss = xxlJobGroup.getAddressList().split(",");
            for (String item: addresss) {
                if (item==null || item.trim().length()==0) {
                    return new ReturnT<String>(500, I18nUtil.getString("jobgroup_field_registryList_unvalid") );
                }
            }
        }

        // process
        xxlJobGroup.setUpdateTime(new Date());

        int ret = xxlJobGroupDao.save(xxlJobGroup);
        return (ret>0)?ReturnT.SUCCESS:ReturnT.FAIL;
    }
}

我们可以看到,这里的大多数方法都是调用了XxlJobService这个服务类,只需要了解这个服务类就可以根据完成原本由web界面管理的对应功能了。

需注意:@PermissionLimit注解,它表示了是否开启token验证的模式。具体的实现逻辑可以去 com.xxl.job.admin.controller.interceptor.PermissionInterceptor中可以看到。

scheduleThread与ringThread详解

小细节：两个主线程的启动都会先阻塞一段时间，以保证初始数据全部加载完成。

scheduleThread

该类主要是负责任务的调度、触发和执行。 ScheduleThread启动后，会初始化自己的调度服务，并且会不断扫描任务表，判断任务是否满足触发条件，如果满足，则会将任务放入调度队列中等待执行。

分析源码

本个线程主要是负责从数据库中拉取带执行任务进行数据载入，并写回下一次读写时间。

分三种情况

  • 当前任务已经过时且超过预读时间秒 -> 直接执行过期调度策略
  • 当前任务已经过时但未超过预读时间秒 -> 直接执行
  • 当前任务未到时间 -> 放入时间轮中进行等待拉取

ringThread

ringThread 的作用是创建一个环形任务队列，将任务按照一定的规则添加到队列中，并按照预定的时间间隔依次触发执行队列中的任务。在任务执行完毕后，ringThread 会将其从队列中移除，然后继续等待下一个时间间隔触发下一个任务的执行。 ringThread 的实现依赖于 JDK 提供的 ScheduledThreadPoolExecutor，它会根据任务的执行时间点和调度策略，自动调度任务的执行。同时，ringThread 还提供了一些控制方法，例如可以手动添加任务到队列中，也可以手动移除队列中的任务，以及修改任务的执行时间等。

JobTriggerPoolHelper.trigger分析

本类中存在两个线程池,一个是fastTriggerPool,负责当前任务处理。slowTriggerPool负责错误重试。

正常用户在调用任务时会直接调用fast中执行，而slow是负责兜底策略。

存在问题: 当任务过多时,会被直接丢弃。因为他的底层拒绝策略就是默认的直接丢弃。

public void start(){
    fastTriggerPool = new ThreadPoolExecutor(
            10,
            XxlJobAdminConfig.getAdminConfig().getTriggerPoolFastMax(),
            60L,
            TimeUnit.SECONDS,
            new LinkedBlockingQueue<Runnable>(1000),
            new ThreadFactory() {
                @Override
                public Thread newThread(Runnable r) {
                    return new Thread(r, "xxl-job, admin JobTriggerPoolHelper-fastTriggerPool-" + r.hashCode());
                }
            });

    slowTriggerPool = new ThreadPoolExecutor(
            10,
            XxlJobAdminConfig.getAdminConfig().getTriggerPoolSlowMax(),
            60L,
            TimeUnit.SECONDS,
            new LinkedBlockingQueue<Runnable>(2000),
            new ThreadFactory() {
                @Override
                public Thread newThread(Runnable r) {
                    return new Thread(r, "xxl-job, admin JobTriggerPoolHelper-slowTriggerPool-" + r.hashCode());
                }
            });
}
avatar
文章
阅读
粉丝