SpringBoot集成Quartz集群

2,163 阅读6分钟

上篇文章 juejin.cn/post/715283… 已了解Quartz的简单使用。我们在项目中很多场景下都需要定时器,有时需要处理大量的计划任务,这时单个节点机器支撑不住生产环境要求,需要部署多个Quartz节点搭建集群。

搭建一个Quartz集群很简单,集群环境首先需要考虑的是如何做持久化?

持久化

Quartz保存数据的默认是采用使用内存的方式,其实在上篇博客中简单实现Quartz的时候,在控制台可以看到JobStoreRAMJobStore使用内存的模式,然后是not clustered表示不是集群中的节点。

Quartz框架有两种任务存储方式。

RAMJobStore存储方式:RAMJobStore将quartz定时的任务信息存储在服务器的内存中,这种方式的优点是可以提供最好的性能,因为任务信息都存储在内存中访问获取数据速度快。缺点是当服务器突然崩溃了再次重启后你的定时任务也就没有了。

数据库存储方式:这种方式是将任务信息存储在数据库中,当服务器突然崩溃了再次重启后任务还是会接着上次继续执行的,不过需要配置一个quartz.propertity的配置文件。

集群搭建

下面以Mysql数据库为例,搭建Quartz集群。

1、创建数据库

解压quartz.jar包,sql脚本位置在org/quartz/impl/jdbcjobstore下,我这选择mysql数据库且使用innodb引擎,对应的脚本文件是tables_mysql_innodb.sql,共计11张表,这些表不需要我们做任何操作,是Quartz框架使用的。

集群节点相互之间不通信,而是通过定时任务持久化加锁的方式来实现集群。

1.qrtz_blob_triggers : 以Blob 类型存储的触发器。
2.qrtz_calendars:存放日历信息, quartz可配置一个日历来指定一个时间范围。
3.qrtz_cron_triggers:存放cron类型的触发器。
4.qrtz_fired_triggers:存放已触发的触发器。
5.qrtz_job_details:存放一个jobDetail信息。
6.qrtz_locks: 存储程序的悲观锁的信息(假如使用了悲观锁)。
7.qrtz_paused_trigger_graps:存放暂停掉的触发器。
8.qrtz_scheduler_state:调度器状态。
9.qrtz_simple_triggers:简单触发器的信息。
10.qrtz_trigger_listeners:触发器监听器。
11.qrtz_triggers:触发器的基本信息。

2、代码实现

下面实现一个简单的任务调度系统。

① 引入maven依赖

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.38</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid-spring-boot-starter</artifactId>
            <version>1.1.10</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>

② application配置

server:
  port: 9001
  servlet:
    context-path: /quartz
spring:
  datasource:
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://localhost:3306/quartz?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=UTC
    username: root
    password: root
  quartz:
    job-store-type: jdbc #数据库方式
    jdbc:
      initialize-schema: never #不初始化表结构
    properties:
      org:
        quartz:
          scheduler:
            instanceId: AUTO #默认主机名和时间戳生成实例ID,可以是任何字符串,但对于所有调度程序来说,必须是唯一的 对应qrtz_scheduler_state INSTANCE_NAME字段
            #instanceName: clusteredScheduler #quartzScheduler
          jobStore:
            class: org.springframework.scheduling.quartz.LocalDataSourceJobStore # springboot>2.5.6后使用这个
            driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate #仅为数据库制作了特定于数据库的代理
            useProperties: false #以指示JDBCJobStore将JobDataMaps中的所有值都作为字符串,因此可以作为名称 - 值对存储而不是在BLOB列中以其序列化形式存储更多复杂的对象。从长远来看,这是更安全的,因为您避免了将非String类序列化为BLOB的类版本问题。
            tablePrefix: qrtz_  #数据库表前缀
            misfireThreshold: 60000 #在被认为“失火”之前,调度程序将“容忍”一个Triggers将其下一个启动时间通过的毫秒数。默认值(如果您在配置中未输入此属性)为60000(60秒)。
            clusterCheckinInterval: 5000 #设置此实例“检入”*与群集的其他实例的频率(以毫秒为单位)。影响检测失败实例的速度。
            isClustered: true #打开群集功能
          threadPool: #连接池
            class: org.quartz.simpl.SimpleThreadPool
            threadCount: 10
            threadPriority: 5
            threadsInheritContextClassLoaderOfInitializingThread: true

③ 定义任务实体类

@Data
public class QuartzBean {

    private String id;

    private String jobName;

    private String jobGroup;

    private String jobDescription;

    private String jobClass;

    private String jobStatus;

    private Date startTime;

    private Integer interval;

    private Date endTime;

    private String cronExpression;

    private JobDataMap jobDataMap;
}

④ 具体Job任务类

public class MyJob extends QuartzJobBean {
    @Override
    protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
        System.out.println("springboot-quartz-cluster-a" + ">>" + sdf.format(new Date()) + ":" + context.getJobDetail().getKey() + "执行中..." + context.getJobDetail().getDescription());
    }
}

⑤ 任务操作service类

import org.quartz.*;
import org.quartz.impl.matchers.GroupMatcher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import javax.annotation.Resource;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;

@Service
public class QuartzJobService {

    @Resource
    private Scheduler scheduler;

    /**
     * 调度任务列表
     * @return
     */
    public List<QuartzBean> getScheduleJobList(){
        List<QuartzBean> list = new ArrayList<>();
        try {
            for(String groupJob: scheduler.getJobGroupNames()){
                for(JobKey jobKey: scheduler.getJobKeys(GroupMatcher.<JobKey>groupEquals(groupJob))){
                    List<? extends Trigger> triggers = scheduler.getTriggersOfJob(jobKey);
                    for (Trigger trigger: triggers) {
                        QuartzBean quartzBean = new QuartzBean();
                        Trigger.TriggerState triggerState = scheduler.getTriggerState(trigger.getKey());
                        JobDetail jobDetail = scheduler.getJobDetail(jobKey);
                        String cronExpression = "";
                        if (trigger instanceof CronTrigger) {
                            CronTrigger cronTrigger = (CronTrigger) trigger;
                            cronExpression = cronTrigger.getCronExpression();
                            quartzBean.setCronExpression(cronExpression);
                        }
                        quartzBean.setStartTime(trigger.getStartTime());
                        quartzBean.setEndTime(trigger.getEndTime());
                        quartzBean.setJobName(jobKey.getName());
                        quartzBean.setJobGroup(jobKey.getGroup());
                        quartzBean.setJobDescription(jobDetail.getDescription());
                        quartzBean.setJobStatus(triggerState.name());
                        quartzBean.setJobClass(jobDetail.getJobClass().toGenericString());
                        quartzBean.setJobDataMap(jobDetail.getJobDataMap());
                        list.add(quartzBean);
                    }
                }
            }
        } catch (SchedulerException e) {
            e.printStackTrace();
        }
        return list;
    }


    /**
     * 创建简单调度任务
     * @param quartzBean
     * @throws Exception
     */
    public void createScheduleSimpleJob(QuartzBean quartzBean) throws Exception{
        Class<? extends Job> jobClass = (Class<? extends Job>) Class.forName(quartzBean.getJobClass());
        //job
        JobDetail jobDetail = JobBuilder.newJob(jobClass)
                .withIdentity(quartzBean.getJobName(), quartzBean.getJobGroup())
                .setJobData(quartzBean.getJobDataMap())
                .withDescription(quartzBean.getJobDescription())
                .build();
        //trigger
        Trigger trigger = null;
        //单次还是循环
        if (quartzBean.getInterval() == null) {
            trigger = TriggerBuilder.newTrigger()
                    .withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
                    .withSchedule(SimpleScheduleBuilder.simpleSchedule())
                    .startAt(quartzBean.getStartTime())
                    .build();
        } else {
            trigger = TriggerBuilder.newTrigger()
                    .withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
                    .withSchedule(SimpleScheduleBuilder.repeatSecondlyForever(quartzBean.getInterval()))
                    .startAt(quartzBean.getStartTime())
                    .endAt(quartzBean.getEndTime())
                    .build();
        }
        scheduler.scheduleJob(jobDetail, trigger);
    }

    /**
     * 创建cron调度任务
     * @param quartzBean
     * @throws Exception
     */
    public void createScheduleCronJob(QuartzBean quartzBean) throws Exception{
        Class<? extends Job> jobClass = (Class<? extends Job>) Class.forName(quartzBean.getJobClass());
        // job
        JobDetail jobDetail = JobBuilder.newJob(jobClass)
                .withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
                .setJobData(quartzBean.getJobDataMap())
                .withDescription(quartzBean.getJobDescription())
                .build();
        // trigger
        CronTrigger trigger = TriggerBuilder.newTrigger()
                .withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
                .withSchedule(CronScheduleBuilder.cronSchedule(quartzBean.getCronExpression()))
                .build();
        scheduler.scheduleJob(jobDetail, trigger);
    }

    /**
     * 暂停任务
     * @param jobName
     * @param jobGroup
     * @throws Exception
     */
    public void pauseScheduleJob(String jobName,String jobGroup) throws Exception{
        JobKey jobKey = JobKey.jobKey(jobName,jobGroup);
        scheduler.pauseJob(jobKey);
    }

    /**
     * 立即执行任务
     * @param jobName
     * @param jobGroup
     * @throws Exception
     */
    public void runJob(String jobName,String jobGroup) throws Exception{
        JobKey jobKey = JobKey.jobKey(jobName,jobGroup);
        scheduler.triggerJob(jobKey);
    }

    /**
     * 更新简单任务
     * @param quartzBean
     * @throws Exception
     */
    public void updateScheduleSimpleJob(QuartzBean quartzBean) throws Exception {
        //原任务触发器
        TriggerKey triggerKey = TriggerKey.triggerKey(quartzBean.getJobName(), quartzBean.getJobGroup());
        //trigger
        Trigger trigger = null;
        //单次还是循环
        if (quartzBean.getInterval() == null) {
            trigger = TriggerBuilder.newTrigger()
                    .withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
                    .withSchedule(SimpleScheduleBuilder.simpleSchedule())
                    .startAt(quartzBean.getStartTime())
                    .build();
        } else {
            trigger = TriggerBuilder.newTrigger()
                    .withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
                    .withSchedule(SimpleScheduleBuilder.repeatMinutelyForever(quartzBean.getInterval()))
                    .startAt(quartzBean.getStartTime())
                    .endAt(quartzBean.getEndTime())
                    .build();
        }
        //重置对应的job
        scheduler.rescheduleJob(triggerKey, trigger);
    }

    /**
     * 更新定时任务Cron
     * @param quartzBean 定时任务信息类
     * @throws SchedulerException
     */
    public void updateScheduleCronJob(QuartzBean quartzBean) throws Exception {
        //原任务触发器
        TriggerKey triggerKey = TriggerKey.triggerKey(quartzBean.getJobName());
        // trigger
        CronTrigger trigger = TriggerBuilder.newTrigger()
                .withIdentity(quartzBean.getJobName())
                .withSchedule(CronScheduleBuilder.cronSchedule(quartzBean.getCronExpression()))
                .build();
        //重置对应的job
        scheduler.rescheduleJob(triggerKey, trigger);
    }

    /**
     * 删除定时任务
     * @param jobName
     * @param jobGroup
     * @throws Exception
     */
    public void deleteScheduleJob(String jobName,String jobGroup) throws Exception {
        JobKey jobKey = JobKey.jobKey(jobName,jobGroup);
        scheduler.deleteJob(jobKey);
    }

    /**
     * 获取任务状态
     * (" BLOCKED ", " 阻塞 ");
     * ("COMPLETE", "完成");
     * ("ERROR", "出错");
     * ("NONE", "不存在");
     * ("NORMAL", "正常");
     * ("PAUSED", "暂停");
     */
    public String getScheduleJobStatus(String jobName,String jobGroup) throws Exception {
        TriggerKey triggerKey = TriggerKey.triggerKey(jobName,jobGroup);
        Trigger.TriggerState state = scheduler.getTriggerState(triggerKey);
        return state.name();
    }

    /**
     * 检查任务是否存在
     * @param jobName
     * @param jobGroup
     * @return
     * @throws Exception
     */
    public Boolean checkExistsScheduleJob(String jobName,String jobGroup) throws Exception {
        JobKey jobKey = JobKey.jobKey(jobName,jobGroup);
        return scheduler.checkExists(jobKey);
    }

}

⑥ 接口controller类

import org.quartz.JobDataMap;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.*;

import java.util.Calendar;
import java.util.Date;
import java.util.List;

@RestController
@RequestMapping("/api")
public class QuartzJobController {


    @Autowired
    private QuartzJobService quartzJobService;

    @GetMapping("get")
    public List<QuartzBean> getQuartzList() throws Exception{
        return quartzJobService.getScheduleJobList();
    }

    @GetMapping("createSimpleJob")
    public String createSimpleJob(String jobName,String jobGroup, String jobDescription) throws Exception {
        QuartzBean quartzBean = new QuartzBean();
        quartzBean.setJobClass("com.syun.quartz.example.MyJob");
        quartzBean.setJobName(jobName);
        quartzBean.setJobGroup(jobGroup);
        quartzBean.setJobDescription(jobDescription);
        JobDataMap map = new JobDataMap();
        map.put("serviceId", "1001");
        quartzBean.setJobDataMap(map);
        // 保存10s中后开始,50s后结束,每隔3秒钟一次
        Calendar newTimeStart = Calendar.getInstance();
        newTimeStart.setTime(new Date());
        newTimeStart.add(Calendar.SECOND,10);
        quartzBean.setStartTime(newTimeStart.getTime());

        Calendar newTimeEnd = Calendar.getInstance();
        newTimeEnd.setTime(new Date());
        newTimeEnd.add(Calendar.SECOND,50);
        quartzBean.setStartTime(newTimeEnd.getTime());

        quartzBean.setInterval(3);
        quartzJobService.createScheduleSimpleJob(quartzBean);
        return "SUCCESS";
    }

    @GetMapping("/createCronJob")
    public String createCronJob(String jobName,String jobGroup,String jobDescription) throws Exception{
        QuartzBean quartzBean = new QuartzBean();
        quartzBean.setJobClass("com.syun.quartz.example.MyJob");
        quartzBean.setJobName(jobName);
        quartzBean.setJobGroup(jobGroup);
        quartzBean.setJobDescription(jobDescription);
        quartzBean.setCronExpression("*/10 * * * * ?");
        JobDataMap map = new JobDataMap();
        map.put("serviceId", "1001");
        quartzBean.setJobDataMap(map);
        quartzJobService.createScheduleCronJob(quartzBean);
        return "SUCCESS";
    }


    @GetMapping(value = "/delete")
    public String delete(String jobName,String jobGroup) throws Exception{
        quartzJobService.deleteScheduleJob(jobName, jobGroup);
        return "SUCCESS";
    }

    @GetMapping(value = "check")
    public String check(String jobName, String jobGroup) throws Exception {
        if(quartzJobService.checkExistsScheduleJob(jobName, jobGroup)){
            return "存在定时任务:"+jobName;
        }else{
            return "不存在定时任务:"+jobName;
        }
    }
}

测试

Gitee Demo 地址:gitee.com/renxiaoshi/…


搭建过程中遇到的问题

① NoClassDefFoundError: org/springframework/jdbc/datasource/embedded/EmbeddedDatabaseType

解决:使用了druid连接池,但是缺少spring-boot-starter-jdbc的依赖

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>

② nested exception is org.quartz.SchedulerConfigException: DataSource name not set

解决:之前配置的是:org.quartz.impl.jdbcjobstore.JobStoreTX,修改为:org.springframework.scheduling.quartz.LocalDataSourceJobStore