上篇文章 juejin.cn/post/715283… 已了解Quartz的简单使用。我们在项目中很多场景下都需要定时器,有时需要处理大量的计划任务,这时单个节点机器支撑不住生产环境要求,需要部署多个Quartz节点搭建集群。
搭建一个Quartz集群很简单,集群环境首先需要考虑的是如何做持久化?
持久化
Quartz保存数据的默认是采用使用内存的方式,其实在上篇博客中简单实现Quartz的时候,在控制台可以看到JobStore是RAMJobStore使用内存的模式,然后是not clustered表示不是集群中的节点。
Quartz框架有两种任务存储方式。
① RAMJobStore存储方式:RAMJobStore将quartz定时的任务信息存储在服务器的内存中,这种方式的优点是可以提供最好的性能,因为任务信息都存储在内存中访问获取数据速度快。缺点是当服务器突然崩溃了再次重启后你的定时任务也就没有了。
② 数据库存储方式:这种方式是将任务信息存储在数据库中,当服务器突然崩溃了再次重启后任务还是会接着上次继续执行的,不过需要配置一个quartz.propertity的配置文件。
集群搭建
下面以Mysql数据库为例,搭建Quartz集群。
1、创建数据库
解压quartz.jar包,sql脚本位置在org/quartz/impl/jdbcjobstore下,我这选择mysql数据库且使用innodb引擎,对应的脚本文件是tables_mysql_innodb.sql,共计11张表,这些表不需要我们做任何操作,是Quartz框架使用的。
集群节点相互之间不通信,而是通过定时任务持久化加锁的方式来实现集群。
1.qrtz_blob_triggers : 以Blob 类型存储的触发器。
2.qrtz_calendars:存放日历信息, quartz可配置一个日历来指定一个时间范围。
3.qrtz_cron_triggers:存放cron类型的触发器。
4.qrtz_fired_triggers:存放已触发的触发器。
5.qrtz_job_details:存放一个jobDetail信息。
6.qrtz_locks: 存储程序的悲观锁的信息(假如使用了悲观锁)。
7.qrtz_paused_trigger_graps:存放暂停掉的触发器。
8.qrtz_scheduler_state:调度器状态。
9.qrtz_simple_triggers:简单触发器的信息。
10.qrtz_trigger_listeners:触发器监听器。
11.qrtz_triggers:触发器的基本信息。
2、代码实现
下面实现一个简单的任务调度系统。
① 引入maven依赖
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.38</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid-spring-boot-starter</artifactId>
<version>1.1.10</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
② application配置
server:
port: 9001
servlet:
context-path: /quartz
spring:
datasource:
driverClassName: com.mysql.jdbc.Driver
url: jdbc:mysql://localhost:3306/quartz?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=UTC
username: root
password: root
quartz:
job-store-type: jdbc #数据库方式
jdbc:
initialize-schema: never #不初始化表结构
properties:
org:
quartz:
scheduler:
instanceId: AUTO #默认主机名和时间戳生成实例ID,可以是任何字符串,但对于所有调度程序来说,必须是唯一的 对应qrtz_scheduler_state INSTANCE_NAME字段
#instanceName: clusteredScheduler #quartzScheduler
jobStore:
class: org.springframework.scheduling.quartz.LocalDataSourceJobStore # springboot>2.5.6后使用这个
driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate #仅为数据库制作了特定于数据库的代理
useProperties: false #以指示JDBCJobStore将JobDataMaps中的所有值都作为字符串,因此可以作为名称 - 值对存储而不是在BLOB列中以其序列化形式存储更多复杂的对象。从长远来看,这是更安全的,因为您避免了将非String类序列化为BLOB的类版本问题。
tablePrefix: qrtz_ #数据库表前缀
misfireThreshold: 60000 #在被认为“失火”之前,调度程序将“容忍”一个Triggers将其下一个启动时间通过的毫秒数。默认值(如果您在配置中未输入此属性)为60000(60秒)。
clusterCheckinInterval: 5000 #设置此实例“检入”*与群集的其他实例的频率(以毫秒为单位)。影响检测失败实例的速度。
isClustered: true #打开群集功能
threadPool: #连接池
class: org.quartz.simpl.SimpleThreadPool
threadCount: 10
threadPriority: 5
threadsInheritContextClassLoaderOfInitializingThread: true
③ 定义任务实体类
@Data
public class QuartzBean {
private String id;
private String jobName;
private String jobGroup;
private String jobDescription;
private String jobClass;
private String jobStatus;
private Date startTime;
private Integer interval;
private Date endTime;
private String cronExpression;
private JobDataMap jobDataMap;
}
④ 具体Job任务类
public class MyJob extends QuartzJobBean {
@Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
System.out.println("springboot-quartz-cluster-a" + ">>" + sdf.format(new Date()) + ":" + context.getJobDetail().getKey() + "执行中..." + context.getJobDetail().getDescription());
}
}
⑤ 任务操作service类
import org.quartz.*;
import org.quartz.impl.matchers.GroupMatcher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import javax.annotation.Resource;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
@Service
public class QuartzJobService {
@Resource
private Scheduler scheduler;
/**
* 调度任务列表
* @return
*/
public List<QuartzBean> getScheduleJobList(){
List<QuartzBean> list = new ArrayList<>();
try {
for(String groupJob: scheduler.getJobGroupNames()){
for(JobKey jobKey: scheduler.getJobKeys(GroupMatcher.<JobKey>groupEquals(groupJob))){
List<? extends Trigger> triggers = scheduler.getTriggersOfJob(jobKey);
for (Trigger trigger: triggers) {
QuartzBean quartzBean = new QuartzBean();
Trigger.TriggerState triggerState = scheduler.getTriggerState(trigger.getKey());
JobDetail jobDetail = scheduler.getJobDetail(jobKey);
String cronExpression = "";
if (trigger instanceof CronTrigger) {
CronTrigger cronTrigger = (CronTrigger) trigger;
cronExpression = cronTrigger.getCronExpression();
quartzBean.setCronExpression(cronExpression);
}
quartzBean.setStartTime(trigger.getStartTime());
quartzBean.setEndTime(trigger.getEndTime());
quartzBean.setJobName(jobKey.getName());
quartzBean.setJobGroup(jobKey.getGroup());
quartzBean.setJobDescription(jobDetail.getDescription());
quartzBean.setJobStatus(triggerState.name());
quartzBean.setJobClass(jobDetail.getJobClass().toGenericString());
quartzBean.setJobDataMap(jobDetail.getJobDataMap());
list.add(quartzBean);
}
}
}
} catch (SchedulerException e) {
e.printStackTrace();
}
return list;
}
/**
* 创建简单调度任务
* @param quartzBean
* @throws Exception
*/
public void createScheduleSimpleJob(QuartzBean quartzBean) throws Exception{
Class<? extends Job> jobClass = (Class<? extends Job>) Class.forName(quartzBean.getJobClass());
//job
JobDetail jobDetail = JobBuilder.newJob(jobClass)
.withIdentity(quartzBean.getJobName(), quartzBean.getJobGroup())
.setJobData(quartzBean.getJobDataMap())
.withDescription(quartzBean.getJobDescription())
.build();
//trigger
Trigger trigger = null;
//单次还是循环
if (quartzBean.getInterval() == null) {
trigger = TriggerBuilder.newTrigger()
.withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
.withSchedule(SimpleScheduleBuilder.simpleSchedule())
.startAt(quartzBean.getStartTime())
.build();
} else {
trigger = TriggerBuilder.newTrigger()
.withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
.withSchedule(SimpleScheduleBuilder.repeatSecondlyForever(quartzBean.getInterval()))
.startAt(quartzBean.getStartTime())
.endAt(quartzBean.getEndTime())
.build();
}
scheduler.scheduleJob(jobDetail, trigger);
}
/**
* 创建cron调度任务
* @param quartzBean
* @throws Exception
*/
public void createScheduleCronJob(QuartzBean quartzBean) throws Exception{
Class<? extends Job> jobClass = (Class<? extends Job>) Class.forName(quartzBean.getJobClass());
// job
JobDetail jobDetail = JobBuilder.newJob(jobClass)
.withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
.setJobData(quartzBean.getJobDataMap())
.withDescription(quartzBean.getJobDescription())
.build();
// trigger
CronTrigger trigger = TriggerBuilder.newTrigger()
.withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
.withSchedule(CronScheduleBuilder.cronSchedule(quartzBean.getCronExpression()))
.build();
scheduler.scheduleJob(jobDetail, trigger);
}
/**
* 暂停任务
* @param jobName
* @param jobGroup
* @throws Exception
*/
public void pauseScheduleJob(String jobName,String jobGroup) throws Exception{
JobKey jobKey = JobKey.jobKey(jobName,jobGroup);
scheduler.pauseJob(jobKey);
}
/**
* 立即执行任务
* @param jobName
* @param jobGroup
* @throws Exception
*/
public void runJob(String jobName,String jobGroup) throws Exception{
JobKey jobKey = JobKey.jobKey(jobName,jobGroup);
scheduler.triggerJob(jobKey);
}
/**
* 更新简单任务
* @param quartzBean
* @throws Exception
*/
public void updateScheduleSimpleJob(QuartzBean quartzBean) throws Exception {
//原任务触发器
TriggerKey triggerKey = TriggerKey.triggerKey(quartzBean.getJobName(), quartzBean.getJobGroup());
//trigger
Trigger trigger = null;
//单次还是循环
if (quartzBean.getInterval() == null) {
trigger = TriggerBuilder.newTrigger()
.withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
.withSchedule(SimpleScheduleBuilder.simpleSchedule())
.startAt(quartzBean.getStartTime())
.build();
} else {
trigger = TriggerBuilder.newTrigger()
.withIdentity(quartzBean.getJobName(),quartzBean.getJobGroup())
.withSchedule(SimpleScheduleBuilder.repeatMinutelyForever(quartzBean.getInterval()))
.startAt(quartzBean.getStartTime())
.endAt(quartzBean.getEndTime())
.build();
}
//重置对应的job
scheduler.rescheduleJob(triggerKey, trigger);
}
/**
* 更新定时任务Cron
* @param quartzBean 定时任务信息类
* @throws SchedulerException
*/
public void updateScheduleCronJob(QuartzBean quartzBean) throws Exception {
//原任务触发器
TriggerKey triggerKey = TriggerKey.triggerKey(quartzBean.getJobName());
// trigger
CronTrigger trigger = TriggerBuilder.newTrigger()
.withIdentity(quartzBean.getJobName())
.withSchedule(CronScheduleBuilder.cronSchedule(quartzBean.getCronExpression()))
.build();
//重置对应的job
scheduler.rescheduleJob(triggerKey, trigger);
}
/**
* 删除定时任务
* @param jobName
* @param jobGroup
* @throws Exception
*/
public void deleteScheduleJob(String jobName,String jobGroup) throws Exception {
JobKey jobKey = JobKey.jobKey(jobName,jobGroup);
scheduler.deleteJob(jobKey);
}
/**
* 获取任务状态
* (" BLOCKED ", " 阻塞 ");
* ("COMPLETE", "完成");
* ("ERROR", "出错");
* ("NONE", "不存在");
* ("NORMAL", "正常");
* ("PAUSED", "暂停");
*/
public String getScheduleJobStatus(String jobName,String jobGroup) throws Exception {
TriggerKey triggerKey = TriggerKey.triggerKey(jobName,jobGroup);
Trigger.TriggerState state = scheduler.getTriggerState(triggerKey);
return state.name();
}
/**
* 检查任务是否存在
* @param jobName
* @param jobGroup
* @return
* @throws Exception
*/
public Boolean checkExistsScheduleJob(String jobName,String jobGroup) throws Exception {
JobKey jobKey = JobKey.jobKey(jobName,jobGroup);
return scheduler.checkExists(jobKey);
}
}
⑥ 接口controller类
import org.quartz.JobDataMap;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.*;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
@RestController
@RequestMapping("/api")
public class QuartzJobController {
@Autowired
private QuartzJobService quartzJobService;
@GetMapping("get")
public List<QuartzBean> getQuartzList() throws Exception{
return quartzJobService.getScheduleJobList();
}
@GetMapping("createSimpleJob")
public String createSimpleJob(String jobName,String jobGroup, String jobDescription) throws Exception {
QuartzBean quartzBean = new QuartzBean();
quartzBean.setJobClass("com.syun.quartz.example.MyJob");
quartzBean.setJobName(jobName);
quartzBean.setJobGroup(jobGroup);
quartzBean.setJobDescription(jobDescription);
JobDataMap map = new JobDataMap();
map.put("serviceId", "1001");
quartzBean.setJobDataMap(map);
// 保存10s中后开始,50s后结束,每隔3秒钟一次
Calendar newTimeStart = Calendar.getInstance();
newTimeStart.setTime(new Date());
newTimeStart.add(Calendar.SECOND,10);
quartzBean.setStartTime(newTimeStart.getTime());
Calendar newTimeEnd = Calendar.getInstance();
newTimeEnd.setTime(new Date());
newTimeEnd.add(Calendar.SECOND,50);
quartzBean.setStartTime(newTimeEnd.getTime());
quartzBean.setInterval(3);
quartzJobService.createScheduleSimpleJob(quartzBean);
return "SUCCESS";
}
@GetMapping("/createCronJob")
public String createCronJob(String jobName,String jobGroup,String jobDescription) throws Exception{
QuartzBean quartzBean = new QuartzBean();
quartzBean.setJobClass("com.syun.quartz.example.MyJob");
quartzBean.setJobName(jobName);
quartzBean.setJobGroup(jobGroup);
quartzBean.setJobDescription(jobDescription);
quartzBean.setCronExpression("*/10 * * * * ?");
JobDataMap map = new JobDataMap();
map.put("serviceId", "1001");
quartzBean.setJobDataMap(map);
quartzJobService.createScheduleCronJob(quartzBean);
return "SUCCESS";
}
@GetMapping(value = "/delete")
public String delete(String jobName,String jobGroup) throws Exception{
quartzJobService.deleteScheduleJob(jobName, jobGroup);
return "SUCCESS";
}
@GetMapping(value = "check")
public String check(String jobName, String jobGroup) throws Exception {
if(quartzJobService.checkExistsScheduleJob(jobName, jobGroup)){
return "存在定时任务:"+jobName;
}else{
return "不存在定时任务:"+jobName;
}
}
}
测试
- 创建简单任务:http://localhost:9001/quartz/api/createSimpleJob?jobName=myjob&jobGroup=mygroup&jobDescription=简单任务,每3s执行一次
- 创建corn任务:http://localhost:9001/quartz/api/createCronJob?jobName=myjob1&jobGroup=mygroup&jobDescription=cron任务,每10s执行一次
Gitee Demo 地址:gitee.com/renxiaoshi/…
搭建过程中遇到的问题
① NoClassDefFoundError: org/springframework/jdbc/datasource/embedded/EmbeddedDatabaseType
解决:使用了druid连接池,但是缺少spring-boot-starter-jdbc的依赖
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
② nested exception is org.quartz.SchedulerConfigException: DataSource name not set
解决:之前配置的是:org.quartz.impl.jdbcjobstore.JobStoreTX,修改为:org.springframework.scheduling.quartz.LocalDataSourceJobStore