一、背景
一般而言利用线程池,需要派发一个提交线程不断的提交任务给线程池;然而当提交给线程池的任务执行需要消耗较长的时间时,提交线程若不等待线程池任务结束就直接结束,可能会导致一些误差:对执行任务数目的统计、线程池状况无法继续检测等;并且如果直接使用ThreadPoolExecutor,如果不添加额外的代码是无法继续观测到线程池运行的状况的,这对一些跑批任务的观察很不利。
二、错误的尝试
public Integer method() {
List<Object> list = new ArrayList<>();
List<List<Object>> partition = Lists.partition(list, 50);
int submit = 0;
for (List<Object> subList : partition) {
log.info("任务提交进度:{}/{}", submit, list.size());
submit += subList.size();
ThreadPoolUtils.threadPoolExecutor.execute(() -> operateJob(subList, operateTotal));
}
// 等待线程池任务结束
while (threadPoolExecutor.getActiveCount() > 0 ||
threadPoolExecutor.getCompletedTaskCount() != threadPoolExecutor.getTaskCount() ||
!threadPoolExecutor.getQueue().isEmpty()) {
try {
log.info("等待线程池中任务结束...");
Thread.sleep(5000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
return operateTotal.intValue();
}
这种方式不行的原因,需要根据ThreadPoolExecutor提供的这几个API来看:
/**
* Returns the approximate number of threads that are actively
* executing tasks.
*
* @return the number of threads
*/
public int getActiveCount() {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
int n = 0;
for (Worker w : workers)
if (w.isLocked())
++n;
return n;
} finally {
mainLock.unlock();
}
}
/**
* Returns the approximate total number of tasks that have
* completed execution. Because the states of tasks and threads
* may change dynamically during computation, the returned value
* is only an approximation, but one that does not ever decrease
* across successive calls.
*
* @return the number of tasks
*/
public long getCompletedTaskCount() {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
long n = completedTaskCount;
for (Worker w : workers)
n += w.completedTasks;
return n;
} finally {
mainLock.unlock();
}
}
/**
* Returns the approximate total number of tasks that have ever been
* scheduled for execution. Because the states of tasks and
* threads may change dynamically during computation, the returned
* value is only an approximation.
*
* @return the number of tasks
*/
public long getTaskCount() {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
long n = completedTaskCount;
for (Worker w : workers) {
n += w.completedTasks;
if (w.isLocked())
++n;
}
return n + workQueue.size();
} finally {
mainLock.unlock();
}
}
这三个方法,分别是获取活跃线程数量、获取完成任务数、获取总任务数,它们不应该被用作在业务意义上的判断线程池是否结束,因为它们仅仅能提供一个对于线程池状态的预估值。 所以利用这种方法,去让提交线程判断线程池是否结束,会造成一个概率BUG,提交线程时而正确判断并等待线程池结束,时而错误判断提前结束导致未能正确得到相关数据。
三、正确的实现
public Integer method() {
List<Object> list = new ArrayList<>();
List<List<Object>> partition = Lists.partition(list, 50);
CountDownLatch countDownLatch = new CountDownLatch(partition.size());
int submit = 0;
for (List<Object> subList : partition) {
log.info("任务提交进度:{}/{}", submit, list.size());
submit += subList.size();
ThreadPoolUtils.threadPoolExecutor.execute(() -> operateBeisenEmp(rosterEmpList, operateTotal, countDownLatch));
}
// 等待线程池任务结束
try {
countDownLatch.await(1, TimeUnit.HOURS);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
return operateTotal.intValue();
}
利用CountDownLatch来阻塞提交线程,注意若使用不带时间的阻塞方法(await()),则需要确保CountDownLatch正确归零,否则应该对任务时间有正确的预估,并使用带有最大等待时间的方法(await(long timeout, TimeUnit unit));