并发编程(六)

这是我参与11月更文挑战的第23天，活动详情查看：2021最后一次更文挑战

线程池

比较核心的类

ThreadPoolExcutor

ScheduledExecutorService

ScheduledThreadPoolExecutor

ThreadPoolExecutor

private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
private final BlockingQueue<Runnable> workQueue;
private final ReentrantLock mainLock = new ReentrantLock();
private final HashSet<Worker> workers = new HashSet<Worker>();

ctl 是线程池状态(高3位)和线程的个数(低29位)

mainLock是内部变量互斥访问控制

works线程集合

 private final class Worker
        extends AbstractQueuedSynchronizer
        implements Runnable
    {

        private static final long serialVersionUID = 6138294804551838833L;
        final Thread thread;
        Runnable firstTask;
        volatile long completedTasks;
        Worker(Runnable firstTask) {
            setState(-1); 
            this.firstTask = firstTask;
            this.thread = getThreadFactory().newThread(this);
        }
 }

work本身就是一把锁

核心参数

public ThreadPoolExecutor(int corePoolSize,
                          int maximumPoolSize,
                          long keepAliveTime,
                          TimeUnit unit,
                          BlockingQueue<Runnable> workQueue,
                          ThreadFactory threadFactory,
                          RejectedExecutionHandler handler) {
    if (corePoolSize < 0 ||
        maximumPoolSize <= 0 ||
        maximumPoolSize < corePoolSize ||
        keepAliveTime < 0)
        throw new IllegalArgumentException();
    if (workQueue == null || threadFactory == null || handler == null)
        throw new NullPointerException();
    this.acc = System.getSecurityManager() == null ?
        null :
    AccessController.getContext();
    this.corePoolSize = corePoolSize;
    this.maximumPoolSize = maximumPoolSize;
    this.workQueue = workQueue;
    this.keepAliveTime = unit.toNanos(keepAliveTime);
    this.threadFactory = threadFactory;
    this.handler = handler;
}

处理流程

• 判断当前线程数是否大于或等于corePoolSize, 如果小于新建线程执行
• 判断队列是否已满， 如果未满，放入队列
• 判断当前线程数是否大于或等于maxPoolSize,如果小于，新建线程处理
• 根据拒绝策略，拒绝任务

优雅关闭

• 生命周期

  • RUNNING: Accept new tasks and process queued tasks
  • SHUTDOWN: Don't accept new tasks, but process queued tasks
  • STOP: Don't accept new tasks, don't process queued tasks,and interrupt in-progress tasks
  • TIDYING: All tasks have terminated, workerCount is zero,the thread transitioning to state TIDYING will run the terminated() hook method
  • TERMINATED: terminated() has completed

executor.shutdown();
boolean flag = true;
do{
    flag = !executor.awaitTermination(500, TimeUnit.MILLISECONDS);
}while(flag);

public boolean awaitTermination(long timeout, TimeUnit unit)
        throws InterruptedException {
    long nanos = unit.toNanos(timeout);
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        for (;;) {
            if (runStateAtLeast(ctl.get(), TERMINATED))
                return true;
            if (nanos <= 0)
                return false;
            nanos = termination.awaitNanos(nanos);
        }
    } finally {
        mainLock.unlock();
    }
}

钩子方法

protected void beforeExecute(Thread t, Runnable r) { }
protected void afterExecute(Runnable r, Throwable t) { }
protected void terminated() { }

shutdown和shutdownNow

• shutdown不会清空任务队列， 会等所有任务执行完成，shutdownNow清空任务队列
• shutdown只会中断空闲的线程， shutdownNow会中断所有线程

源码分析

• execute

  public void execute(Runnable command) {
      if (command == null)
          throw new NullPointerException();
      int c = ctl.get();
      //小于核心线程数 添加线程
      if (workerCountOf(c) < corePoolSize) {
          if (addWorker(command, true))
              return;
          c = ctl.get();
      }
      //添加到队列
      if (isRunning(c) && workQueue.offer(command)) {
          int recheck = ctl.get();
          if (! isRunning(recheck) && remove(command))
              reject(command);
          //工作线程是否为0
          else if (workerCountOf(recheck) == 0)
              addWorker(null, false);
      }
      //拒绝策略
      else if (!addWorker(command, false))
          reject(command);
  }
  

• addWorker

  private boolean addWorker(Runnable firstTask, boolean core) {
      retry:
      for (;;) {
          int c = ctl.get();
          int rs = runStateOf(c);
  
          // 线程池状态不会，第一个任务为null或者队列空
          if (rs >= SHUTDOWN &&
              ! (rs == SHUTDOWN &&
                 firstTask == null &&
                 ! workQueue.isEmpty()))
              return false;
  
          for (;;) {
              int wc = workerCountOf(c);
              //工作线程达到上限
              if (wc >= CAPACITY ||
                  wc >= (core ? corePoolSize : maximumPoolSize))
                  return false;
              //增加worker成功
              if (compareAndIncrementWorkerCount(c))
                  break retry;
              //运行状态为shutdown， 重试 
              c = ctl.get();  // Re-read ctl
              if (runStateOf(c) != rs)
                  continue retry;
              // else CAS failed due to workerCount change; retry inner loop
          }
      }
  
      boolean workerStarted = false;
      boolean workerAdded = false;
      Worker w = null;
      try {
          w = new Worker(firstTask);
          final Thread t = w.thread;
          if (t != null) {
              final ReentrantLock mainLock = this.mainLock;
              mainLock.lock();
              try {
                  int rs = runStateOf(ctl.get());
  				
                  if (rs < SHUTDOWN ||
                      (rs == SHUTDOWN && firstTask == null)) {
                      //线程正在运行
                      if (t.isAlive()) 
                          throw new IllegalThreadStateException();
                      //添加到worker集合
                      workers.add(w);
                      int s = workers.size();
                      if (s > largestPoolSize)
                          largestPoolSize = s;
                      workerAdded = true;
                  }
              } finally {
                  mainLock.unlock();
              }
              //启动线程
              if (workerAdded) {
                  t.start();
                  workerStarted = true;
              }
          }
      } finally {
          if (! workerStarted)
              addWorkerFailed(w);
      }
      return workerStarted;
  }
  

• Work #run

  Worker(Runnable firstTask) {
      setState(-1);
      this.firstTask = firstTask;
      this.thread = getThreadFactory().newThread(this);
  }
  

  public void run() {
      runWorker(this);
  }
  

  final void runWorker(Worker w) {
      Thread wt = Thread.currentThread();
      Runnable task = w.firstTask;
      w.firstTask = null;
      //中断线程
      w.unlock(); 
      boolean completedAbruptly = true;
      try {
          while (task != null || (task = getTask()) != null) {
              w.lock();
              if ((runStateAtLeast(ctl.get(), STOP) ||
                   (Thread.interrupted() &&
                    runStateAtLeast(ctl.get(), STOP))) &&
                  !wt.isInterrupted())
                  wt.interrupt();
              try {
                  beforeExecute(wt, task);
                  Throwable thrown = null;
                  try {
                      task.run();
                  } catch (RuntimeException x) {
                      thrown = x; throw x;
                  } catch (Error x) {
                      thrown = x; throw x;
                  } catch (Throwable x) {
                      thrown = x; throw new Error(x);
                  } finally {
                      afterExecute(task, thrown);
                  }
              } finally {
                  task = null;
                  w.completedTasks++;
                  w.unlock();
              }
          }
          completedAbruptly = false;
      } finally {
          processWorkerExit(w, completedAbruptly);
      }
  }
  

  private Runnable getTask() {
      boolean timedOut = false; // Did the last poll() time out?
  
      for (;;) {
          int c = ctl.get();
          int rs = runStateOf(c);
          if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
              decrementWorkerCount();
              return null;
          }
  
          int wc = workerCountOf(c);
          boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;
  
          if ((wc > maximumPoolSize || (timed && timedOut))
              && (wc > 1 || workQueue.isEmpty())) {
              if (compareAndDecrementWorkerCount(c))
                  return null;
              continue;
          }
  
          try {
              Runnable r = timed ?
                  workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
              workQueue.take();
              if (r != null)
                  return r;
              timedOut = true;
          } catch (InterruptedException retry) {
              timedOut = false;
          }
      }
  }
  

  private void processWorkerExit(Worker w, boolean completedAbruptly) {
          if (completedAbruptly) // If abrupt, then workerCount wasn't adjusted
              decrementWorkerCount();
  
          final ReentrantLock mainLock = this.mainLock;
          mainLock.lock();
          try {
              completedTaskCount += w.completedTasks;
              workers.remove(w);
          } finally {
              mainLock.unlock();
          }
          tryTerminate();
          int c = ctl.get();
          if (runStateLessThan(c, STOP)) {
              if (!completedAbruptly) {
                  int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
                  if (min == 0 && ! workQueue.isEmpty())
                      min = 1;
                  if (workerCountOf(c) >= min)
                      return; 
              }
              addWorker(null, false);
          }
      }
  

拒绝策略

• AbortPolicy : 抛出异常
• CallerRunsPolicy 任务在自己线程执行
• DiscardPolicy 直接丢弃
• DiscardOldestPolicy 丢弃最早的任务

工具类

• singleThreadExecutor
• fixedThreadPool
• cachedThreadPool
• scheduledThreadPool
• singleThreadScheduledExecutor

ScheduledThreadPoolExecutor

执行延迟任务和周期执行任务

AtFixedRate按固定频率执行，与任务执行时间无关， 任务执行时间必须小于间隔时间

WithFixedDelay按固定间隔执行， 与任务执行时间有关

是通过DelayedWOrkQueue来实现的，底层原理和DelayQueue一样，针对任务的取消进行了优化。

CompletableFuture

常用方法

• runAsync 与supplyAsync

  runAsync没有返回值， supplyAsync有返回值。

• thenRun, thenAccept, thenApply

  不用通过get获取，直接能拿到结果

• thenCompose, thenCombine

  thenCompose取消嵌套， 传入的参数是Function类型， 返回值是CompletionStage子类

  thenCombine传入supplier函数和BIFunction，将supplier的返回值和前面的值结合起来做些操作

• 组合 allOf anyOf

四种任务原型

• Runnable: runAsync thenRun
• Consumer: thenAccept
• Supplier: supplierAsync
• Function: thenApply

CompletionStage原理

CompletableFuture不仅实现了Future接口，还实现了CompletableStage接口， 所有方法返回值都是自己。默认情况下依赖ForkJoinPool

 private static final Executor asyncPool = useCommonPool ?
        ForkJoinPool.commonPool() : new ThreadPerTaskExecutor();

• 任务类型的适配

  supplyAsync方法内部会把Supplier方法转换成AsyncSupply,提交给ForkJoinPool执行。

  runAsync会把run方法转换成AsyncRun

  thenRun/thenApply/thenAccept会转换成UniRun/uniApply/uniAccept

• 任务链式执行

  每个CompletableFuture内部都有一个栈，存储的是后续依赖它的任务，这个栈是Treiber Stack。

thenApply和thenApplyAsync的区别

前置任务没有执行，都会将当前任务的下个任务入栈

当前任务执行完毕， thenApply立刻会在当前线程执行， 不会入栈。thenApplyAsync会在forkJoinPool执行，入栈出栈

allOf 和anyOf

allOf中每两个生成BiRelay，BiRelay只是中转任务，本身没有任务代码，只是参照输入的两个Future是否完成，如果完成就从栈中弹出依赖他的BiRelay继续执行

ForkJoinPool

分治算法的多线程并行计算框架,需要任务继承RecursiveAction或RecursiveTask。RecursiveAction没有返回值，RecursiveTask有返回值。

核心数据结构

public class ForkJoinPool extends AbstractExecutorService {

    //状态变量
    volatile long ctl;  
	//工作队列--工作窃取队列
    volatile WorkQueue[] workQueues;    
    //线程工厂
    final ForkJoinWorkerThreadFactory factory;
	//下一个Work的下标
    int indexSeed;
    static final class WorkQueue {
             
        
        //线程的局部队列
        ForkJoinTask<?>[] array;   
        final ForkJoinPool pool;  
        //所有者线程， null表示共享
        final ForkJoinWorkerThread owner; 
    }
}

public class ForkJoinWorkerThread extends Thread {
     final ForkJoinPool pool;                
    final ForkJoinPool.WorkQueue workQueue; 
}

private ForkJoinPool(int parallelism,
                         ForkJoinWorkerThreadFactory factory,
                         UncaughtExceptionHandler handler,
                         int mode,
                         String workerNamePrefix) {
    this.workerNamePrefix = workerNamePrefix;
    this.factory = factory;
    this.ueh = handler;
    this.config = (parallelism & SMASK) | mode;
    long np = (long)(-parallelism); // offset ctl counts
    this.ctl = ((np << AC_SHIFT) & AC_MASK) | ((np << TC_SHIFT) & TC_MASK);
}

工作窃取

这个队列有两个操作:

• Worker线程在队列头部，对top指针执行加，减，单线程
• 其他worker线程在队列尾部，对base进行累加，执行出队，多线程，CAS操作

特点:

• 队列是环形的，base一直累加，不会减少，top会累加，减少
• 队列满了会扩容，扩容会变成2倍的大小

状态控制

ctl被分为五部分

• 最高的16个比特位标识Active线程数-parallelism
• 次高的16个比特位标识Total线程数
• 1个比特位，如果是1，表示整个ForkJoinPool正在关闭
• 15个比特位，标识阻塞栈的栈定线程的wait count
• 16个比特位，标识阻塞栈的栈顶线程对应的ID

阻塞栈

把所有空闲的work线程放在一个栈里， 也就是TreiberStack

状态

• 空闲状态，在阻塞栈里
• 活跃状态, 在执行ForkJoinTask,未阻塞
• 阻塞状态 在执行ForkJoinTask,但阻塞了