任务编排: asyncTool

往之不谏_来者可追
487 阅读5分钟

任务编排: asyncTool

安装

  1. 从 gitee 上拉取最新的代码

    git clone https://gitee.com/jd-platform-opensource/asyncTool.git

  2. 进入 asyncTool 目录, 进行打包

    // Linux系统 cd asyncTool          Windows系统  选择进入即可
// 打包命令, 跳过测试
mvn clean install -DskipTests

  3. 项目中引入

    具体的gav可以查看asyncTool项目的pom.xml文件, 可能和现在的不同。

    有序代码中有使用到日志框架，所以看下面代码的话，可以先引入下面的坐标

    <dependency>
    <groupId>com.jd.platform</groupId>
    <artifactId>asyncTool</artifactId>
    <version>1.4.1-SNAPSHOT</version>
</dependency>
<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.2.3</version>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.7.25</version>
</dependency>

  4. 引入完成后, 可以编写代码了, 下面为一个简单的测试

    可以改变 Async.beginWork 中的超时时间来看不同的效果

    package org.gjy.m8.tool;

import com.jd.platform.async.callback.ICallback;
import com.jd.platform.async.callback.IWorker;
import com.jd.platform.async.executor.Async;
import com.jd.platform.async.worker.WorkResult;
import com.jd.platform.async.wrapper.WorkerWrapper;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Map;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;

public class AsyncToolTest {
    private static final Logger log = LoggerFactory.getLogger(AsyncToolTest.class);

    @Test
    public void test1() throws ExecutionException, InterruptedException {
        DeWork w = new DeWork();
        WorkerWrapper<String, User> workerWrapper = new WorkerWrapper.Builder<String, User>()
                .worker(w)
                .param("0")
                .id("first")
                .callback(w)
                .build();
        Async.beginWork(1000, workerWrapper);
        log.info("workerWrapper.getWorkResult() {}", workerWrapper.getWorkResult());
        Async.shutDown();
    }

    private static final class DeWork implements IWorker<String, User>, ICallback<String, User> {

        @Override
        public void result(boolean b, String s, WorkResult<User> workResult) {
            try {
                TimeUnit.SECONDS.sleep(2);
                log.info("result {},{},{}", b, s, workResult.getResult());
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }

        @Override
        public void begin() {
            log.info("begin");
        }

        @Override
        public User action(String s, Map<String, WorkerWrapper> map) {
            try {
                TimeUnit.SECONDS.sleep(1);
                log.info("action s={},map={}", s, map);
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
            return new User("user0");
        }

        @Override
        public User defaultValue() {
            User user = new User("default");
            log.info("defaultValue user: {}", user);
            return user;
        }
    }

    @Data
    @AllArgsConstructor
    @NoArgsConstructor
    private static final class User {
        private String name;
    }

}

基本组件

IWorker

@FunctionalInterface
public interface IWorker<T, V> {
    // 工作方法
    V action(T object, Map<String, WorkerWrapper> allWrappers);
	// 超时，异常，返回的默认值
    default V defaultValue() {
        return null;
    }
}

一个最小的任务执行单元，通常是一个网络调用，或耗时操作，T和V分别表示入参和出参。多个worker之间没有关联。

ICallback

@FunctionalInterface
public interface ICallback<T, V> {
    // 任务开始时的回调
    default void begin() {
    }
    // 执行完毕后给v赋值
    void result(boolean success, T param, WorkResult<V> workResult);
}

每个worker的回调，worker执行完毕后，会执行该接口，带着执行成功、失败、原始入参和详细的结果。

WorkerWrapper

public class WorkerWrapper<T, V> {}

组合了IWorker和ICallback，是最小的调度单元。通过编排wrapper之间的关系，达到组合各个worker顺序的目的。wrapper的类型和worker的类型一致。

任务编排

粗略来看，分为三种：顺序，并行，顺序和并行。先看顺序，并行，以及顺序和并行简单的写法，最后着重多写几个复杂的操作。

为了简单些，先写出公用的工作类。将Work0复制2份，分别命名为Work1，Work2，Work3，Work4，除类名改变之外，其他的不需要变化。

代码如下：

package org.gjy.m8.tool;

import com.jd.platform.async.callback.ICallback;
import com.jd.platform.async.callback.IWorker;
import com.jd.platform.async.worker.WorkResult;
import com.jd.platform.async.wrapper.WorkerWrapper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Map;
import java.util.concurrent.TimeUnit;

public class Work0 implements IWorker<String, String>, ICallback<String, String> {
    private static final Logger log = LoggerFactory.getLogger(Work0.class);
    private final String CLASS_NAME = Work0.class.getSimpleName();

    @Override
    public void begin() {
        log.info("{} begin", CLASS_NAME);
    }

    @Override
    public void result(boolean success, String param, WorkResult<String> workResult) {
        if (success) { // 执行成功
            log.info("{} success={}, param={}, result={}", CLASS_NAME, true, param, workResult);
        } else {
            log.error("{} success={}, param={}, result={}", CLASS_NAME, false, param, workResult);
        }
    }

    @Override
    public String action(String param, Map<String, WorkerWrapper> allWrappers) {
        try {
            TimeUnit.SECONDS.sleep(1);
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
        log.info("{} param={},wrappers={}", CLASS_NAME, param, allWrappers);
        return CLASS_NAME + " action";
    }

    @Override
    public String defaultValue() {
        log.info("{} defaultValue", CLASS_NAME);
        return CLASS_NAME;
    }
}

顺序

在编写WorkWrapper时，如果使用（next）按照预想的执行顺序相反的写，如果使用（depend）按照预想的顺序写，按照这种方式，会简单点。

可以改变 Async.beginWork 的超时时间，或改变Work action中的睡眠时间，看不同的效果。

超时的时候返回的是 defaultValue 方法中的默认值，合理的评估时间，要不然会导致后面的任务得不到执行。

package org.gjy.m8.tool;

import com.jd.platform.async.executor.Async;
import com.jd.platform.async.wrapper.WorkerWrapper;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.concurrent.ExecutionException;

public class AsyncTool2Test {

    private static final Logger log = LoggerFactory.getLogger(AsyncTool2Test.class);

    @Test
    public void seq() throws ExecutionException, InterruptedException {
        Work0 work0 = new Work0();
        Work1 work1 = new Work1();
        Work2 work2 = new Work2();

        // 执行顺序 0-1-2
        WorkerWrapper<String, String> wrapper2 = new WorkerWrapper.Builder<String, String>()
                .worker(work2)
                .callback(work2)
                .param("work2")
                .build();
        WorkerWrapper<String, String> wrapper1 = new WorkerWrapper.Builder<String, String>()
                .worker(work1)
                .callback(work1)
                .param("work1")
                .next(wrapper2)
                .build();
        WorkerWrapper<String, String> wrapper0 = new WorkerWrapper.Builder<String, String>()
                .worker(work0)
                .callback(work0)
                .param("work0")
                .next(wrapper1)
                .build();

        log.info("执行任务开始");
        Async.beginWork(1500, wrapper0);
        log.info("任务执行结束");
        Async.shutDown();
    }
}

并行

想要并行时，可以将所有的Work 放在 Async.beginWork 最后。

package org.gjy.m8.tool;

import com.jd.platform.async.executor.Async;
import com.jd.platform.async.wrapper.WorkerWrapper;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.concurrent.ExecutionException;

public class AsyncTool2Test {

    private static final Logger log = LoggerFactory.getLogger(AsyncTool2Test.class);

    @Test
    public void parallel() throws ExecutionException, InterruptedException {
        Work0 work0 = new Work0();
        Work1 work1 = new Work1();
        Work2 work2 = new Work2();

        WorkerWrapper<String, String> wrapper2 = new WorkerWrapper.Builder<String, String>()
                .worker(work2)
                .callback(work2)
                .param("work2")
                .build();
        WorkerWrapper<String, String> wrapper1 = new WorkerWrapper.Builder<String, String>()
                .worker(work1)
                .callback(work1)
                .param("work1")
                .build();
        WorkerWrapper<String, String> wrapper0 = new WorkerWrapper.Builder<String, String>()
                .worker(work0)
                .callback(work0)
                .param("work0")
                .build();

        log.info("执行任务开始");
        Async.beginWork(1500, wrapper0, wrapper2, wrapper1);
        log.info("任务执行结束");
        Async.shutDown();
    }
}

顺序和并行

一个简单的示例，0 -> 1,2 -> 3。0执行完毕后，开启1,2执行，1,2执行完毕后执行3。

有两种实现方式，第一种（只是用 next）

@Test
public void mixed1() throws ExecutionException, InterruptedException {
    Work0 work0 = new Work0();
    Work1 work1 = new Work1();
    Work2 work2 = new Work2();
    Work3 work3 = new Work3();

    WorkerWrapper<String, String> wrapper3 = new WorkerWrapper.Builder<String, String>()
        .worker(work3).callback(work3).param("work03").build();

    WorkerWrapper<String, String> wrapper2 = new WorkerWrapper.Builder<String, String>()
        .worker(work2).callback(work2).param("work2").next(wrapper3).build();

    WorkerWrapper<String, String> wrapper1 = new WorkerWrapper.Builder<String, String>()
        .worker(work1).callback(work1).param("work1").next(wrapper3).build();

    WorkerWrapper<String, String> wrapper0 = new WorkerWrapper.Builder<String, String>()
        .worker(work0).callback(work0).param("work0").next(wrapper1, wrapper2).build();

    log.info("执行任务开始");
    Async.beginWork(6000, wrapper0);
    log.info("任务执行结束");
    Async.shutDown();
}

第二种（next depend）

@Test
public void mixed2() throws ExecutionException, InterruptedException {
    Work0 work0 = new Work0();
    Work1 work1 = new Work1();
    Work2 work2 = new Work2();
    Work3 work3 = new Work3();

    WorkerWrapper<String, String> wrapper0 = new WorkerWrapper.Builder<String, String>()
        .worker(work0).callback(work0).param("work0").build();
    WorkerWrapper<String, String> wrapper3 = new WorkerWrapper.Builder<String, String>()
        .worker(work3).callback(work3).param("work3").build();
    WorkerWrapper<String, String> wrapper2 = new WorkerWrapper.Builder<String, String>()
        .worker(work2).callback(work2)
        .next(wrapper3).depend(wrapper0).param("work2").build();
    WorkerWrapper<String, String> wrapper1 = new WorkerWrapper.Builder<String, String>()
        .worker(work1).callback(work1)
        .next(wrapper3).depend(wrapper0).param("work1").build();

    log.info("执行任务开始");
    Async.beginWork(6000, wrapper0);
    log.info("任务执行结束");
    Async.shutDown();
}

总结

通过 next 和 depend 可以有效地指定任务的执行顺序。

最后一个案例，讲了一下遇到特殊情况，next参数的选择。执行结构如下：

0001.png

@Test
public void mixed3() throws ExecutionException, InterruptedException {
    Work0 work0 = new Work0();
    Work1 work1 = new Work1();
    Work2 work2 = new Work2();
    Work3 work3 = new Work3();
    Work4 work4 = new Work4();

    WorkerWrapper<String, String> wrapper0 = new WorkerWrapper.Builder<String, String>()
        .worker(work0).callback(work0).param("work0").build();
    WorkerWrapper<String, String> wrapper1 = new WorkerWrapper.Builder<String, String>()
        .worker(work1).callback(work1).param("work1").build();

    WorkerWrapper<String, String> wrapper2 = new WorkerWrapper.Builder<String, String>()
        .worker(work2).callback(work2).depend(wrapper0).param("work2").build();
    WorkerWrapper<String, String> wrapper3 = new WorkerWrapper.Builder<String, String>()
        .worker(work3).callback(work3).depend(wrapper1).param("work3").build();
    
    // next中的selfIsMust参数默认为true，表示强依赖自己，必定执行，如果为false的话，此项任务中wrapper4会执行失败
    WorkerWrapper<String, String> wrapper4 = new WorkerWrapper.Builder<String, String>()
        .worker(work4).callback(work4).depend(wrapper3).next(wrapper2, false).param("work4").build();

    Async.beginWork(5000, Executors.newCachedThreadPool(), wrapper0, wrapper1);
    Async.shutDown();
}

更多的测试代码，可以在 gitee 上面找到 asyncTool 项目，查看测试用例。

avatar
文章
阅读
粉丝