Reactor3 Flux#join()源码解读:

669 阅读8分钟

示例Demo:

@Test public void fun7(){
    Flux<Integer> flux1 = Flux.just(1, 2, 3);
    Flux<Integer> flux2 = Flux.just(4, 5, 6);
    flux1.join(flux2,x->Flux.just(x).delayElements(Duration.ofSeconds(0)),Flux::just,(x,y)->x.toString() + "-" + y.toString())
            .subscribe(x -> System.out.println("onNext: " + x));
    sleep(1);
}

join api源码:

//创建FluxJoin,包装flux1、flux2、leftEnd、rightEnd、 resultSelector。由类型可知,当flux1和flux2产生订阅后,flux1下发T flux2下发TRight,Function<? super T, ? extends Publisher<TLeftEnd>> leftEnd在收到下发T元素后产生一个Publisher<TLeftEnd>源,同样Function<? super TRight, ? extends Publisher<TRightEnd>> rightEnd产生一个Publisher<TRightEnd>源。这两个源好比一个中间调控,产生订阅用来控制TLeft和TRight元素。resultSelector用于将flux1和flux2发射的元素进行转换得到新元素,通过订阅进行消费该新元素。flux1、flux2以及内部产生的两个源Publisher<TLeftEnd>、Publisher<TLeftEnd>都需要订阅,因此会有4个订阅。另外Flux操作api中,如果没使用Scheduler调度器,则只在单线程中执行。
public final <TRight, TLeftEnd, TRightEnd, R> Flux<R> join(
		Publisher<? extends TRight> other,
		Function<? super T, ? extends Publisher<TLeftEnd>> leftEnd,
		Function<? super TRight, ? extends Publisher<TRightEnd>> rightEnd,
		BiFunction<? super T, ? super TRight, ? extends R> resultSelector
) {
	return onAssembly(new FluxJoin<T, TRight, TLeftEnd, TRightEnd, R>(
			this, other, leftEnd, rightEnd, resultSelector));
}

订阅核心源码:

//这里创建一个综合的JoinSubscription parent,用来管理flux1、flux2、Publisher<TLeftEnd>、Publisher<TLeftEnd>下发的元素。结合后面的源码可以看到,它维护了一个MpscLinkedQueue的队列数据结构用来存储这4个源下发的元素。
@Override
public void subscribe(CoreSubscriber<? super R> actual) {
    //创建JoinSubscription,包装订阅者actual、leftEnd、rightEnd、resultSelector
	JoinSubscription<TLeft, TRight, TLeftEnd, TRightEnd, R> parent =
			new JoinSubscription<>(actual,
					leftEnd,
					rightEnd,
					resultSelector);

	actual.onSubscribe(parent);

    //创建2个订阅者left和right,分别用于订阅flux1和flux2并将订阅到的元素上交parent
    //外层parent引用交给内层LeftRightSubscriber维护
	LeftRightSubscriber left = new LeftRightSubscriber(parent, true);
	//parent管理内层订阅者,这样left和right的两个订阅者LeftRightSubscriber通过订阅将flux1、flux2下发的元素交给parent,通过parent内部包装的resultSelector将元素转换进而由actual消费。但在订阅过程中,会调用leftEnd#apply(item)、rightEnd#apply(item)。
	parent.cancellations.add(left);
	LeftRightSubscriber right = new LeftRightSubscriber(parent, false);
	parent.cancellations.add(right);

    //需要flux1先产生订阅,注意如果不使用调度器flux1先订阅完,之后flux2再订阅
	source.subscribe(left);
	//flux2先产生订阅
	other.subscribe(right);
}

接下来看flux1、flux2订阅源码:

//首先明白发布订阅流程:
Publisher#subscribe(Subscriber) -> Subscriber#onSubscribe(Subscription) -> Subscription#request(long) -> Subscriber#onNext(item)


再来看flux1、flux2订阅源码:
    //flux1.subscribe(left),这里对flux1产生订阅。传入一个订阅者LeftRightSubscriber left
    source.subscribe(left);
    
    //flux2.subscribe(right),这里对flux2产生订阅。传入一个订阅者LeftRightSubscriber right
	other.subscribe(right);


实际上这两个订阅逻辑类似,接下来看source.subscribe(left)源码:
    void fastPath() {
		final T[] a = array;
		final int len = a.length;
		final Subscriber<? super T> s = actual;

        //遍历FluxArray中的array下发元素
		for (int i = index; i != len; i++) {
			if (cancelled) {
				return;
			}

			T t = a[i];

			if (t == null) {
				s.onError(new NullPointerException("The " + i + "th array element was null"));
				return;
			}
            //调用left订阅者即上一步传入的LeftRightSubscriber left消费下发元素
			s.onNext(t);
		}
		if (cancelled) {
			return;
		}
		s.onComplete();
	}


接下来看s.onNext(t)源码:
//LeftRightSubscriber#onNext
	@Override
	public void onNext(Object t) {
	    //将t元素上交给parent,上一步创建left和right两个LeftRightSubscriber时给定了isLeft,因此isLeft用来标识是left订阅者还是right订阅者,而它俩的消费逻辑便是将t元素上交给其外层parent(上一步创建left和right两个LeftRightSubscriber时同样给定了parent引用指向,而parent同样包装了)
		parent.innerValue(isLeft, t);
	}


接下来看parent.innerValue(isLeft, t)源码:
    //JoinSubscription#innerValue(boolean isLeft, Object o)
	@Override
	public void innerValue(boolean isLeft, Object o) {
	    //常量LEFT_VALUE=1  RIGHT_VALUE=2
	    //调用JoinSubscription.queueBiOffer#test即JoinSubscription中维护的MpscLinkedQueue#test
	    //存储下发元素
		queueBiOffer.test(isLeft ? LEFT_VALUE : RIGHT_VALUE, o);
		//消费元素
		drain();
	}
	
	
先来看queueBiOffer.test(isLeft ? LEFT_VALUE : RIGHT_VALUE, o)源码:
	//通过MpscLinkedQueue存储元素,同时建立链表关系,MpscLinkedQueue中维护的volatile变量producerNode指向nextNextNode节点,该节点存储下发元素
	@Override
	@SuppressWarnings("unchecked")
	public boolean test(E e1, E e2) {
		Objects.requireNonNull(e1, "The offered value 'e1' must be non-null");
		Objects.requireNonNull(e2, "The offered value 'e2' must be non-null");

        //用传入的LEFT_VALUE或RIGHT_VALUE、item元素分别创建两个LinkedQueueNode
		final LinkedQueueNode<E> nextNode = new LinkedQueueNode<>(e1);
		final LinkedQueueNode<E> nextNextNode = new LinkedQueueNode<>(e2);

        //新建一个prevProducerNode,并将MpscLinkedQueue中维护的volatile变量producerNode指向nextNextNode
		final LinkedQueueNode<E> prevProducerNode = PRODUCER_NODE_UPDATER.getAndSet(this, nextNextNode);
		
		// Should a producer thread get interrupted here the chain WILL be broken until that thread is resumed
		// and completes the store in prev.next.
		
		//nextNode节点 next指针指向nextNextNode
		nextNode.soNext(nextNextNode);
		
		//prevProducerNode节点 next指针指向nextNode
		prevProducerNode.soNext(nextNode); // StoreStore

		return true;
	}
	
再来看JoinSubscription#drain()源码:
    
    //FluxJoin.JoinSubscription#drain()
    void drain() {
        //首次订阅,同一消费者对于同一元素只能消费一次
		if (WIP.getAndIncrement(this) != 0) {
			return;
		}

		int missed = 1;
		Queue<Object> q = queue;    //拿到JoinSubscription中维护的MpscLinkedQueue
		Subscriber<? super R> a = actual;

		for (; ; ) {
			for (; ; ) {
				if (cancellations.isDisposed()) {
					q.clear();
					return;
				}

				Throwable ex = error;
				if (ex != null) {
					q.clear();
					cancellations.dispose();
					errorAll(a);
					return;
				}

				boolean d = active == 0;

                //从队列弹出上一步nextNode节点包装的mode
				Integer mode = (Integer) q.poll();

				boolean empty = mode == null;

				if (d && empty) {

					lefts.clear();
					rights.clear();
					cancellations.dispose();

					a.onComplete();
					return;
				}

				if (empty) {
					break;
				}

				Object val = q.poll();

                //对于left订阅产生的元素处理
				if (mode == LEFT_VALUE) {
					@SuppressWarnings("unchecked") TLeft left = (TLeft) val;

					int idx = leftIndex++;
					
					//存入Map,idx为key由0开始递增,left为下发元素
					lefts.put(idx, left);

					Publisher<TLeftEnd> p;

					try {
					    //通过下发元素产生新的源Publisher<TLeftEnd>
						p = Objects.requireNonNull(leftEnd.apply(left),
								"The leftEnd returned a null Publisher");
					}
					catch (Throwable exc) {
						Exceptions.addThrowable(ERROR,
								this,
								Operators.onOperatorError(this, exc, left,
										actual.currentContext()));
						errorAll(a);
						return;
					}
                    
                    //创建新的订阅者,同样维护一个外部parent引用。当它订阅Publisher<TLeftEnd>时,会调用其onNext(item)也就是FluxGroupJoin.LeftRightEndSubscriber#onNext(Object t),我们跳到下面去看它的onNext(Object t)源码:
					LeftRightEndSubscriber end =
							new LeftRightEndSubscriber(this, true, idx);
					
					//parent同样将订阅者LeftRightEndSubscriber管理起来
					cancellations.add(end);

                    //通过源码得知,订阅时调用parent.innerClose(isLeft, end),向queue中存入LEFT_CLOSE或RIGHT_CLOSE和end
					p.subscribe(end);

					ex = error;
					if (ex != null) {
						q.clear();
						cancellations.dispose();
						errorAll(a);
						return;
					}

					long r = requested;
					long e = 0L;
                    
                    //如果单线程执行,rights很显然没存储flux2下发的元素,因此只有当引入Scheduler调度器产生异步时才可能从rights得到值
					for (TRight right : rights.values()) {

						R w;

						try {
						    //对flux1、flux2下发的元素进行resultSelector.apply计算得到结果w
							w = Objects.requireNonNull(resultSelector.apply(left,
									right),
									"The resultSelector returned a null value");
						}
						catch (Throwable exc) {
							Exceptions.addThrowable(ERROR,
									this,
									Operators.onOperatorError(this,
											exc, right, actual.currentContext()));
							errorAll(a);
							return;
						}

						if (e != r) {
						    //消费结果w
							a.onNext(w);

							e++;
						}
						else {
							Exceptions.addThrowable(ERROR,
									this,
									Exceptions.failWithOverflow("Could not " + "emit value due to lack of requests"));
							q.clear();
							cancellations.dispose();
							errorAll(a);
							return;
						}
					}

					if (e != 0L) {
						Operators.produced(REQUESTED, this, e);
					}
				}
				//同mode == LEFT_VALUE
				else if (mode == RIGHT_VALUE) {
					@SuppressWarnings("unchecked") TRight right = (TRight) val;

					int idx = rightIndex++;

					rights.put(idx, right);

					Publisher<TRightEnd> p;

					try {
						p = Objects.requireNonNull(rightEnd.apply(right),
								"The rightEnd returned a null Publisher");
					}
					catch (Throwable exc) {
						Exceptions.addThrowable(ERROR,
								this,
								Operators.onOperatorError(this, exc, right,
										actual.currentContext()));
						errorAll(a);
						return;
					}

					LeftRightEndSubscriber end =
							new LeftRightEndSubscriber(this, false, idx);
					cancellations.add(end);

					p.subscribe(end);

					ex = error;
					if (ex != null) {
						q.clear();
						cancellations.dispose();
						errorAll(a);
						return;
					}

					long r = requested;
					long e = 0L;

					for (TLeft left : lefts.values()) {

						R w;

						try {
							w = Objects.requireNonNull(resultSelector.apply(left,
									right),
									"The resultSelector returned a null value");
						}
						catch (Throwable exc) {
							Exceptions.addThrowable(ERROR,
									this,
									Operators.onOperatorError(this, exc, left,
											actual.currentContext()));
							errorAll(a);
							return;
						}

						if (e != r) {
							a.onNext(w);

							e++;
						}
						else {
							Exceptions.addThrowable(ERROR,
									this,
									Exceptions.failWithOverflow("Could not emit " + "value due to lack of requests"));
							q.clear();
							cancellations.dispose();
							errorAll(a);
							return;
						}
					}

					if (e != 0L) {
						Operators.produced(REQUESTED, this, e);
					}
				}
				//flux1下发元素t时,同样创建一个内部源Publisher<TLeftEnd>并创建LeftRightEndSubscriber对其订阅。消费逻辑很简单,从Map lefts中移除flux1下发的当前元素,这样在flux2订阅消费下发元素时parent遍历lefts便拿不到当前flux1下发元素,也就是结果不会进入resultSelector进行计算。同理下面的else if (mode == RIGHT_CLOSE)也是如此
				else if (mode == LEFT_CLOSE) {
					LeftRightEndSubscriber end = (LeftRightEndSubscriber) val;

					lefts.remove(end.index);
					cancellations.remove(end);
				}
				else if (mode == RIGHT_CLOSE) {
					LeftRightEndSubscriber end = (LeftRightEndSubscriber) val;

					rights.remove(end.index);
					cancellations.remove(end);
				}
			}

			missed = WIP.addAndGet(this, -missed);
			if (missed == 0) {
				break;
			}
		}
	}

//上面的p.subscribe(end)订阅实现:
static <T, R> boolean trySubscribeScalarMap(Publisher<? extends T> source,
		CoreSubscriber<? super R> s,
		Function<? super T, ? extends Publisher<? extends R>> mapper,
		boolean fuseableExpected) {
	if (source instanceof Callable) {
		T t;

		try {
			t = ((Callable<? extends T>) source).call();
		}
		catch (Throwable e) {
			Operators.error(s, Operators.onOperatorError(e, s.currentContext()));
			return true;
		}

		if (t == null) {
			Operators.complete(s);
			return true;
		}

		Publisher<? extends R> p;

		try {
			p = Objects.requireNonNull(mapper.apply(t),
					"The mapper returned a null Publisher");
		}
		catch (Throwable e) {
			Operators.error(s, Operators.onOperatorError(null, e, t, s.currentContext()));
			return true;
		}

		if (p instanceof Callable) {
			R v;

			try {
				v = ((Callable<R>) p).call();
			}
			catch (Throwable e) {
				Operators.error(s, Operators.onOperatorError(null, e, t, s.currentContext()));
				return true;
			}

			if (v != null) {
				s.onSubscribe(Operators.scalarSubscription(s, v));
			}
			else {
				Operators.complete(s);
			}
		}
		else {
			if (!fuseableExpected || p instanceof Fuseable) {
				p.subscribe(s);
			}
			else {
				p.subscribe(new FluxHide.SuppressFuseableSubscriber<>(s));
			}
		}

		return true;
	}

	return false;
}

    //内部的两个源Publisher<TLeftEnd>和Publisher<TRightEnd>订阅时会调用该方法:
    //LeftRightEndSubscriber#onNext(Object t)
	@Override
	public void onNext(Object t) {
		if (Operators.terminate(SUBSCRIPTION, this)) {
			parent.innerClose(isLeft, this);
		}
	}
    

    //再来看parent.innerClose(isLeft, this)实现:
        @Override
		public void innerClose(boolean isLeft, LeftRightEndSubscriber index) {
		    //往queue中存储LEFT_CLOSE或RIGHT_CLOSE、LeftRightEndSubscriber
			queueBiOffer.test(isLeft ? LEFT_CLOSE : RIGHT_CLOSE, index);
			//else if (mode == LEFT_CLOSE)、else if (mode == RIGHT_CLOSE)
			drain();
		}

总结:Flux#join#subscribe()通过创建一个JoinSubscription用来管理flux1、flux2下发元素同时也用来管理中间调控操作Publisher、Publisher的订阅过程。当产生订阅时,只需要让flux1、flux2产生订阅,并将下发元素上交JoinSubscription存储起来。在JoinSubscription内部根据flux1、flux2下发元素分别创建新的源即作Publisher、Publisher。JoinSubscription维护一个Queue用来存储flux1、flux2下发的元素以及Publisher、Publisher的订阅者,同时用Map结构存储flux1、flux2下发元素。当消费元素时,从queue中弹出元素,根据不同mode(首先只会有1或2)进行相应处理,在每次处理中先创建内部源进行调控,调控则是将内部源的订阅者存入queue同时给定mode为3或4,调控订阅会从JoinSubscription.Map中删除当前订阅消费的元素,这样下次消费时,便不会再有该元素。之后当有异步时,消费left或right时,遍历rights或lefts,将下发元素和遍历得到的存储元素通过resultSelector计算得到结果并订阅消费。