gateway 自定义负载均衡实现

Rollway
2,005 阅读5分钟

环境：

    <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-gateway</artifactId>
        <version>2.2.5.RELEASE</version>
    </dependency>

前言:

老项目某个业务使用的是ehcache缓存,现有个需求需要对缓存进行优化,现有网关是默认轮询机制负载均衡,现在需要将一个用户的请求转发到固定的服务器,所以我们需要提供一个新的loadBalancer,用户名,手机设备都可以是判断的依据，这里选择根据设备号进行转发

1.原有的负载均衡过滤器实现

传统spring项目是DispatcherServlet进行转发调用，网关gateway(webflux)是DispatcherHandler 处理，逻辑一样也是根据handlermaping 找handler处理，这里我们找到最后的WebHandler(FilteringWebHandler) 打个断点看看

image.png 可以看到默认的负载均衡过滤器是LoadBalancerClientFilter， 而在该类的注释上标注了 in favour of {@link ReactiveLoadBalancerClientFilter}
推荐使用ReactiveLoadBalancerClientFilter 那么如何切换呢，我们找到这个类的自动装配类GatewayReactiveLoadBalancerClientAutoConfiguration

@Bean
@ConditionalOnBean(LoadBalancerClientFactory.class)
@ConditionalOnMissingBean(ReactiveLoadBalancerClientFilter.class)
@Conditional(OnNoRibbonDefaultCondition.class)
public ReactiveLoadBalancerClientFilter gatewayLoadBalancerClientFilter(
      LoadBalancerClientFactory clientFactory, LoadBalancerProperties properties) {
   return new ReactiveLoadBalancerClientFilter(clientFactory, properties);
}
//这里表示下面有任意一项符合就可以
private static final class OnNoRibbonDefaultCondition extends AnyNestedCondition {
   private OnNoRibbonDefaultCondition() {
      super(ConfigurationPhase.REGISTER_BEAN);
   }
   @ConditionalOnProperty(value = "spring.cloud.loadbalancer.ribbon.enabled",
         havingValue = "false")
   static class RibbonNotEnabled {
   }
  @ConditionalOnMissingClass("org.springframework.cloud.netflix.ribbon.RibbonLoadBalancerClient")
   static class RibbonLoadBalancerNotPresent {
   }
}

它是基于配置的,只要修改配置spring.cloud.loadbalancer.ribbon.enabled，就会换成reactive的实现

image.png

那么接下来 我们需要修改这个过滤器，添加自定义逻辑

ReactiveLoadBalancerClientFilter

public class ReactiveLoadBalancerClientFilter implements GlobalFilter, Ordered {
 
   private final LoadBalancerClientFactory clientFactory;
   private LoadBalancerProperties properties;
   public ReactiveLoadBalancerClientFilter(LoadBalancerClientFactory clientFactory,
         LoadBalancerProperties properties) {
      this.clientFactory = clientFactory;
      this.properties = properties;
   }

   @Override
   @SuppressWarnings("Duplicates")
   public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
      ...
      return choose(exchange).doOnNext(response -> {
         URI uri = exchange.getRequest().getURI();
         根据loadbalancer返回的服务实例改写请求host lb:xx-xx-xx -> http://x.x.x.x:xx
         DelegatingServiceInstance serviceInstance = new DelegatingServiceInstance(
               response.getServer(), overrideScheme);
         URI requestUrl = reconstructURI(serviceInstance, uri);
         exchange.getAttributes().put(GATEWAY_REQUEST_URL_ATTR, requestUrl);
      }).then(chain.filter(exchange));
   }

   private Mono<Response<ServiceInstance>> choose(ServerWebExchange exchange) {
      URI uri = exchange.getAttribute(GATEWAY_REQUEST_URL_ATTR);
      获取请求路径相关的loadbalancer getInstance内部调用 NamedContextFactory#getInstance 
      ReactorLoadBalancer<ServiceInstance> loadBalancer = this.clientFactory
            .getInstance(uri.getHost(), ReactorLoadBalancer.class,
                  ServiceInstance.class);
      调用loadbalancer返回instance服务实例 默认的实现只有一个RoundRobinLoadBalancer， 这里传入了一个 空的DefaultRequest对象，其实里面啥都没有
      return loadBalancer.choose(createRequest());
   }

   private Request createRequest() {
      return ReactiveLoadBalancer.REQUEST;
   }
}

从上面看下来，那么我们知道关键在于重写RoundRobinLoadBalancer 或者 RoundRobinLoadBalancer.choose方法了,我们再看看如何返回host相关的loadbalancer的

public <T> T getInstance(String name, ResolvableType type) {
   这里相当于为每个请求host创建了个子容器 毫无疑问 肯定会落入相关缓存 
   AnnotationConfigApplicationContext context = getContext(name);
   获取ReactorLoadBalancer实现类名称
   String[] beanNames = BeanFactoryUtils.beanNamesForTypeIncludingAncestors(context,
         type);
   if (beanNames.length > 0) {
      for (String beanName : beanNames) {
         if (context.isTypeMatch(beanName, type)) {
            获取ReactorLoadBalancer实例
            return (T) context.getBean(beanName);
         }
      }
   }
   return null;
}

我们看看 RoundRobinLoadBalancer实现逻辑

public class RoundRobinLoadBalancer implements ReactorServiceInstanceLoadBalancer {
   public Mono<Response<ServiceInstance>> choose(Request request) {
      获取host相关实例列表
      if (serviceInstanceListSupplierProvider != null) {
         ServiceInstanceListSupplier supplier = serviceInstanceListSupplierProvider
               .getIfAvailable(NoopServiceInstanceListSupplier::new);
         return supplier.get().next().map(this::getInstanceResponse);
      }
      ServiceInstanceSupplier supplier = this.serviceInstanceSupplier
            .getIfAvailable(NoopServiceInstanceSupplier::new);
      return supplier.get().collectList().map(this::getInstanceResponse);
   }

   private Response<ServiceInstance> getInstanceResponse(
         List<ServiceInstance> instances) {
      根据AtomicInteger对实例size取余将请求均匀打到各个实例上 那么毫无疑问 我们要改写getInstanceResponse这个方法
      int pos = Math.abs(this.position.incrementAndGet());
      ServiceInstance instance = instances.get(pos % instances.size());
      return new DefaultResponse(instance);
   }
}

从上面可以看到 request对象是没有任何作用的，而我们是需要根据设备号进行固定转发的，设备号相关信息肯定要传进来，最好是直接传进来一个 ServerWebExchange对象，那么我们可以重写ReactiveLoadBalancerClientFilter调用loadBalancer.choose(createRequest())方法，重写一个Request对象，添加当前请求的ServerWebExchange信息，最后重写RoundRobinLoadBalancer 的getInstanceResponse方法。

那么第一步，修改RoundRobinLoadBalancer，我们看下它的装配方式

@Configuration(proxyBeanMethods = false)
@ConditionalOnDiscoveryEnabled
public class LoadBalancerClientConfiguration {
  @Bean
  @ConditionalOnMissingBean
  public ReactorLoadBalancer<ServiceInstance> reactorServiceInstanceLoadBalancer(
        Environment environment,
        LoadBalancerClientFactory loadBalancerClientFactory) {
     String name = environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
     return new RoundRobinLoadBalancer(loadBalancerClientFactory.getLazyProvider(name,
           ServiceInstanceListSupplier.class), name);
  }

当我们在上方创建方法处打断点时发现，这里并不是服务启动就进行注册的，而是每个不同lb:host的请求进入都会创建一次，这也印证了上方一个host 对应一个AnnotationConfigApplicationContext。LoadBalancerClientConfiguration带有@Configuration注解，服务启动不创建，那么代表没有import或者spring.factories文件中标明，而是在AnnotationConfigApplicationContext创建之初设置进去的。 这就有点麻烦了，我们需要看看子容器是如何创建的，毕竟我们可能需要让这个RoundRobinLoadBalancer不再创建，并且创建我们自定义的ReactorLoadBalancer，我们看看 AnnotationConfigApplicationContext创建过程

NamedContextFactory#getContext

protected AnnotationConfigApplicationContext getContext(String name) {
   if (!this.contexts.containsKey(name)) {
      synchronized (this.contexts) {
         if (!this.contexts.containsKey(name)) {
            this.contexts.put(name, createContext(name));
         }
      }
   }
   return this.contexts.get(name);
}

protected AnnotationConfigApplicationContext createContext(String name) {
   AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
   ...
   //C extends NamedContextFactory.Specification
   for (Map.Entry<String, C> entry : this.configurations.entrySet()) {
      if (entry.getKey().startsWith("default.")) {
         for (Class<?> configuration : entry.getValue().getConfiguration()) {
            context.register(configuration);
         }
      }
   }
   //这里的defaultConfigType 就是LoadBalancerClientConfiguration 果然是在创建之初放进去的
   context.register(PropertyPlaceholderAutoConfiguration.class,
         this.defaultConfigType);
   ...
   context.refresh();
   return context;
}

既然上方 LoadBalancerClientConfiguration是以 defaultConfigType 形式直接塞进去的，进一步追溯，发现 LoadBalancerClientFactory这个构造方法直接写死了，并且LoadBalancerClientConfiguration里面还负责创建其他的bean ，那我们就不动它了，看看 for (Map.Entry<String, C> entry : this.configurations.entrySet()) ， configurations 是什么，这里面的value也是会被注册进去的

image.png

NamedContextFactory的实例是LoadBalancerClientFactory，看看它的装配方式

public class LoadBalancerAutoConfiguration {
    //这里是ObjectProvider 类似懒加载，注入的时候不处理，调用getIfAvailable 从容器中获取，这里我们看到获取的是LoadBalancerClientSpecification 集合
   private final ObjectProvider<List<LoadBalancerClientSpecification>> configurations;

   @ConditionalOnMissingBean
   @Bean
   public LoadBalancerClientFactory loadBalancerClientFactory() {
      LoadBalancerClientFactory clientFactory = new LoadBalancerClientFactory();
      配置就是在这里设置的
      clientFactory.setConfigurations(
            this.configurations.getIfAvailable(Collections::emptyList));
      return clientFactory;
   }
}

看到了 LoadBalancerClientSpecification ，继续追

public class LoadBalancerClientConfigurationRegistrar
      implements ImportBeanDefinitionRegistrar {
   private static void registerClientConfiguration(BeanDefinitionRegistry registry,
         Object name, Object configuration) {
      BeanDefinitionBuilder builder = BeanDefinitionBuilder
            .genericBeanDefinition(LoadBalancerClientSpecification.class);
      builder.addConstructorArgValue(name);
      builder.addConstructorArgValue(configuration);
      registry.registerBeanDefinition(name + ".LoadBalancerClientSpecification",
            builder.getBeanDefinition());
   }
   
   @Override
   public void registerBeanDefinitions(AnnotationMetadata metadata,
         BeanDefinitionRegistry registry) {
      Map<String, Object> attrs = metadata
            .getAnnotationAttributes(LoadBalancerClients.class.getName(), true);
      if (attrs != null && attrs.containsKey("value")) {
         AnnotationAttributes[] clients = (AnnotationAttributes[]) attrs.get("value");
         for (AnnotationAttributes client : clients) {
            registerClientConfiguration(registry, getClientName(client),
                  client.get("configuration"));
         }
      }
      ...
   }
}

我们找到了这样一段代码 ImportBeanDefinitionRegistrar.registerBeanDefinitions是可以额外注册beanDefinition的 ，这里则是根据@LoadBalancerClients中 defaultConfiguration 额外注册 LoadBalancerClientSpecification ，那么知道怎么玩了，在任意一个bean上面标注@LoadBalancerClients这样一个注解，defaultConfiguration填上 一个配置类class，这个配置类就会在创建loadbalancer子容器中添加进去

2.自定义负载均衡实现

注解挂载

@LoadBalancerClients(defaultConfiguration = DeviceBalancerClientConfiguration.class)
任意bean直接挂上这个注解

自定义配置类

@Configuration(proxyBeanMethods = false)
public class DeviceBalancerClientConfiguration {
    //自定义DeviceLoadBalancer 可以直接继承 RoundRobinLoadBalancer 或者自己实现 ReactorServiceInstanceLoadBalancer这个接口
    @Bean
    public DeviceLoadBalancer reactorServiceInstanceLoadBalancer(
            Environment environment,
            LoadBalancerClientFactory loadBalancerClientFactory) {
        String name = environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new DeviceLoadBalancer(loadBalancerClientFactory.getLazyProvider(name,
                ServiceInstanceListSupplier.class), name);
    }
}

自定义过滤器

public class DeviceReactiveLoadBalancerClientFilter extends ReactiveLoadBalancerClientFilter {
    重写choose方法 重写Request对象 添加 ServerWebExchange请求信息
    private Mono<Response<ServiceInstance>> choose(ServerWebExchange exchange) {
        URI uri = exchange.getAttribute(GATEWAY_REQUEST_URL_ATTR);
        ReactorLoadBalancer<ServiceInstance> loadBalancer = this.clientFactory
                .getInstance(uri.getHost(), ReactorLoadBalancer.class,
                        ServiceInstance.class);
        if (loadBalancer == null) {
            throw new NotFoundException("No loadbalancer available for " + uri.getHost());
        }
        return loadBalancer.choose(new ExchangeRequest(exchange));
    }

    @Data
    public static class ExchangeRequest implements Request {
        private ServerWebExchange exchange;
        public ExchangeRequest(ServerWebExchange exchange) {
            this.exchange = exchange;
        }
    }
}

自定义loadbalancer

public class DeviceLoadBalancer extends RoundRobinLoadBalancer{
 
    private Response<ServiceInstance> getInstanceResponse(List<ServiceInstance> instances,Request request) {
        DeviceReactiveLoadBalancerClientFilter.ExchangeRequest exchangeRequest = (DeviceReactiveLoadBalancerClientFilter.ExchangeRequest) request;
        String deviceId = exchangeRequest.getExchange().getRequest().getHeaders().getFirst(GatewayConstants.HEADER_DEVICE_ID);
        int pos;
        if(StringUtils.isNotBlank(deviceId)){
            pos = Math.abs(deviceId.hashCode());
        }else {
            pos = Math.abs(this.position.incrementAndGet());
        }
        ServiceInstance instance = instances.get(pos % instances.size());
        return new DefaultResponse(instance);
    }
}

最后自定义配置类不需要在主容器中实例化，扫描时去掉就好

@ComponentScan (excludeFilters =  @ComponentScan.Filter(type = FilterType.ASSIGNABLE_TYPE, classes = DeviceBalancerClientConfiguration.class))
avatar
文章
阅读
粉丝