soul源码研究-http长轮询同步数据

今天我们研究下soul 的http长轮询同步数据到网关这块。

1、启动soul-admin

配置 application.xml 配置文件，把websocket、zookeeper 关闭，打开 http。

soul:
  database:
    dialect: mysql
    init_script: "META-INF/schema.sql"
    init_enable: true
  sync:
    websocket:
      enabled: false
#    zookeeper:
#      enabled: true
#      url: localhost:2181
#      sessionTimeout: 5000
#      connectionTimeout: 2000
    http:
      enabled: true

启动 SoulAdminBootstrap。

2、启动soul-bootstrap

配置application-local.yml，把 dubbo、websocket、zookeeper 关闭，把http 打开

soul :
    file:
      enabled: true
    corss:
      enabled: true
#    dubbo :
#      parameter: multi
    sync:
#        websocket :
#             urls: ws://localhost:9095/websocket

#        zookeeper:
#             url: localhost:2181
#             sessionTimeout: 5000
#             connectionTimeout: 2000
        http:
             url : http://localhost:9095

启动SoulBootstrapApplication。

源码解析：

1、分析soul-admin的http源码部分
1. 通过前两篇文章的分析，我们从sole-admin的application.xml的配置文件分析，配置文件开启了http长轮询方式同步数据。我们全文搜索 soul.sync.http，发现了DataSyncConfiguration类会加载application.xml的http配置信息，并执行HttpLongPollingDataChangedListener的构造函数。同时会进行AbstractDataChangedListener的afterPropertiesSet，在AbstractDataChangedListener初始化完成后执行，afterPropertiesSet方法的作用就是从DB查询AppAuth、Rlugin、Rule、Selector、Metadata数据并初始化到本地缓存(AbstractDataChangedListener的CACHE)，然后执行HttpLongPollingDataChangedListener的afterInitialize方法，开启定时任务，每5分钟执行一次，把AppAuth、Rlugin、Rule、Selector、Metadata 的从DB查询并缓存到本地缓存(AbstractDataChangedListener的CACHE)。
  
  这些操作执行完成后，还会执行DataChangedEventDispatcher的afterPropertiesSet方法，初始化listeners数据，进行数据变更事件的监听。

@Configuration
public class DataSyncConfiguration {

    /**
     * http long polling.
     */
    @Configuration
    @ConditionalOnProperty(name = "soul.sync.http.enabled", havingValue = "true")
    @EnableConfigurationProperties(HttpSyncProperties.class)
    static class HttpLongPollingListener {

        @Bean
        @ConditionalOnMissingBean(HttpLongPollingDataChangedListener.class)
        public HttpLongPollingDataChangedListener httpLongPollingDataChangedListener(final HttpSyncProperties httpSyncProperties) {
            return new HttpLongPollingDataChangedListener(httpSyncProperties);
        }

    }
    、、、
}

@Slf4j
@SuppressWarnings("all")
public class HttpLongPollingDataChangedListener extends AbstractDataChangedListener {

    private static final String X_REAL_IP = "X-Real-IP";

    private static final String X_FORWARDED_FOR = "X-Forwarded-For";

    private static final String X_FORWARDED_FOR_SPLIT_SYMBOL = ",";

    private static final ReentrantLock LOCK = new ReentrantLock();

    /**
     * Blocked client.
     */
    private final BlockingQueue<LongPollingClient> clients;

    private final ScheduledExecutorService scheduler;

    private final HttpSyncProperties httpSyncProperties;

    /**
     * Instantiates a new Http long polling data changed listener.
     * @param httpSyncProperties the HttpSyncProperties
     */
    public HttpLongPollingDataChangedListener(final HttpSyncProperties httpSyncProperties) {
        this.clients = new ArrayBlockingQueue<>(1024);
        this.scheduler = new ScheduledThreadPoolExecutor(1,
                SoulThreadFactory.create("long-polling", true));
        this.httpSyncProperties = httpSyncProperties;
    }

		/**
		* HttpLongPollingDataChangedListener 初始化完成后执行
		*/
    @Override
    protected void afterInitialize() {
        long syncInterval = httpSyncProperties.getRefreshInterval().toMillis();
        // Periodically check the data for changes and update the cache
        scheduler.scheduleWithFixedDelay(() -> {
            log.info("http sync strategy refresh config start.");
            try {
                this.refreshLocalCache();
                log.info("http sync strategy refresh config success.");
            } catch (Exception e) {
                log.error("http sync strategy refresh config error!", e);
            }
        }, syncInterval, syncInterval, TimeUnit.MILLISECONDS);
        log.info("http sync strategy refresh interval: {}ms", syncInterval);
    }

		/**
		* 把AppAuth、Rlugin、Rule、Selector、Metadata 的从DB查询并缓存到本地缓存(AbstractDataChangedListener的CACHE)
		*/
    private void refreshLocalCache() {
        this.updateAppAuthCache();
        this.updatePluginCache();
        this.updateRuleCache();
        this.updateSelectorCache();
        this.updateMetaDataCache();
    }
    、、、
}

@Slf4j
@SuppressWarnings("all")
public abstract class AbstractDataChangedListener implements DataChangedListener, InitializingBean {
		、、、
		@Override
    public final void afterPropertiesSet() {
        updateAppAuthCache();
        updatePluginCache();
        updateRuleCache();
        updateSelectorCache();
        updateMetaDataCache();
        afterInitialize();
    }
    、、、
}

protected <T> void updateCache(final ConfigGroupEnum group, final List<T> data) {
    String json = GsonUtils.getInstance().toJson(data);
    ConfigDataCache newVal = new ConfigDataCache(group.name(), json, Md5Utils.md5(json), System.currentTimeMillis());
    ConfigDataCache oldVal = CACHE.put(newVal.getGroup(), newVal);
    log.info("update config cache[{}], old: {}, updated: {}", group, oldVal, newVal);
}

@Component
public class DataChangedEventDispatcher implements ApplicationListener<DataChangedEvent>, InitializingBean {

    private ApplicationContext applicationContext;

    private List<DataChangedListener> listeners;

    public DataChangedEventDispatcher(final ApplicationContext applicationContext) {
        this.applicationContext = applicationContext;
    }

    @Override
    @SuppressWarnings("unchecked")
    public void onApplicationEvent(final DataChangedEvent event) {
        for (DataChangedListener listener : listeners) {
            switch (event.getGroupKey()) {
                case APP_AUTH:
                    listener.onAppAuthChanged((List<AppAuthData>) event.getSource(), event.getEventType());
                    break;
                case PLUGIN:
                    listener.onPluginChanged((List<PluginData>) event.getSource(), event.getEventType());
                    break;
                case RULE:
                    listener.onRuleChanged((List<RuleData>) event.getSource(), event.getEventType());
                    break;
                case SELECTOR:
                    listener.onSelectorChanged((List<SelectorData>) event.getSource(), event.getEventType());
                    break;
                case META_DATA:
                    listener.onMetaDataChanged((List<MetaData>) event.getSource(), event.getEventType());
                    break;
                default:
                    throw new IllegalStateException("Unexpected value: " + event.getGroupKey());
            }
        }
    }

    @Override
    public void afterPropertiesSet() {
        Collection<DataChangedListener> listenerBeans = applicationContext.getBeansOfType(DataChangedListener.class).values();
        this.listeners = Collections.unmodifiableList(new ArrayList<>(listenerBeans));
    }

我们进行debug验证下，启动soul-admin。发现项目启动时DataSyncConfiguration类会加载application.xml的http配置信息，并执行HttpLongPollingDataChangedListener的构造函数。同时会进行AbstractDataChangedListener的afterPropertiesSet，在AbstractDataChangedListener初始化完成后执行。最后会执行DataChangedEventDispatcher的afterPropertiesSet方法，初始化listeners数据，进行数据变更事件的监听。

2、分析soul-bootstrap的http源码部分

从soul-bootstrap的配置文件application-local.yml 新增soul.sync.http.url 配置获悉，我们需要查询soul.sync.http.url引用的地方，找到了HttpSyncDataConfiguration。HttpSyncDataConfiguration会执行SyncDataService的构造函数。刷新soul-admin、soul-bootstrap 端的APP_AUTH、PLUGIN、RULE、SELECTOR、META_DATA缓存。

@Slf4j
public class HttpSyncDataService implements SyncDataService, AutoCloseable {

    private static final AtomicBoolean RUNNING = new AtomicBoolean(false);

    private static final Gson GSON = new Gson();

    /**
     * default: 10s.
     */
    private Duration connectionTimeout = Duration.ofSeconds(10);

    /**
     * only use for http long polling.
     */
    private RestTemplate httpClient;

    private ExecutorService executor;

    private HttpConfig httpConfig;

    private List<String> serverList;

    private DataRefreshFactory factory;

    public HttpSyncDataService(final HttpConfig httpConfig, final PluginDataSubscriber pluginDataSubscriber,
                               final List<MetaDataSubscriber> metaDataSubscribers, final List<AuthDataSubscriber> authDataSubscribers) {
        this.factory = new DataRefreshFactory(pluginDataSubscriber, metaDataSubscribers, authDataSubscribers);
        this.httpConfig = httpConfig;
        this.serverList = Lists.newArrayList(Splitter.on(",").split(httpConfig.getUrl()));
        this.httpClient = createRestTemplate();
        this.start();
    }
    
    private RestTemplate createRestTemplate() {
        OkHttp3ClientHttpRequestFactory factory = new OkHttp3ClientHttpRequestFactory();
        factory.setConnectTimeout((int) this.connectionTimeout.toMillis());
        factory.setReadTimeout((int) HttpConstants.CLIENT_POLLING_READ_TIMEOUT);
        return new RestTemplate(factory);
    }

		/**
		* 当RUNNING 为false时， 设置为true， 并执行里面的方法
		*/
    private void start() {
        // It could be initialized multiple times, so you need to control that.
        if (RUNNING.compareAndSet(false, true)) {
            // fetch all group configs. 缓存APP_AUTH、PLUGIN、RULE、SELECTOR、META_DATA数据到本地缓存
            this.fetchGroupConfig(ConfigGroupEnum.values());
            int threadSize = serverList.size();
            this.executor = new ThreadPoolExecutor(threadSize, threadSize, 60L, TimeUnit.SECONDS,
                    new LinkedBlockingQueue<>(),
                    SoulThreadFactory.create("http-long-polling", true));
            // start long polling, each server creates a thread to listen for changes.
            this.serverList.forEach(server -> this.executor.execute(new HttpLongPollingTask(server)));
        } else {
            log.info("soul http long polling was started, executor=[{}]", executor);
        }
    }
    
    class HttpLongPollingTask implements Runnable {

        private String server;

        private final int retryTimes = 3;

        HttpLongPollingTask(final String server) {
            this.server = server;
        }

        @Override
        public void run() {
        		// 当RUNNING为true时，重试3次，刷新soul-admin、soul-bootstrap 端的APP_AUTH、PLUGIN、RULE、SELECTOR、META_DATA缓存。 只有当soul-bootstrap关闭时，RUNNING才为false。 并开启死循环，实时检测soul-admin进行的相关变动，保存到soul-bootstrap本地缓存。
            while (RUNNING.get()) {
                for (int time = 1; time <= retryTimes; time++) {
                    try {
                        doLongPolling(server);
                    } catch (Exception e) {
                        // print warnning log.
                        if (time < retryTimes) {
                            log.warn("Long polling failed, tried {} times, {} times left, will be suspended for a while! {}",
                                    time, retryTimes - time, e.getMessage());
                            ThreadUtils.sleep(TimeUnit.SECONDS, 5);
                            continue;
                        }
                        // print error, then suspended for a while.
                        log.error("Long polling failed, try again after 5 minutes!", e);
                        ThreadUtils.sleep(TimeUnit.MINUTES, 5);
                    }
                }
            }
            log.warn("Stop http long polling.");
        }
    }
    
    private void doLongPolling(final String server) {
        MultiValueMap<String, String> params = new LinkedMultiValueMap<>(8);
        for (ConfigGroupEnum group : ConfigGroupEnum.values()) {
            ConfigData<?> cacheConfig = factory.cacheConfigData(group);
            String value = String.join(",", cacheConfig.getMd5(), String.valueOf(cacheConfig.getLastModifyTime()));
            params.put(group.name(), Lists.newArrayList(value));
        }
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
        HttpEntity httpEntity = new HttpEntity(params, headers);
        String listenerUrl = server + "/configs/listener";
        log.debug("request listener configs: [{}]", listenerUrl);
        JsonArray groupJson = null;
        try {
            String json = this.httpClient.postForEntity(listenerUrl, httpEntity, String.class).getBody();
            log.debug("listener result: [{}]", json);
            groupJson = GSON.fromJson(json, JsonObject.class).getAsJsonArray("data");
        } catch (RestClientException e) {
            String message = String.format("listener configs fail, server:[%s], %s", server, e.getMessage());
            throw new SoulException(message, e);
        }
        if (groupJson != null) {
            // fetch group configuration async.
            ConfigGroupEnum[] changedGroups = GSON.fromJson(groupJson, ConfigGroupEnum[].class);
            if (ArrayUtils.isNotEmpty(changedGroups)) {
                log.info("Group config changed: {}", Arrays.toString(changedGroups));
                this.doFetchGroupConfig(server, changedGroups);
            }
        }
    }

启动soul-bootstrap，进行debug验证。

总结

至此，我们清楚了http长轮询的使用情况，本地也进行了多次的sole-bootstrap 和 soul-admin 的Plugin(插件)、SelectorList(选择器)、RulesList(选择器配置规则时)变更，情况如同上面我们看的一样。soul-admin进行AppAuth、Rlugin、Rule、Selector、Metadata的缓存初始化，并开启定时任务每5分钟执行一次进行AppAuth、Rlugin、Rule、Selector、Metadata的缓存更新，还会开启监听，监听AppAuth、Rlugin、Rule、Selector、Metadata的变更事件。 Soul-bootstrap 会加载对应http配置，第一次执行设置RUNNING为true，当RUNNING为true时会一直循环刷新soul-admin、soul-bootstrap 的AppAuth、Rlugin、Rule、Selector、Metadata缓存。soul-bootstrap 开启的死循环会在项目关闭时更新RUNNING为false。

soul源码研究-http长轮询同步数据

1、启动soul-admin

2、启动soul-bootstrap

源码解析：

1、 分析soul-admin的http源码部分

2、 分析soul-bootstrap的http源码部分

总结

1、分析soul-admin的http源码部分

2、分析soul-bootstrap的http源码部分