小知识,大挑战!本文正在参与“程序员必备小知识”创作活动。
1. 动态配置服务
官方的解释:动态配置服务让您能够以中心化、外部化和动态化的方式管理所有环境的配置。动态配置消除了配置变更时重新部署应用和服务的需要。配置中心化管理让实现无状态服务更简单,也让按需弹性扩展服务更容易。
其实我们在平时业务项目中的应该场景还是挺多的,除了上面的说配置中心化,我们也可以做一些动态配置,比如接口限流,线程池参数的调优,本地缓存一致性等等。
本文主要剖析nacos v1.2.0 源码来分析一下nacos的实现原理。
下面我们从官方的demo入手
public class ConfigExample {
public static void main(String[] args) throws NacosException, InterruptedException {
String serverAddr = "localhost";
String dataId = "test";
String group = "DEFAULT_GROUP";
Properties properties = new Properties();
properties.put("serverAddr", serverAddr);
ConfigService configService = NacosFactory.createConfigService(properties);
String content = configService.getConfig(dataId, group, 5000);
System.out.println(content);
// 客户端添加 Listener 进行监听:
configService.addListener(dataId, group, new Listener() {
@Override
public void receiveConfigInfo(String configInfo) {
System.out.println("receive:" + configInfo);
}
@Override
public Executor getExecutor() {
return null;
}
});
boolean isPublishOk = configService.publishConfig(dataId, group, "content");
System.out.println(isPublishOk);
Thread.sleep(3000);
content = configService.getConfig(dataId, group, 5000);
System.out.println(content);
boolean isRemoveOk = configService.removeConfig(dataId, group);
System.out.println(isRemoveOk);
Thread.sleep(3000);
content = configService.getConfig(dataId, group, 5000);
System.out.println(content);
Thread.sleep(300000);
}
}
2. 配置中心核心 ConfigService
2.1 ConfigFactory.createConfigService(properties)
首先是 ConfigFactory 通过构造函数,反射创建ConfigService对象实例
public static ConfigService createConfigService(Properties properties) throws NacosException {
try {
Class<?> driverImplClass = Class.forName("com.alibaba.nacos.client.config.NacosConfigService");
Constructor constructor = driverImplClass.getConstructor(Properties.class);
ConfigService vendorImpl = (ConfigService) constructor.newInstance(properties);
return vendorImpl;
} catch (Throwable e) {
throw new NacosException(NacosException.CLIENT_INVALID_PARAM, e);
}
}
看下ConfigServer对应的构造函数,主要做了3件事情
- 设置编码方式,默认UTF-8
- 初始化命名空间 initNamespace
- 创建 MetricsHttpAgent Http连接器
- 初始化一个线程池(根据定时器的检查情况决定是否启动其他任务)和一个定时器(定时检查配置信息,ClientWorker 中 new LongPollingRunnable())
public NacosConfigService(Properties properties) throws NacosException {
String encodeTmp = properties.getProperty(PropertyKeyConst.ENCODE);
if (StringUtils.isBlank(encodeTmp)) {
encode = Constants.ENCODE;
} else {
encode = encodeTmp.trim();
}
// 1. 初始化命名空间
initNamespace(properties);
// 2. 创建 ServerHttpAgent,用户登录
agent = new MetricsHttpAgent(new ServerHttpAgent(properties));
// 3. 维护nacos服务列表
agent.start();
// 4. 更新维护配置
worker = new ClientWorker(agent, configFilterChainManager, properties);
}
2.2 new ServerHttpAgent(properties)
ServerHttpAgent,代码中实现了定时任务调度,登录Nacos(客户端获取配置、服务注册列表需要建立连接),时间是5秒一次。
public ServerHttpAgent(Properties properties) throws NacosException {
// 1. 服务列表管理,通过properties解析
serverListMgr = new ServerListManager(properties);
// 2. 用户名,密码路径
securityProxy = new SecurityProxy(properties);
// 3. namespaceId
namespaceId = properties.getProperty(PropertyKeyConst.NAMESPACE);
// 4. 初始化编码方式,secretKey 和maxRetry
init(properties);
// 5. 登录
securityProxy.login(serverListMgr.getServerUrls());
// 线程池
ScheduledExecutorService executorService = new ScheduledThreadPoolExecutor(1, new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setName("com.alibaba.nacos.client.config.security.updater");
t.setDaemon(true);
return t;
}
});
// 定时调度登录,间隔多少时间调用登录接口
executorService.scheduleWithFixedDelay(new Runnable() {
@Override
public void run() {
securityProxy.login(serverListMgr.getServerUrls());
}
}, 0, securityInfoRefreshIntervalMills, TimeUnit.MILLISECONDS);
}
2.3 ServerHttpAgent#start()
调用ServerHttpAgent中的start方法,通过定时任务,维护nacos的列表
public synchronized void start() throws NacosException {
if (isStarted || isFixed) {
return;
}
// runnable接口,维护serverUrlList
GetServerListTask getServersTask = new GetServerListTask(addressServerUrl);
for (int i = 0; i < initServerlistRetryTimes && serverUrls.isEmpty(); ++i) {
// 判断服务列表是否发生改变,如果发生改变,则更新服务列表
getServersTask.run();
try {
this.wait((i + 1) * 100L);
} catch (Exception e) {
LOGGER.warn("get serverlist fail,url: {}", addressServerUrl);
}
}
if (serverUrls.isEmpty()) {
LOGGER.error("[init-serverlist] fail to get NACOS-server serverlist! env: {}, url: {}", name,
addressServerUrl);
throw new NacosException(NacosException.SERVER_ERROR,
"fail to get NACOS-server serverlist! env:" + name + ", not connnect url:" + addressServerUrl);
}
// 将自己在丢到定时任务里面执行,执行时间为30秒一次
TimerService.scheduleWithFixedDelay(getServersTask, 0L, 30L, TimeUnit.SECONDS);
isStarted = true;
}
2.4 核心方法 ClientWorker
核心方法,配置进行同步,缓存, 本地和远程差异化比较。
public ClientWorker(final HttpAgent agent, final ConfigFilterChainManager configFilterChainManager, final Properties properties) {
this.agent = agent;
this.configFilterChainManager = configFilterChainManager;
// Initialize the timeout parameter
init(properties);
// 创建工作线程池,设置为守护线程
executor = Executors.newScheduledThreadPool(1, new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setName("com.alibaba.nacos.client.Worker." + agent.getName());
t.setDaemon(true);
return t;
}
});
// 创建线程池,长轮询
executorService = Executors.newScheduledThreadPool(Runtime.getRuntime().availableProcessors(), new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setName("com.alibaba.nacos.client.Worker.longPolling." + agent.getName());
t.setDaemon(true);
return t;
}
});
executor.scheduleWithFixedDelay(new Runnable() {
@Override
public void run() {
try {
// 检查配置信息
checkConfigInfo();
} catch (Throwable e) {
LOGGER.error("[" + agent.getName() + "] [sub-check] rotate check error", e);
}
}
}, 1L, 10L, TimeUnit.MILLISECONDS);
}
2.5 ClientWorker#checkConfigInfo
public void checkConfigInfo() {
// 分任务
int listenerSize = cacheMap.get().size();
// 向上取整为批数
int longingTaskCount = (int) Math.ceil(listenerSize / ParamUtil.getPerTaskConfigSize());
if (longingTaskCount > currentLongingTaskCount) {
for (int i = (int) currentLongingTaskCount; i < longingTaskCount; i++) {
// 要判断任务是否在执行 这块需要好好想想。 任务列表现在是无序的。变化过程可能有问题
executorService.execute(new LongPollingRunnable(i));
}
currentLongingTaskCount = longingTaskCount;
}
}
防止一次需要更新的太多,时间慢,所以将配置检测的任务分批次执行,分成多个LongPollingRunnable执行,核心代码在LongPollingRunnable中
2.6 LongPollingRunnable#run()
class LongPollingRunnable implements Runnable {
private int taskId;
public LongPollingRunnable(int taskId) {
this.taskId = taskId;
}
@Override
public void run() {
List<CacheData> cacheDatas = new ArrayList<CacheData>();
List<String> inInitializingCacheList = new ArrayList<String>();
try {
// check failover config
for (CacheData cacheData : cacheMap.get().values()) {
if (cacheData.getTaskId() == taskId) {
cacheDatas.add(cacheData);
try {
// 检测本地配置是否发生变化,标记该配置是否可以用本地缓存的
checkLocalConfig(cacheData);
if (cacheData.isUseLocalConfigInfo()) {
cacheData.checkListenerMd5();
}
} catch (Exception e) {
LOGGER.error("get local config info error", e);
}
}
}
// check server config
// 把可能改变的配置,发送到服务器端进行比较,返回修改配置的dataid, groupId
List<String> changedGroupKeys = checkUpdateDataIds(cacheDatas, inInitializingCacheList);
LOGGER.info("get changedGroupKeys:" + changedGroupKeys);
for (String groupKey : changedGroupKeys) {
String[] key = GroupKey.parseKey(groupKey);
String dataId = key[0];
String group = key[1];
String tenant = null;
if (key.length == 3) {
tenant = key[2];
}
try {
// 从服务端获取更新的key的值, 并进行缓存,生成文件快照
String[] ct = getServerConfig(dataId, group, tenant, 3000L);
CacheData cache = cacheMap.get().get(GroupKey.getKeyTenant(dataId, group, tenant));
cache.setContent(ct[0]);
if (null != ct[1]) {
cache.setType(ct[1]);
}
LOGGER.info("[{}] [data-received] dataId={}, group={}, tenant={}, md5={}, content={}, type={}",
agent.getName(), dataId, group, tenant, cache.getMd5(),
ContentUtils.truncateContent(ct[0]), ct[1]);
} catch (NacosException ioe) {
String message = String.format(
"[%s] [get-update] get changed config exception. dataId=%s, group=%s, tenant=%s",
agent.getName(), dataId, group, tenant);
LOGGER.error(message, ioe);
}
}
// 检查新的值,并设置是否首次更新和初始化状态
for (CacheData cacheData : cacheDatas) {
// cacheData 首次出现在cacheMap中&首次check更新
if (!cacheData.isInitializing() || inInitializingCacheList
.contains(GroupKey.getKeyTenant(cacheData.dataId, cacheData.group, cacheData.tenant))) {
cacheData.checkListenerMd5();
cacheData.setInitializing(false);
}
}
inInitializingCacheList.clear();
// 递归
executorService.execute(this);
} catch (Throwable e) {
// If the rotation training task is abnormal, the next execution time of the task will be punished
LOGGER.error("longPolling error : ", e);
executorService.schedule(this, taskPenaltyTime, TimeUnit.MILLISECONDS);
}
}
}