1.背景
前段时间线上有一些超时报警。看了下日志有一些异常堆栈让我非常的困惑,异常现实无法获取到连接,我寻思着lettuce不是共享连接,怎么还有这种情况。
再认真观察了一下日志,发现这些堆栈都和pipeline相关.
两个问题
1.lettuce的pipeline会特殊处理?
2.业务同学使用pipeline主要是为了减少for循环带来的网络IO。如果是lettuce我们是否还需要考虑这一点?
2.看源码
带着两个问题,我翻出了lettuce的源码一探究竟。因为我们项目使用的spring-data-redis。所以我从spring源码开始。
RedisTemplate#execute
@Nullable
public <T> T execute(RedisCallback<T> action, boolean exposeConnection, boolean pipeline) {
RedisConnectionFactory factory = getRequiredConnectionFactory();
RedisConnection conn = RedisConnectionUtils.getConnection(factory, enableTransactionSupport);
try {
boolean existingConnection = TransactionSynchronizationManager.hasResource(factory);
RedisConnection connToUse = preProcessConnection(conn, existingConnection);
RedisConnection connToExpose = (exposeConnection ? connToUse : createRedisConnectionProxy(connToUse));
T result = action.doInRedis(connToExpose);
return postProcessResult(result, connToUse, existingConnection);
} finally {
RedisConnectionUtils.releaseConnection(conn, factory, enableTransactionSupport);
}
}
// 没用的代码都被删掉了
所有redis的操作执行都会调用这个方法。
1.获取工厂。这里就是我们配置的lettuce的工厂。
2.调用RedisConnectionUtils.getConnection方法获取连接。
3.拿到连接调用action.doInRedis。对于pipeline的这个action参考下文executePipelined源码逻辑
我们继续关注获取连接的逻辑.
LettuceConnectionFactory#getConnection
public RedisConnection getConnection() {
assertInitialized();
if (isClusterAware()) {
return getClusterConnection();
}
LettuceConnection connection;
connection = doCreateLettuceConnection(getSharedConnection(), connectionProvider, getTimeout(), getDatabase());
connection.setConvertPipelineAndTxResults(convertPipelineAndTxResults);
return connection;
}
protected StatefulRedisConnection<byte[], byte[]> getSharedConnection() {
return shareNativeConnection && !isClusterAware()
? (StatefulRedisConnection) getOrCreateSharedConnection().getConnection()
: null;
}
protected LettuceConnection doCreateLettuceConnection(
@Nullable StatefulRedisConnection<byte[], byte[]> sharedConnection, LettuceConnectionProvider connectionProvider,
long timeout, int database) {
LettuceConnection connection = new LettuceConnection(sharedConnection, connectionProvider, timeout, database);
connection.setPipeliningFlushPolicy(this.pipeliningFlushPolicy);
return connection;
}
1.调用getSharedConnection获取SharedConnection(第一次会创建)
SharedConnection是LettuceConnectionFactory的一个内部类。他的getConnection可以获取到共享的连接。第一次会调用getNativeConnection创建一个。成功之后这个共享链接会一直被使用。并不会被释放。除非调用LettuceConnectionFactory的destroy方法才会reset此连接。 代码如下
StatefulConnection<E, E> getConnection() {
synchronized (this.connectionMonitor) {
if (this.connection == null) {
this.connection = getNativeConnection();
}
if (getValidateConnection()) {
validateConnection();
}
return this.connection;
}
}
doCreateLettuceConnection
每次执行都会new一个LettuceConnection对象。入参为sharedConnection和一个connectionProvider。这两个对象都保存在LettuceConnectionFactory中。
sharedConnection就是上面的共享连接。connectionProvider使用创建真正连接的Provider。LettuceConnection会将sharedConnection存储到全局变量以备使用。
如果是普通redis请求,直接通过sharedConnection获取对应连接处理即可。但是对于pipeline却非如此。剧透一下,对于pipeline会通过connectionProvider获取一个asyncDedicatedConn。
executePipelined
public List<Object> executePipelined(RedisCallback<?> action, @Nullable RedisSerializer<?> resultSerializer) {
return execute((RedisCallback<List<Object>>) connection -> {
connection.openPipeline();
boolean pipelinedClosed = false;
try {
Object result = action.doInRedis(connection);
if (result != null) {
throw new InvalidDataAccessApiUsageException(
"Callback cannot return a non-null value as it gets overwritten by the pipeline");
}
List<Object> closePipeline = connection.closePipeline();
pipelinedClosed = true;
return deserializeMixedResults(closePipeline, resultSerializer, hashKeySerializer, hashValueSerializer);
} finally {
if (!pipelinedClosed) {
connection.closePipeline();
}
}
});
}
上面的action.doInRedis其实就是这块代码
1.开启connection.openPipeline。
2.调用pipeline的逻辑代码action.doInRedis
3.closePipeline。其实就是聚集返回的结果然后反序列化返回。
LettuceConnection#openPipeline
public void openPipeline() {
if (!isPipelined) {
isPipelined = true;
ppline = new ArrayList<>();
flushState = this.pipeliningFlushPolicy.newPipeline();
flushState.onOpen(this.getOrCreateDedicatedConnection());
}
}
这个LettuceConnection对象就是上面代码new出来的。对象里面有一个SharedConnection。
LettuceConnection#getOrCreateDedicatedConnection
这个方法会调用doGetAsyncDedicatedConnection创建一个StatefulConnection保存在LettuceConnection的asyncDedicatedConn全局变量。具体创建逻辑和共享连接一摸一样。
private StatefulConnection<byte[], byte[]> getOrCreateDedicatedConnection() {
if (asyncDedicatedConn == null) {
asyncDedicatedConn = doGetAsyncDedicatedConnection();
}
return asyncDedicatedConn;
}
protected StatefulConnection<byte[], byte[]> doGetAsyncDedicatedConnection() {
StatefulConnection connection = connectionProvider.getConnection(StatefulConnection.class);
if (customizedDatabaseIndex()) {
potentiallySelectDatabase(dbIndex);
}
return connection;
}
前面都是连接的初始化过程。
命令执行
List list = redisTemplate.executePipelined((RedisCallback<Object>) connection -> {
connection.stringCommands().set("11".getBytes(), "22".getBytes());
return null;
});
最终会调用LettuceStringCommands的set方法
public Boolean set(byte[] key, byte[] value) {
return connection.invoke().from(RedisStringAsyncCommands::set, key, value)
.get(Converters.stringToBooleanConverter());
}
LettuceInvoker invoke() {
return invoke(getAsyncConnection());
}
这里其实包装了一下invoke,通过getAsyncConnection获取连接
RedisClusterAsyncCommands<byte[], byte[]> getAsyncConnection() {
if (isQueueing() || isPipelined()) {
return getAsyncDedicatedConnection();
}
if (asyncSharedConn != null) {
if (asyncSharedConn instanceof StatefulRedisConnection) {
return ((StatefulRedisConnection<byte[], byte[]>) asyncSharedConn).async();
}
if (asyncSharedConn instanceof StatefulRedisClusterConnection) {
return ((StatefulRedisClusterConnection<byte[], byte[]>) asyncSharedConn).async();
}
}
return getAsyncDedicatedConnection();
}
对于pipeline直接获取LettuceConnection的dedicatedConnection返回。其实到这里已经很清楚。
释放连接:RedisConnectionUtils.releaseConnection
会调用LettuceConnection的close方法。
public void close() throws DataAccessException {
super.close();
if (isClosed) {
return;
}
isClosed = true;
if (asyncDedicatedConn != null) {
try {
if (customizedDatabaseIndex()) {
potentiallySelectDatabase(defaultDbIndex);
}
connectionProvider.release(asyncDedicatedConn);
} catch (RuntimeException ex) {
throw convertLettuceAccessException(ex);
}
}
if (subscription != null) {
if (subscription.isAlive()) {
subscription.doClose();
}
subscription = null;
}
this.dbIndex = defaultDbIndex;
}
如果asyncDedicatedConn不为空则release调用连接,对于SharedConnection不进行释放。到这里已经水落石出了。
3.总结
对于普通请求,直接通过SharedConnection共享执行。线程之间共享链接互不影响。
但是对于pipeline,则会从连接池获取。如果没有空闲连接,则会去创建。如果最大连接数太小,就会出现上面的异常。并且只有pipeline执行完成之后链接才会被释放,所以lettuce的pipeline和jedis的模式没什么区别,阻塞式调用。