背景
关于httpclient的三个超时参数，之前都有接触过，但还是停留在概念层面。最近恰好遇到个相关的问题，顺便做个验证，加深对这三个参数的理解。
简单描述下遇到的问题：项目某个调用依赖，测试环境未出现异常，但是上线偶现，实质上就是httpclient的超时设置线下、线上存在差别。
说明
示例使用java开发，引入的httpclient是apache版本。具体如下。
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.4.4</version>
</dependency>
准备工作
先准备一个用来测试的client，后续实验中会修改对应的超时参数。还需要一个测试例子。而测试的地址，使用了苹果开发获取公钥的https://appleid.apple.com/auth/keys。具体原因是这个地址访问稳定，而且耗时较长，有正常的返回。下面是本地ping的结果，可以看到基本稳定在210ms左右，所以非常适合用来做超时验证。
$ ping appleid.apple.com
PING appleid.apple.com.akadns.net (17.141.5.102) 56(84) bytes of data.
64 bytes from appleid-prn-s.apple.com (17.141.5.102): icmp_seq=1 ttl=41 time=210 ms
64 bytes from appleid-prn-s.apple.com (17.141.5.102): icmp_seq=2 ttl=41 time=211 ms
64 bytes from appleid-prn-s.apple.com (17.141.5.102): icmp_seq=3 ttl=41 time=210 ms
64 bytes from appleid-prn-s.apple.com (17.141.5.102): icmp_seq=4 ttl=41 time=210 ms
64 bytes from appleid-prn-s.apple.com (17.141.5.102): icmp_seq=5 ttl=41 time=210 ms
64 bytes from appleid-prn-s.apple.com (17.141.5.102): icmp_seq=6 ttl=41 time=210 ms
64 bytes from appleid-prn-s.apple.com (17.141.5.102): icmp_seq=7 ttl=41 time=210 ms
- 测试代码
import org.junit.Test;
import java.util.concurrent.TimeUnit;
public class HttpClientTest {
String url = "https://appleid.apple.com/auth/keys";
@Test
public void runTest() {
for (int i = 0; i < 20; i++) {
String msg = "";
try {
HttpClientUtil.get(url);
TimeUnit.SECONDS.sleep(1); //避免被限流，隔一秒请求!!
} catch (Exception e) {
msg = e.getMessage();
}
System.out.println(String.format("id: %s\texp:%s", i, msg));
}
}
}
- client
import org.apache.http.HttpStatus;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.util.EntityUtils;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
public class HttpClientUtil {
private static PoolingHttpClientConnectionManager cm;
private static RequestConfig requestConfig;
static {
cm = new PoolingHttpClientConnectionManager();
cm.setMaxTotal(1);
cm.setDefaultMaxPerRoute(2);
requestConfig = RequestConfig.custom()
//修改这几个参数来验证
.setConnectTimeout(10000)
.setConnectionRequestTimeout(1)
.setSocketTimeout(10000)
.build();
}
private static CloseableHttpClient getClient() {
return HttpClients.custom().setConnectionManager(cm).setDefaultRequestConfig(requestConfig).build();
}
public static String get(String url) throws IOException {
CloseableHttpClient httpClient = getClient();
HttpGet httpGet = new HttpGet(url);
CloseableHttpResponse response = httpClient.execute(httpGet);
if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
if (response.getEntity() != null) {
return EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
}
}
return String.format("code:%s", response.getStatusLine().getStatusCode());
}
}
验证
思路是分别设置不同的参数值，保证每次只有一种情况出现超时，再通过具体的报错信息和堆栈，理清实际的超时判断逻辑。 以下分别是三个参数的验证过程
connectTimeout
设置该值为远小于ping时长的值，运行试例后得到验证结果，全部是 org.apache.http.conn.ConnectTimeoutException: Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out。通过堆栈和异常结果，可知这个参数用来控制建立连接的超时。
requestConfig = RequestConfig.custom()
.setConnectTimeout(100) //100ms < 210ms
.setConnectionRequestTimeout(10000)
.setSocketTimeout(10000)
.build();
结果为全部connect timed out
id: 0 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 1 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 2 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 3 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 4 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 5 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 6 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 7 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 8 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 9 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 10 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 11 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 12 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 13 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 14 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 15 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 16 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 17 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 18 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
id: 19 exp:Connect to appleid.apple.com:443 [appleid.apple.com/17.141.5.102] failed: connect timed out
socketTimeout
将该值设置为100，其他较高。 报错全部为, java.net.SocketTimeoutException: Read timed out，debug从堆栈看出超时在读取阶段。
requestConfig = RequestConfig.custom()
.setConnectTimeout(10000)
.setConnectionRequestTimeout(10000)
.setSocketTimeout(100)
.build();
再将socketTimeout 设置为220，可以看到是部分失败。
requestConfig = RequestConfig.custom()
.setConnectTimeout(10000)
.setConnectionRequestTimeout(10000)
.setSocketTimeout(220)
.build();
结果：
id: 0 exp:
id: 1 exp:
id: 2 exp:Read timed out
id: 3 exp:Read timed out
id: 4 exp:
id: 5 exp:Read timed out
id: 6 exp:Read timed out
id: 7 exp:Read timed out
id: 8 exp:Read timed out
id: 9 exp:Read timed out
id: 10 exp:
id: 11 exp:
id: 12 exp:Read timed out
id: 13 exp:
id: 14 exp:
id: 15 exp:
id: 16 exp:
id: 17 exp:
id: 18 exp:
id: 19 exp:
connectionRequestTimeout
将该参数设置为如下的值:
requestConfig = RequestConfig.custom()
.setConnectTimeout(10000)
.setConnectionRequestTimeout(1)
.setSocketTimeout(10000)
.build();
如果还是上面的测试代码，是没有问题可以正常运行的。原因是虽然连接池中只有一个链接，但是依次运行，连接用完就释放了，不会抛出connectionRequestTimeout的异常。所以这次需要改下测试代码，如下：
String url = "https://appleid.apple.com/auth/keys";
@Test
public void runTest() throws InterruptedException {
for (int i = 0; i < 20; i++) {
String msg = "";
try {
// TimeUnit.SECONDS.sleep(1); //避免被限流，隔一秒请求!!
new Thread(new Runnable() {
@Override
public void run() {
try {
HttpClientUtil.get(url);
} catch (Exception e) {
e.printStackTrace();
}
}
}).start();
} catch (Exception e) {
msg = e.getMessage();
}
System.out.println(String.format("id: %s\texp:%s", i, msg));
}
Thread.currentThread().join();
}
放在多线程里，让产生竞争态，一个连接没有使用完毕，再次尝试从连接池获取的时候，就会超过设置的ConnectionRequestTimeout时间，所以这个参数控制从连接池获取连接。再查看堆栈，确认了结论是正确的。
final int timeout = config.getConnectionRequestTimeout();
managedConn = connRequest.get(timeout > 0 ? timeout : 0, TimeUnit.MILLISECONDS);
错误堆栈:
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:286)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
总结
以上就是所有的验证过程，最后说下结论：
- ConnectTimeout 控制建立连接，超时抛出
org.apache.http.conn.ConnectTimeoutException异常。 - ConnectionRequestTimeout 控制从连接池获取连接，超时抛出
org.apache.http.conn.ConnectionPoolTimeoutException异常。 - SocketTimeout 控制从连接读取数据，超时抛出
java.net.SocketTimeoutException异常。
具体的其实可以在对应代码的注释中得到，试验会更直观和方便理解。
/**
* Returns the timeout in milliseconds used when requesting a connection
* from the connection manager. A timeout value of zero is interpreted
* as an infinite timeout.
* <p>
* A timeout value of zero is interpreted as an infinite timeout.
* A negative value is interpreted as undefined (system default).
* </p>
* <p>
* Default: {@code -1}
* </p>
*/
public int getConnectionRequestTimeout() {
return connectionRequestTimeout;
}
/**
* Determines the timeout in milliseconds until a connection is established.
* A timeout value of zero is interpreted as an infinite timeout.
* <p>
* A timeout value of zero is interpreted as an infinite timeout.
* A negative value is interpreted as undefined (system default).
* </p>
* <p>
* Default: {@code -1}
* </p>
*/
public int getConnectTimeout() {
return connectTimeout;
}
/**
* Defines the socket timeout ({@code SO_TIMEOUT}) in milliseconds,
* which is the timeout for waiting for data or, put differently,
* a maximum period inactivity between two consecutive data packets).
* <p>
* A timeout value of zero is interpreted as an infinite timeout.
* A negative value is interpreted as undefined (system default).
* </p>
* <p>
* Default: {@code -1}
* </p>
*/
public int getSocketTimeout() {
return socketTimeout;
}
