问题
隔壁新建的开发组发生了一个bug,文件上传接口,上传2G大小文件时,文件被截断为2G整。
分析
因为配置的是MinioClient,原本设置地上传文件大小为100m,后来有需求要增加文件大小,来不及开发切片上传,想要临时更改配置撑过用户使用阶段,所以更改了服务器文件大小限制、nginx文件大小限制,结果发现大于2G的文件被截断为2G。自己手动存到minio的文件不受影响,故开始排查
1.nginx文件大小限制
Nginx 限制文件大小可以通过 client_max_body_size 指令来设置,该指令通常在 http、server 或
location 块中设置,如果不设置,默认上传大小为1M。
client_max_body_size 10M;
2.springboot文件大小限制
multipart参数接收文件的大小限制
spring.servlet.multipart.max-file-size=50MB
spring.servlet.multipart.max-request-size=50MB
3.minio文件大小限制
minio是不限制文件大小的
结果
经过排查,发现文件的大小在上传minio的时候被固定为2G,原因在于
try {
this.client.putObject((PutObjectArgs)((PutObjectArgs.Builder)((PutObjectArgs.Builder)PutObjectArgs.builder().bucket(this.getBucketName(bucketName))).object(fileName)).stream(stream, (long)stream.available(), -1L).contentType(contentType).build());
} catch (Exception var16) {
var16.printStackTrace();
} finally {
try {
if (stream != null) {
stream.close();
}
} catch (Exception var15) {
var15.printStackTrace();
}
}
上述代码中的(long)stream.available(),源码为
/**
* Returns an estimate of the number of bytes that can be read (or
* skipped over) from this input stream without blocking by the next
* invocation of a method for this input stream. The next invocation
* might be the same thread or another thread. A single read or skip of this
* many bytes will not block, but may read or skip fewer bytes.
*
* <p> Note that while some implementations of {@code InputStream} will return
* the total number of bytes in the stream, many will not. It is
* never correct to use the return value of this method to allocate
* a buffer intended to hold all data in this stream.
*
* <p> A subclass' implementation of this method may choose to throw an
* {@link IOException} if this input stream has been closed by
* invoking the {@link #close()} method.
*
* <p> The {@code available} method for class {@code InputStream} always
* returns {@code 0}.
*
* <p> This method should be overridden by subclasses.
*
* @return an estimate of the number of bytes that can be read (or skipped
* over) from this input stream without blocking or {@code 0} when
* it reaches the end of the input stream.
* @exception IOException if an I/O error occurs.
*/
public int available() throws IOException {
return 0;
}
可以看到方法返回的值为int,然后强转为long当作文件大小去上传参数使用,那么int的最大值是等于2G的,问题就在这里
210241024*1024 = 2,147,483,648 > int范围