封装了支持S3协议的文件服务器的分片上传、断点续传、预签名URL上传，理论上minio、阿里云OSS、腾讯云等对象储存都可以

包含的主要功能和示例

• 创建bucket
• 删除bucket
• 文件上传
• 拷贝文件
• 删除文件
• 文件下载
• 设置文件标签
• 上传文件指定时间自动删除
• 上传文件并加密
• 分片上传
• 断点续传
• 生成预签名url，直接前端上传不经过后端

源码地址

源码地址

使用demo地址

demo地址

前置测试环境

首先使用docker-compose安装了最新的minio用于测试

version: '3'

  minio:
    image: minio/minio
    container_name: minio
    restart: always
    ports:
      - "9000:9000" # api端口
      - "9001:9001" # 控制台端口
    environment:
      MINIO_ROOT_USER: "miniouser"      # 设置你的访问账户(用于控制台访问)
      MINIO_ROOT_PASSWORD: "123456789"  # 设置你的访问密钥(用于控制台访问)
    volumes:
      - /mnt/minio/files:/data                   # 将容器中的 /data 文件夹挂载到主机上的指定文件夹 我使用的是/mnt/minio/files
    command: server /data --console-address ":9001"

项目引入依赖

这里说明一下cn.allbs这个group用于jdk1.8，com.alltobs用于jdk17+，但是cn.allbs中有一点点jdk17，遇到的话降级点版本。

<dependency>
  <groupId>com.alltobs</groupId>
  <artifactId>alltobs-oss</artifactId>
  <version>1.0.0</version>
</dependency>

项目配置

配置文件

oss:
  endpoint: http://xxx.xxx.xxx.xxx:9000
  # 所在地区
  region: cn-north-1
  # minio账号或者
  access-key: adadmin
  # 密码
  secret-key: 123456778
  # 设置一个默认的文件桶，比如不同项目使用同一个文件库，以项目为文件桶分隔
  bucket-name: test
  # 设置会过期的子文件桶
  expiring-buckets:
    temp-bucket-1: 30  # 生命周期30天
    temp-bucket-2: 60  # 生命周期60天

• endpoint就是安装minio或者腾讯云、阿里云之类的对象储存地址
• region所在地区
• access-key可以直接使用控制台账号，但是建议在控制台中生成账号和密钥
• secret-key可以直接使用控制台密钥，但是建议在控制台中生成账号和密钥
• bucket-name 设置一个默认的文件桶，比如不同项目使用同一个文件库，以项目为文件桶分隔
• expiring-buckets 里面设置的是包含生命周期的文件夹，创建位置位于bucket-name下，比如bucket-name叫test,那么会在test目录下创建一个生命周期为30天的文件夹temp-bucket-1，生命周期为60天的文件夹temp-bucket-2

启动类注解@EnableAllbsOss

使用引入

@Resource
private OssTemplate ossTemplate;

使用演示

以下所有方法都遵循一个原则，当bucket-name不为空时，所有操作都在这个bucket下进行。

自动创建主目录和带过期时间的子目录

就会自动创建文件桶如下，一个位于test目录下且十天后会自动删除的expire-bucket-1

创建bucket

@PostMapping("/createBucket")  
public ResponseEntity<String> createBucket(@RequestParam String bucketName) {  
    ossTemplate.createBucket(bucketName);  
    return ResponseEntity.ok("Bucket created: " + bucketName);  
}

查询所有bucket

@GetMapping("/getAllBuckets")  
public R<List<String>> getAllBuckets() {  
    return R.ok(ossTemplate.getAllBuckets());  
}

删除指定bucket

@DeleteMapping("/removeBucket")  
public R<String> removeBucket(@RequestParam String bucketName) {  
    ossTemplate.removeBucket(bucketName);  
    return R.ok("Bucket removed: " + bucketName);  
}

上传文件

@PostMapping("/putObject")  
public R<String> putObject(@RequestParam String bucketName, @RequestParam MultipartFile file) {  
    String fileName = file.getOriginalFilename();  
    try {  
        String uuid = UUID.randomUUID() + "." + FileUtil.getFileType(fileName);  
        ossTemplate.putObject(bucketName, uuid, file.getInputStream());  
        return R.ok("File uploaded: " + uuid);  
    } catch (IOException e) {  
        return R.fail("Failed to upload file: " + fileName);  
    }  
}

查询指定目录下指定前缀的文件

@GetMapping("/getAllObjectsByPrefix")  
public R<Set<String>> getAllObjectsByPrefix(@RequestParam String bucketName, @RequestParam String prefix) {   
    return R.ok(ossTemplate.getAllObjectsByPrefix(bucketName, prefix).stream().map(S3Object::key).collect(Collectors.toSet()));  
}

查看文件(字节数组)

@GetMapping("/getObject")  
public R<byte[]> getObject(@RequestParam String bucketName, @RequestParam String objectName) {  
    try (var s3Object = ossTemplate.getObject(bucketName, objectName)) {  
        return R.ok(s3Object.readAllBytes());  
    } catch (IOException e) {  
        return R.fail(e.getLocalizedMessage());  
    }  
}

下载文件

@GetMapping("/download")  
public void download(@RequestParam String bucketName, @RequestParam String objectName, HttpServletResponse response) {  
    try (ResponseInputStream<GetObjectResponse> inputStream = ossTemplate.getObject(bucketName, objectName);  
         OutputStream outputStream = response.getOutputStream()) {  
  
        // 获取文件的Content-Type  
        String contentType = inputStream.response().contentType();  
        response.setContentType(contentType);  
  
        // 设置响应头：Content-Disposition，用于浏览器下载文件时的文件名  
        response.setHeader("Content-Disposition", "attachment; filename=\"" + objectName + "\"");  
  
        // 直接将输入流中的数据传输到输出流  
        inputStream.transferTo(outputStream);  
  
        // 刷新输出流，确保所有数据已写出  
        outputStream.flush();  
  
    } catch (IOException e) {  
        // 如果出错，设置响应状态为404  
        response.setStatus(HttpServletResponse.SC_NOT_FOUND);  
    }  
}

查看文件链接带过期时间

@GetMapping("/getObjectURL")  
public R<String> getObjectURL(@RequestParam String bucketName, @RequestParam String objectName, @RequestParam int minutes) {  
    return R.ok(ossTemplate.getObjectURL(bucketName, objectName, minutes));  
}

删除文件

@DeleteMapping("/removeObject")  
public R<String> removeObject(@RequestParam String bucketName, @RequestParam String objectName) {  
    ossTemplate.removeObject(bucketName, objectName);  
    return R.ok("Object removed: " + objectName);  
}

文件复制

@PostMapping("/copyObject")  
public R<String> copyObject(@RequestParam String sourceBucketName, @RequestParam String sourceKey,  
                            @RequestParam String destinationBucketName, @RequestParam String destinationKey) {  
    ossTemplate.copyObject(sourceBucketName, sourceKey, destinationBucketName, destinationKey);  
    return R.ok("Object copied from " + sourceKey + " to " + destinationKey);  
}

查询指定文件的访问权限

@GetMapping("/getObjectAcl")  
public R<String> getObjectAcl(@RequestParam String bucketName, @RequestParam String objectName) {  
    return R.ok(ossTemplate.getObjectAcl(bucketName, objectName).toString());  
}

设置指定文件的访问权限

@PostMapping("/setObjectAcl")  
public R<String> setObjectAcl(@RequestParam String bucketName, @RequestParam String objectName, @RequestParam String acl) {  
    ossTemplate.setObjectAcl(bucketName, objectName, acl);  
    return R.ok("ACL set for object: " + objectName);  
}

启用或者关闭指定bucket版本控制

@PostMapping("/setBucketVersioning")  
public R<String> setBucketVersioning(@RequestParam String bucketName, @RequestParam boolean enable) {  
    ossTemplate.setBucketVersioning(bucketName, enable);  
    return R.ok("Versioning set to " + (enable ? "Enabled" : "Suspended") + " for bucket: " + bucketName);  
}

给指定文件打标签

@PostMapping("/setObjectTags")  
public R<String> setObjectTags(@RequestParam String bucketName, @RequestParam String objectName, @RequestBody Map<String, String> tags) {  
    ossTemplate.setObjectTags(bucketName, objectName, tags);  
    return R.ok("Tags set for object: " + objectName);  
}

获取指定文件的标签

@GetMapping("/getObjectTags")  
public R<Map<String, String>> getObjectTags(@RequestParam String bucketName, @RequestParam String objectName) {  
    return R.ok(ossTemplate.getObjectTags(bucketName, objectName));  
}

上传一个会定时删除得文件

@PostMapping("putObjectWithExpiration")  
public R<String> putObjectWithExpiration(@RequestParam String bucketName, @RequestParam MultipartFile file, @RequestParam int days) {  
    String fileName = file.getOriginalFilename();  
    try {  
        String uuid = UUID.randomUUID() + "." + FileUtil.getFileType(fileName);  
        ossTemplate.putObjectWithExpiration(bucketName, uuid, file.getInputStream(), days);  
        return R.ok("File uploaded: " + uuid);  
    } catch (IOException e) {  
        return R.fail("Failed to upload file: " + fileName);  
    }  
}

上传加密文件

需要在服务端配置KMS，这里使用得是默认的AES256，其他可以调putObjectWithEncryption方法

@PostMapping("/uploadWithEncryption")  
public R<String> uploadWithEncryption(@RequestParam String bucketName,  
                                      @RequestParam("file") MultipartFile file) {  
    String uuid = UUID.randomUUID() + "." + FileUtil.getFileType(file.getOriginalFilename());  
    try (InputStream inputStream = file.getInputStream()) {  
        PutObjectResponse response = ossTemplate.uploadWithEncryption(  
                bucketName,  
                uuid,                inputStream,                file.getSize(),  
                file.getContentType()  
        );  
        return R.ok("File uploaded with encryption: " + response.toString());  
    } catch (IOException e) {  
        return R.fail("File upload failed: " + e.getMessage());  
    }  
}

分片上传（包含20%概率失败，模仿多线程环境下可能部分分片上传失败的情况）

/**
     * 分片上传
     *
     * @param bucketName 桶名
     * @param file       文件
     * @return R
     */
    @PostMapping("/uploadMultipart")
    public R<String> uploadMultipart(@RequestParam String bucketName,
                                     @RequestParam MultipartFile file) throws IOException, InterruptedException {
        String objectName = file.getOriginalFilename();
        // 初始化分片上传
        String uploadId = ossTemplate.initiateMultipartUpload(bucketName, objectName);
        System.out.println("Upload ID: " + uploadId);

        // 将文件按部分大小（5MB）分块上传
        long partSize = 5 * 1024 * 1024;
        long fileSize = file.getSize();
        int partCount = (int) Math.ceil((double) fileSize / partSize);

        // 用于存储已上传的部分
        List<CompletedPart> completedParts = Collections.synchronizedList(new ArrayList<>());
        List<Integer> failedParts = Collections.synchronizedList(new ArrayList<>());

        // 创建线程池
        ExecutorService executor = Executors.newFixedThreadPool(Math.min(partCount, 10));

        for (int i = 0; i < partCount; i++) {
            final int partNumber = i + 1;
            long startPos = i * partSize;
            long size = Math.min(partSize, fileSize - startPos);

            executor.submit(() -> {
                int retryCount = 0;
                boolean success = false;
//                while (retryCount < 3 && !success) {
                try (InputStream inputStream = file.getInputStream()) {
                    inputStream.skip(startPos);
                    byte[] buffer = new byte[(int) size];
                    int bytesRead = inputStream.read(buffer, 0, (int) size);

                    if (bytesRead > 0) {
                        // 模拟部分上传失败的情况
                        if (Math.random() < 0.2) { // 20%的几率模拟失败
                            throw new IOException("Simulated upload failure for part " + partNumber);
                        }

                        CompletedPart part = ossTemplate.uploadPart(bucketName, objectName, uploadId, partNumber, buffer);
                        completedParts.add(part);
                        success = true;
                    }
                } catch (IOException e) {
                    // 可以在这边加入重试机制,参考下面注释代码，为了确保一定有失败的部分所以注释
//                        retryCount++;
//                        if (retryCount >= 3) {
//                            failedParts.add(partNumber);
//                        }
                    e.printStackTrace();
                }
//                }
            });
        }

        executor.shutdown();
        executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);

        // 检查是否成功上传了所有部分
        if (completedParts.size() == partCount && failedParts.isEmpty()) {
            // 在完成上传之前，按 partNumber 升序排序
            completedParts.sort(Comparator.comparing(CompletedPart::partNumber));

            // 完成分片上传
            ossTemplate.completeMultipartUpload(bucketName, objectName, uploadId, completedParts);
            return R.ok("Upload completed successfully uploadId: " + uploadId);
        } else {
            // 如果有部分上传失败，取消上传,为了测试断点续传所有注释
//            ossTemplate.abortMultipartUpload(bucketName, objectName, uploadId);
            return R.fail("Upload failed, some parts are missing or failed. uploadId: " + uploadId);
        }
    }

断点续传

uploadId是上一步分片上传获取到的，可以做个的记录，方便断点续传时使用。我这边测试方法是分片上传过程中直接终止了服务。

/**
 * 断点续传
 *
 * @param bucketName 桶名
 * @param file       文件
 * @param uploadId   上传 ID
 * @return R
 */
@PostMapping("/resumeMultipart")
public R<String> resumeMultipart(@RequestParam String bucketName,
                                 @RequestParam MultipartFile file,
                                 @RequestParam String uploadId) throws IOException, InterruptedException {
    String objectName = file.getOriginalFilename();

    // 将文件读入内存
    byte[] fileBytes = file.getBytes();

    // 获取已经上传的部分
    List<CompletedPart> completedParts = ossTemplate.listParts(bucketName, objectName, uploadId);

    // 找出已经上传的部分编号
    Set<Integer> uploadedPartNumbers = completedParts.stream()
            .map(CompletedPart::partNumber)
            .collect(Collectors.toSet());

    // 继续上传未完成的部分
    long partSize = 5 * 1024 * 1024;
    long fileSize = fileBytes.length;
    int partCount = (int) Math.ceil((double) fileSize / partSize);

    ExecutorService executor = Executors.newFixedThreadPool(Math.min(partCount - uploadedPartNumbers.size(), 10));

    for (int i = 0; i < partCount; i++) {
        final int partNumber = i + 1;

        // 跳过已上传的部分
        if (uploadedPartNumbers.contains(partNumber)) {
            continue;
        }

        long startPos = i * partSize;
        long size = Math.min(partSize, fileSize - startPos);

        executor.submit(() -> {
            try {
                byte[] buffer = Arrays.copyOfRange(fileBytes, (int) startPos, (int) (startPos + size));
                CompletedPart part = ossTemplate.uploadPart(bucketName, objectName, uploadId, partNumber, buffer);
                synchronized (completedParts) {
                    completedParts.add(part);
                }
            } catch (Exception e) {
                e.printStackTrace();
            }
        });
    }

    executor.shutdown();
    executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);

    // 按 partNumber 升序排序
    completedParts.sort(Comparator.comparing(CompletedPart::partNumber));

    // 完成分片上传
    ossTemplate.completeMultipartUpload(bucketName, objectName, uploadId, completedParts);

    return R.ok("Upload resumed and completed successfully");
}

前端不通过后台服务器使用预签名的表单上传数据

使用这种方式可以让客户端能够直接与 S3 进行交互，减少了服务器的负担，并且可以利用 S3 的上传能力进行大文件的处理。

@GetMapping("/generatePreSignedUrl")  
public R<String> generatePreSignedUrl(@RequestParam String bucketName,  
                                      @RequestParam String objectName,  
                                      @RequestParam int expiration) {  
    String preSignedUrl = ossTemplate.generatePreSignedUrlForPut(bucketName, objectName, expiration);  
    return R.ok(preSignedUrl);  
}

前面的demo

<!DOCTYPE html>  
<html lang="en">  
<head>  
    <meta charset="UTF-8">  
    <meta name="viewport" content="width=device-width, initial-scale=1.0">  
    <title>文件上传</title>  
    <style>        body {  
            font-family: Arial, sans-serif;  
            margin: 20px;  
        }  
  
        .upload-container {  
            max-width: 500px;  
            margin: 0 auto;  
            padding: 20px;  
            border: 2px solid #ccc;  
            border-radius: 10px;  
            text-align: center;  
        }  
  
        .file-input {  
            margin-bottom: 20px;  
        }  
  
        .progress-bar {  
            width: 100%;  
            background-color: #f3f3f3;  
            border-radius: 5px;  
            overflow: hidden;  
            margin-bottom: 10px;  
        }  
  
        .progress {  
            height: 20px;  
            background-color: #4caf50;  
            width: 0;  
        }  
    </style>  
</head>  
<body>  
<div class="upload-container">  
    <h2>上传文件到 S3</h2>  
    <input type="file" id="fileInput" class="file-input"/>  
    <div class="progress-bar">  
        <div class="progress" id="progressBar"></div>  
    </div>  
    <button onclick="uploadFile()">上传文件</button>  
    <p id="statusText"></p>  
</div>  
  
<script>  
    async function uploadFile() {  
        const fileInput = document.getElementById('fileInput');  
        const file = fileInput.files[0];  
  
        if (!file) {  
            alert('请选择一个文件进行上传');  
            return;  
        }  
  
        // 获取预签名 URL        const response = await fetch(`/oss/generatePreSignedUrl?bucketName=myBucket&objectName=${file.name}&expiration=15`);  
        const result = await response.json();  
  
        if (result.code !== 200) {  
            document.getElementById('statusText').innerText = '获取预签名URL失败';  
            return;  
        }  
  
        const presignedUrl = result.data;  // 从返回的 JSON 数据中提取预签名的 URL  
        const xhr = new XMLHttpRequest();  
        xhr.open('PUT', presignedUrl, true);  
        xhr.setRequestHeader('Content-Type', file.type);  
  
        // 更新进度条  
        xhr.upload.onprogress = function(event) {  
            if (event.lengthComputable) {  
                const percentComplete = (event.loaded / event.total) * 100;  
                document.getElementById('progressBar').style.width = percentComplete + '%';  
            }  
        };  
  
        // 处理上传完成后的事件  
        xhr.onload = function() {  
            if (xhr.status === 200) {  
                document.getElementById('statusText').innerText = '文件上传成功！';  
            } else {  
                document.getElementById('statusText').innerText = '文件上传失败，请重试。';  
            }  
        };  
  
        // 错误处理  
        xhr.onerror = function() {  
            document.getElementById('statusText').innerText = '文件上传过程中出现错误。';  
        };  
  
        xhr.send(file);  
    }  
</script>  
</body>  
</html>