Springboot 集成Minio

拂晓银砾
1,426 阅读7分钟

Minio概念

MinIO is an object storage solution that provides an Amazon Web Services S3-compatible API and supports all core S3 features. MinIO is built to deploy anywhere - public or private cloud, baremetal infrastructure, orchestrated environments, and edge infrastructure.

Minio搭建

在linux服务器上执行

mkdir -p /blessing/minio/data
mkdir -p /blessing/minio/config
docker run -dit  \
   --restart always \
   -p 9000:9000 \
   -p 9090:9090 \
   --name minio \
   -v  /blessing/minio/data:/data \
   -v  /blessing/minio/config:/root/.minio \
   -e "MINIO_ROOT_USER=DawnSilverGravel" \
   -e "MINIO_ROOT_PASSWORD=DawnSilverGravel" \
   quay.io/minio/minio:RELEASE.2023-06-02T23-17-26Z.fips  server /data --console-address ":9090" --address ":9000"

使用docker启动参数具体在参考文档的第二个链接，此时RedHat Mminio镜像仓库没有最新的latest，故使用RELEASE.2023-06-02T23-17-26Z.fips版本。

如果没有docker,同样在参考文档的第二链接，然后点击Linux标签页找到安装命令，在Minio版本可以找到对应的版本，以下是下载minio二进制的命令。

mkdir -p /blessing/minio/data
# 查看liunx系统架构，如果是x86_64就是AMD的
arch
# 下载minio二进制文件
wget https://dl.min.io/server/minio/release/linux-amd64/minio -P /blessing/minio
# 赋予权限
cd /blessing/minio
chmod +x minio
# 启动minio --console-address控制台端口，默认是9001，默认上传端口是9000
# 账号密码默认：minioadmin/minioadmin
MINIO_ROOT_USER=DawnSilverGravel MINIO_ROOT_PASSWORD=DawnSilverGravel ./minio server /blessing/minio/data --console-address ":9090"

启动之后在浏览器访问http://yourAddress:9090 在Access Keys页面生成密钥得到access-key和secret-key。

image.png

Minio使用

Java API

在官方的文档中（下方参考文档第一个链接），以及MinioClient类中，和下方参考文档的第三个链接中，都可以找到相关方法的示例,这些方法参数都有一个共同的特点，使用了Builder模式，从中也使用了Consumer<T>函数式接口，可以学习一下其思想。

bucket method描述
bucketExists检查bucket是否存在
listBuckets获取所有bucket的信息
makeBucket创建给定区域和对象锁特性的bucket
removeBucket删除一个无数据的bucket
getBucketTags获取bucket标签
setBucketTags设置bucket标签，之前的tags会被删除
deleteBucketTags删除bucket标签
getBucketVersioning获取bucket版本
setBucketVersioning设置bucket版本(版本默认是无，设置了之后就只能配置Enabled和Suspended)
getObjectLockConfigurationArgs获取bucket的对象锁配置，makeBucket新建bucketobjectLock参数为true
setObjectLockConfigurationArgs设置bucket对象锁配置,makeBucket新建bucketobjectLock参数为true，限制对对象的修改
deleteObjectLockConfigurationArgs删除bucket对象锁配置makeBucket新建bucketobjectLock参数为true
......其余暂不了解
object method描述
putObject以流的形式上传对象
uploadObject以文件的形式上传对象
copyObject创建一个对象并复制minio服务器上的另一个对象的数据，两个对象不能相同
composeObject组合minio服务器端不同的源对象来创建一个对象,每个对象大小不可以低于5M
uploadSnowballObjects在单个put中上传多个文件，SnowballObject中size与指定的流大小要一致
removeObject删除一个对象
removeObjects删除多个对象
statObject获取一个对象的对象信息和元数据
getObject获取对象数据，并返回InputStream流；，该流必须关闭以释放网络资源
getPresignedObjectUrl获取一个对象的http方法、过期时间、自定义参数的URL，GetPresignedObjectUrlArgs有GET、PUT、HEAD三种类型的http方法
downloadObject以本地文件的形式下载一个对象，DownloadObjectArgs中的fileName中文件名如果本地存在将会下载失败
getPresignedPostFormData获取一个对象PostPolicy的表单数据的POST方法URL,然后使用该URL上传文件
listObjects获取一个bucket里对象列表
getObjectTags获取对象的标签
setObjectTags设置对象标签
deleteObjectTags删除对象标签
getObjectRetention获取对象保留策略配置,指定创建bucket创建时需要设置objectLock为true
setObjectRetention设置对象保留策略配置,指定创建bucket创建时需要设置objectLock为true
enableObjectLegalHold启用对对象的合法保留,默认为false，上传对象时可设置，指定bucket创建时要设置objectLock为true
disableObjectLegalHold禁用对对象的合法保留,指定bucket创建时设置objectLock为true
isObjectLegalHoldEnabled返回对象是否启用合法保留，启用为true,指定bucket创建时要设置objectLock为true
selectObjectContent通过SQL表达式选择对象的内容（不知如何使用）
restoreObject归档一个对象（不知如何使用）

以PutObjectArgs为例：

minio没有文件夹的概念，但是可以使用object(/blessing/yourFileName)可以创建一个blessing文件夹

针对于putObject方法中的参数对象PutObjectArgs 中的stream方法有其中两个参数objectSize与partSize,有以下描述：

  1. 允许最大对象为5TB
  2. 允许最小的分片为5M
  3. 允许最大分片为5GB（这里注释看了一下代码应该是写错了，版本8.5.3）
  4. 最大的分片数量为10000

image.png

从图中可以得知这是一个父类，则下图中的子类同样有上述规则。

image.png

项目结构与配置

项目结构

image.png

pom.xml

<dependencies>
    <dependency>
        <groupId>io.minio</groupId>
        <artifactId>minio</artifactId>
        <version>8.5.3</version>
    </dependency>
    <dependency>
        <groupId>cn.hutool</groupId>
        <artifactId>hutool-all</artifactId>
        <version>5.8.15</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.projectlombok/lombok -->
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>1.18.26</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-configuration-processor</artifactId>
        <optional>true</optional>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
        <version>2.7.12</version>
    </dependency>
</dependencies>

application.yml

minio:
  init-buckets:
    - dawn-silver-gravel
    - blessing-star
  operation-bucket: dawn-star
  access-key: yourAccessKey
  secret-key: yourSecretKey
  # 默认端口9000
  endpoint: http://yourAddress:yourPort

config配置

package com.example.config;

import io.minio.MinioClient;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import javax.annotation.Resource;

/**
 * description:
 *
 * @author DawnStar
 * date: 2023/6/14
 */
@Configuration
@EnableConfigurationProperties(MinioProperties.class)
public class MinioConfiguration {
    @Resource
    private MinioProperties minioProperties;

    @Bean
    public MinioClient minioClient() {
        return MinioClient.builder().credentials(minioProperties.getAccessKey(), minioProperties.getSecretKey())
                .endpoint(minioProperties.getEndpoint())
                .build();
    }

}

package com.example.config;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;

/**
 * description:
 *
 * @author DawnStar
 * date: 2023/6/14
 */
@ConfigurationProperties(prefix = "minio")
@Component
public class MinioProperties {
    private String accessKey;

    private String secretKey;

    private String endpoint;

    private String[] initBuckets;

    private String operationBucket;

    public String getOperationBucket() {
        return operationBucket;
    }

    public void setOperationBucket(String operationBucket) {
        this.operationBucket = operationBucket;
    }

    public String[] getInitBuckets() {
        return initBuckets;
    }

    public void setInitBuckets(String[] initBuckets) {
        this.initBuckets = initBuckets;
    }

    public String getAccessKey() {
        return accessKey;
    }

    public void setAccessKey(String accessKey) {
        this.accessKey = accessKey;
    }

    public String getSecretKey() {
        return secretKey;
    }

    public void setSecretKey(String secretKey) {
        this.secretKey = secretKey;
    }

    public String getEndpoint() {
        return endpoint;
    }

    public void setEndpoint(String endpoint) {
        this.endpoint = endpoint;
    }
}

文件工具类

package com.example.util;

import java.io.*;
import java.nio.file.Files;
import java.util.Random;
import java.util.concurrent.ThreadLocalRandom;

/**
 * description:
 * 生成文件工具类
 *
 * @author DawnStar
 * date: 2023/6/15
 */
public class MinioFileUtils {
    private final static ThreadLocalRandom RANDOM = ThreadLocalRandom.current();

    public static InputStream createFile(String fileName) {
        File file = new File(fileName);
        try (BufferedWriter bufferedWriter = new BufferedWriter(
                new OutputStreamWriter(Files.newOutputStream(file.toPath())))) {
            bufferedWriter.write("这是一个测试文件");
            return Files.newInputStream(file.toPath());
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    public static File getUploadFile(String fileName) {
        File file = new File(fileName);
        try (BufferedWriter bufferedWriter = new BufferedWriter(
                new OutputStreamWriter(Files.newOutputStream(file.toPath())))) {
            bufferedWriter.write(generateContent(fileName));
            return file;
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }



    public static String generateContent(String fileName) {
        StringBuilder stringBuilder = new StringBuilder();
        stringBuilder.append(fileName).append("\n");
        int anInt = RANDOM.nextInt(300, 500);
        int lineLength = 50;
        for (int i = 0; i < anInt; i++) {
            for (int i1 = 0; i1 < lineLength; i1++) {
                char c = (char) RANDOM.nextInt(30, 122);
                stringBuilder.append(c);
            }
            stringBuilder.append("\n");
        }

        return stringBuilder.toString();
    }


    public static String generateContentPartSize(String fileName,long partSize) {
        if (partSize < 5 * 1024 * 1024) {
            throw new IllegalArgumentException("非法参数");
        }
        StringBuilder stringBuilder = new StringBuilder();
        stringBuilder.append(fileName).append("\n");
        int lineLength = 50;
        long l = partSize / 50 + 1;
        for (long i = 0; i < l; i++) {
            for (int i1 = 0; i1 < lineLength; i1++) {
                char c = (char) RANDOM.nextInt(30, 122);
                stringBuilder.append(c);
            }
            stringBuilder.append("\n");
        }

        return stringBuilder.toString();
    }


}

Minio 部分Java API 使用

MinioBucketApi
package com.example.template;

import io.minio.*;
import io.minio.messages.*;
import lombok.SneakyThrows;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import javax.annotation.Resource;
import java.util.HashMap;
import java.util.List;

/**
 * description:
 * minio 方法测试
 *
 * @author DawnStar
 * date: 2023/6/15
 */
@Component
public class MinioBucketApi {

    @Resource
    private MinioClient minioClient;

    @Value("${minio.operation-bucket}")
    private String buckName;

    /**
     * 桶标签操作
     */
    @SneakyThrows
    public void bucketTagsOperation() {
        System.out.println("***********************桶标签操作*******************************");
        Tags tags = Tags.newBucketTags(new HashMap<String, String>(2) {{
            this.put("blessing", "star");
            this.put("images", "image");
        }});
        SetBucketTagsArgs setBucketTagsArgs = SetBucketTagsArgs.builder().bucket(buckName).tags(tags).build();
        minioClient.setBucketTags(setBucketTagsArgs);
        System.out.println("================获取桶标签========================");
        GetBucketTagsArgs getBucketTagsArgs = GetBucketTagsArgs.builder().bucket(buckName).build();
        Tags bucketTags = minioClient.getBucketTags(getBucketTagsArgs);
        System.out.println(buckName + "标签：" + bucketTags.get());

        System.out.println("================设置桶标签========================");
        SetBucketTagsArgs newSetBucketTagsArgs = SetBucketTagsArgs.builder().bucket(buckName).tags(new HashMap<String, String>(1) {{
            this.put("newTag", "newTag");
        }}).build();
        minioClient.setBucketTags(newSetBucketTagsArgs);
        Tags bucketTags1 = minioClient.getBucketTags(getBucketTagsArgs);
        System.out.println(buckName + "标签：" + bucketTags1.get());

        System.out.println("================删除桶标签=======================");
        DeleteBucketTagsArgs deleteBucketTagsArgs = DeleteBucketTagsArgs.builder().bucket(buckName).build();
        minioClient.deleteBucketTags(deleteBucketTagsArgs);
        Tags newTags = minioClient.getBucketTags(getBucketTagsArgs);
        System.out.println(buckName + "标签：" + newTags.get());
        System.out.println("***********************************************************************************");

    }


    /**
     * 桶版本操作
     */
    @SneakyThrows
    public void bucketVersionOperation() {
        System.out.println("***********************桶版本操作*******************************");
        System.err.println("桶版本默认没有，开启了就只有两种状态：Enable 和 Suspended状态");
        System.out.println("=================获取桶版本===================");
        GetBucketVersioningArgs getBucketVersioningArgs = GetBucketVersioningArgs.builder().bucket(buckName).build();
        VersioningConfiguration bucketVersioning = minioClient.getBucketVersioning(getBucketVersioningArgs);
        System.out.println(buckName + "版本：" + bucketVersioning.status().toString() + bucketVersioning.isMfaDeleteEnabled());

        System.out.println("===================设置桶版本==================");
        VersioningConfiguration.Status enabled = VersioningConfiguration.Status.ENABLED;
        VersioningConfiguration configuration = new VersioningConfiguration(enabled, true);
        SetBucketVersioningArgs setBucketVersioningArgs = SetBucketVersioningArgs.builder().bucket(buckName).config(configuration).build();
        minioClient.setBucketVersioning(setBucketVersioningArgs);
        bucketVersioning = minioClient.getBucketVersioning(getBucketVersioningArgs);
        System.out.println(buckName + "版本：" + bucketVersioning.status().toString() + bucketVersioning.isMfaDeleteEnabled());
        System.out.println("***********************************************************************************");

    }

    /**
     * 桶基本操作
     */
    @SneakyThrows
    public void bucketBasicOperation() {
        System.out.println("***********************桶基本操作*******************************");
        System.out.println("===============检查桶是否存在========================");
        BucketExistsArgs bucketExistsArgs = BucketExistsArgs.builder().bucket(buckName).build();
        boolean bucketExists = minioClient.bucketExists(bucketExistsArgs);
        System.out.println(buckName + "存在：" + bucketExists);
        // 获取桶列表
        System.out.println("=================获取桶列表======================");
        List<Bucket> buckets = minioClient.listBuckets();
        buckets.forEach(bucket -> System.out.println(bucket.name()));

        System.out.println("=================新建桶============================");
        MakeBucketArgs makeBucketArgs = MakeBucketArgs.builder().bucket("test").objectLock(true).build();
        // 创建已有bucket会创建失败
        minioClient.makeBucket(makeBucketArgs);
        buckets = minioClient.listBuckets();
        buckets.forEach(bucket -> System.out.println(bucket.name()));


        // 删除桶,有文件就会删除失败
        System.out.println("=================删除桶======================");
        RemoveBucketArgs removeBucketArgs = RemoveBucketArgs.builder().bucket("test").build();
        minioClient.removeBucket(removeBucketArgs);
        buckets = minioClient.listBuckets();
        buckets.forEach(bucket -> System.out.println(bucket.name()));
        System.out.println("***********************************************************************************");

    }


    @SneakyThrows
    public void objectLockConfigurationOperation() {
        System.out.println("***********************桶对象锁配置*******************************");

        GetObjectLockConfigurationArgs getObjectLockConfigurationArgs = GetObjectLockConfigurationArgs.builder().bucket(buckName).build();
        ObjectLockConfiguration objectLockConfiguration = minioClient.getObjectLockConfiguration(getObjectLockConfigurationArgs);
        System.out.println("获取当前的bucket对象锁配置"+objectLockConfiguration.mode() + " " + objectLockConfiguration.duration());


        objectLockConfiguration = new ObjectLockConfiguration(RetentionMode.COMPLIANCE, new RetentionDurationDays(1));
        minioClient.setObjectLockConfiguration(SetObjectLockConfigurationArgs.builder().bucket(buckName).config(objectLockConfiguration).build());
        objectLockConfiguration = minioClient.getObjectLockConfiguration(getObjectLockConfigurationArgs);
        System.out.println("设置后的bucket对象锁配置"+objectLockConfiguration.mode() + " " + objectLockConfiguration.duration());


        minioClient.deleteObjectLockConfiguration(DeleteObjectLockConfigurationArgs.builder().bucket(buckName).build());
        objectLockConfiguration = minioClient.getObjectLockConfiguration(getObjectLockConfigurationArgs);
        System.out.println("删除桶锁对象配置："+objectLockConfiguration.mode() + " " + objectLockConfiguration.duration());
        System.out.println("***********************************************************************************");

    }

}
MinioObjectApi
package com.example.template;

import cn.hutool.http.ContentType;
import cn.hutool.http.HttpRequest;
import cn.hutool.http.HttpResponse;
import cn.hutool.http.HttpUtil;
import com.example.util.MinioFileUtils;
import io.minio.*;
import io.minio.http.Method;
import io.minio.messages.*;
import lombok.SneakyThrows;
import okhttp3.*;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import javax.annotation.Resource;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;

/**
 * description:
 *
 * @author DawnStar
 * date: 2023/6/16
 */
@Component
public class MinioObjectApi {
    @Resource
    private MinioClient minioClient;

    @Value("${minio.operation-bucket}")
    private String buckName;

    @SneakyThrows
    public void putObject() {
        // 上传已知的大小的文件
        System.out.println("************************************putObject操作***********************************************");
        String fileName = "test.txt";
        InputStream inputStream = MinioFileUtils.createFile(fileName);
        System.out.println("==================上传已知的大小的文件==========================");
        PutObjectArgs firstObjectArgs = PutObjectArgs.builder()
                .bucket(buckName)
                .object(fileName)
                // -1 表示默认分片为 5M
                .stream(inputStream, inputStream.available(), -1).build();
        ObjectWriteResponse objectWriteResponse = minioClient.putObject(firstObjectArgs);
        System.out.println(objectWriteResponse.versionId() + " " + objectWriteResponse.etag());

        // 上传未知大小的文件
        System.out.println("==================上传未知的大小的文件==========================");
        PutObjectArgs secondObjectArgs = PutObjectArgs.builder()
                .bucket(buckName)
                .object(fileName)
                .stream(inputStream, -1, PutObjectArgs.MIN_MULTIPART_SIZE)
                .build();
        objectWriteResponse = minioClient.putObject(secondObjectArgs);
        System.out.println(objectWriteResponse.versionId() + " " + objectWriteResponse.etag());

        // 上传未知大小的文件
        System.out.println("==================上传内存中的数据==========================");
        String content = "这是一个内存运行的数据";
        Map<String, String> headers = new HashMap<>();
        headers.put("X-Amz-Storage-Class", "REDUCED_REDUNDANCY");
        Map<String, String> userMetadata = new HashMap<>();
        byte[] contentBytes = content.getBytes(StandardCharsets.UTF_8);
        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(contentBytes);
        PutObjectArgs thirdObjectArgs = PutObjectArgs.builder()
                .bucket(buckName)
                // 将创建一个名为putMethod文件夹
                .object("putMethod/content.txt")
                // 两者不能同时为-1
                .stream(byteArrayInputStream, -1, PutObjectArgs.MIN_MULTIPART_SIZE)
                .headers(headers)
                .userMetadata(userMetadata)
                .build();
        objectWriteResponse = minioClient.putObject(thirdObjectArgs);
        System.out.println(objectWriteResponse.versionId() + " " + objectWriteResponse.etag());
        System.out.println("***********************************************************************************");

    }

    @SneakyThrows
    public void uploadObject() {
        System.out.println("************************************uploadObject操作***********************************************");

        String fileName = "uploadObject.txt";
        File file = MinioFileUtils.getUploadFile(fileName);
        UploadObjectArgs uploadObjectArgs = UploadObjectArgs.builder()
                .bucket(buckName)
                .object(fileName)
//                .filename(file.toPath())
                .filename(file.toPath().toString(), UploadObjectArgs.MAX_PART_SIZE)
                .build();
        ObjectWriteResponse objectWriteResponse = minioClient.uploadObject(uploadObjectArgs);
        System.out.println(objectWriteResponse.versionId() + " " + objectWriteResponse.etag());
        System.out.println("***********************************************************************************");
    }

    @SneakyThrows
    public void copyObject() {
        String fileName = "copy.txt";
        putObjectByPartSize("", fileName);
        System.out.println("************************************copyObject操作***********************************************");
        // object() 与 source()不可相同
        CopyObjectArgs copyObjectArgs = CopyObjectArgs.builder()
                .bucket(buckName)
                .object("copyMethod/" + fileName)
                .source(CopySource.builder().bucket(buckName).object(fileName).build())
                .build();
        ObjectWriteResponse objectWriteResponse = minioClient.copyObject(copyObjectArgs);
        System.out.println(objectWriteResponse.versionId() + " " + objectWriteResponse.etag());
        System.out.println("***********************************************************************************");
    }

    @SneakyThrows
    public void composeObject() {
        System.out.println("************************************composeObject操作***********************************************");
        List<ComposeSource> composeSources = new ArrayList<>();
        String prefix = "subComposeObject";
        for (int i = 0; i < 2; i++) {
            String composeFileName = prefix + i + ".txt";
            putObjectByPartSize("composeMethod", composeFileName);
            ComposeSource composeSource = ComposeSource.builder()
                    .bucket(buckName)
                    .object("composeMethod/" + composeFileName)
                    .build();
            composeSources.add(composeSource);
        }
        String fileName = "composeObject.txt";
        ComposeObjectArgs composeObjectArgs = ComposeObjectArgs.builder()
                .bucket(buckName)
                .object("composeMethod/" + fileName)
                // 每个文件大小不可以低于5M
                .sources(composeSources)
                .build();
        ObjectWriteResponse objectWriteResponse = minioClient.composeObject(composeObjectArgs);
        System.out.println(objectWriteResponse.versionId() + " " + objectWriteResponse.etag());
        System.out.println("***********************************************************************************");

    }

    @SneakyThrows
    public void uploadSnowballObjects() {
        System.out.println("************************************uploadSnowballObjects操作***********************************************");
        String prefix = "uploadSnowballMethod";
        List<SnowballObject> objects = new ArrayList<>();
        // 参数中size必须与文件大小相等
        String content = MinioFileUtils.generateContent(prefix + "my-object-one");
        byte[] contentBytes = content.getBytes(StandardCharsets.UTF_8);

        InputStream inputStream = MinioFileUtils.createFile(prefix + "my-object-two");

        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(contentBytes);
        objects.add(
                new SnowballObject(
                        prefix + "/my-object-one",
                        byteArrayInputStream,
                        byteArrayInputStream.available(),
                        null));
        objects.add(
                new SnowballObject(
                        prefix + "/my-object-two",
                        inputStream,
                        inputStream.available(),
                        null));
        ObjectWriteResponse objectWriteResponse = minioClient.uploadSnowballObjects(
                UploadSnowballObjectsArgs.builder().bucket(buckName).objects(objects).build());
        System.out.println(objectWriteResponse.versionId() + " " + objectWriteResponse.etag());
        System.out.println("***********************************************************************************");
    }

    @SneakyThrows
    public void removeObject() {
        System.out.println("************************************removeObject与removeObjects操作***********************************************");
        System.out.println("====================删除单个对象================================");
        RemoveObjectArgs removeObjectArgs = RemoveObjectArgs.builder().bucket(buckName).object("test").build();

        // 删除不存在的对象也不会有问题
        minioClient.removeObject(removeObjectArgs);
        DeleteObject deleteObject = new DeleteObject("test");

        System.out.println("====================删除多个对象================================");
        String prefix = "removeMethod";
        DeleteObject deleteObject1 = new DeleteObject(prefix + "/removeObject.txt");
        putObjectByPartSize(prefix, "removeObject.txt");
        RemoveObjectsArgs removeObjectsArgs = RemoveObjectsArgs.builder().bucket(buckName).objects(new ArrayList<>() {{
            this.add(deleteObject);
            this.add(deleteObject1);
        }}).build();
        Iterable<Result<DeleteError>> results = minioClient.removeObjects(removeObjectsArgs);
        // 删除不存在的对象也不会有问题
        for (Result<DeleteError> deleteErrorResult : results) {
            System.out.println(deleteErrorResult.get().toString());
        }
        System.out.println("***********************************************************************************");
    }


    @SneakyThrows
    public void statObject() {
        System.out.println("************************************statObjects操作***********************************************");
        uploadObject("statMethod", "statObject.txt");
        StatObjectArgs statObjectArgs = StatObjectArgs.builder()
                .bucket(buckName)
                .object("statMethod/statObject.txt")
                .build();
        StatObjectResponse statObjectResponse = minioClient.statObject(statObjectArgs);
        System.out.println("statObject：" + statObjectResponse.etag() + " " + statObjectResponse.object() + " " + statObjectResponse.versionId());
        System.out.println("***********************************************************************************");
    }


    @SneakyThrows
    public void getObject() {
        System.out.println("************************************getObjects操作***********************************************");
        uploadObject("getMethod", "getObject.txt");
        GetObjectArgs getObjectArgs = GetObjectArgs.builder()
                .bucket(buckName)
                .object("getMethod/getObject.txt")
                // 起始位置
                .offset(10L)
                // 数据长度
                .length(100L)
                .build();
        GetObjectResponse response = minioClient.getObject(getObjectArgs);
        byte[] bytes = response.readAllBytes();
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(bytes.length);
        System.out.println(byteArrayOutputStream);
        byteArrayOutputStream.close();
        response.close();
        System.out.println("***********************************************************************************");


    }


    @SneakyThrows
    public void getPresignedObjectUrl() {
        System.out.println("************************************getPresignedObjectUrl操作***********************************************");

        GetPresignedObjectUrlArgs getPresignedObjectUrlArgs = GetPresignedObjectUrlArgs.builder()
                .bucket(buckName)
                .object("getPresignedMethod/notGetPresignedObjectUrl.txt")
                .expiry(24, TimeUnit.HOURS)
                .method(Method.GET)
                .build();
        String presignedObjectUrl = minioClient.getPresignedObjectUrl(getPresignedObjectUrlArgs);
        System.out.println("没有该文件的路径" + presignedObjectUrl);


        // 获取上传路径
        GetPresignedObjectUrlArgs putPresignedObjectUrlArgs = GetPresignedObjectUrlArgs.builder()
                .bucket(buckName)
                .object("getPresignedMethod/getPresignedObjectUrl.txt")
                .expiry(24, TimeUnit.HOURS)
                .method(Method.PUT)
                .build();
        presignedObjectUrl = minioClient.getPresignedObjectUrl(putPresignedObjectUrlArgs);
        System.out.println("PUT方法上传路径：" + presignedObjectUrl);

        // 上传数据
        HttpRequest request = HttpUtil.createRequest(cn.hutool.http.Method.PUT, presignedObjectUrl);
        byte[] bytes = MinioFileUtils.generateContent("getPresignedObjectUrl.txt").getBytes(StandardCharsets.UTF_8);
        HttpResponse execute = request.body(bytes).contentType(ContentType.TEXT_PLAIN.getValue()).execute();
        execute.close();


        GetPresignedObjectUrlArgs targetPresignedObjectUrlArgs = GetPresignedObjectUrlArgs.builder()
                .bucket(buckName)
                .object("getPresignedMethod/getPresignedObjectUrl.txt")
                .expiry(24, TimeUnit.HOURS)
                .method(Method.GET)
                .build();
        presignedObjectUrl = minioClient.getPresignedObjectUrl(targetPresignedObjectUrlArgs);
        System.out.println("获取路径：" + presignedObjectUrl);


        GetPresignedObjectUrlArgs headPresignedObjectUrlArgs = GetPresignedObjectUrlArgs.builder()
                .bucket(buckName)
                .object("getPresignedMethod/getPresignedObjectUrl.txt")
                .method(Method.HEAD)
                .build();
        presignedObjectUrl = minioClient.getPresignedObjectUrl(headPresignedObjectUrlArgs);
        System.out.println("HEAD方法路径，请通过postman请求该路径：" + presignedObjectUrl);
        System.out.println("***********************************************************************************");

    }


    @SneakyThrows
    public void downLoadObject() {
        System.out.println("************************************downLoadObject操作***********************************************");
        uploadObject("downloadMethod", "downloadObject.txt");
        DownloadObjectArgs downloadObjectArgs = DownloadObjectArgs.builder()
                .bucket(buckName)
                .object("downloadMethod/downloadObject.txt")
                .filename("downloadObject.txt")
                .build();
        // 文件名存在将会下载失败
        System.err.println("文件名存在将会下载失败");
        minioClient.downloadObject(downloadObjectArgs);
        File file = new File("downloadObject.txt");
        BufferedReader bufferedReader = new BufferedReader(new FileReader(file));
        String readLine;
        while ((readLine = bufferedReader.readLine()) != null) {
            System.out.print(readLine);
        }
        bufferedReader.close();
        System.out.println("***********************************************************************************");
    }

    @Value("${minio.endpoint}")
    private String url;

    @SneakyThrows
    public void getPresignedPostFormData() {
        System.out.println("************************************getPresignedPostFormData操作***********************************************");

        // 创建一个时效为7天的策略
        PostPolicy policy = new PostPolicy(buckName, ZonedDateTime.now().plusDays(7));
        String objectName = "PostFormMethod/postFormData.txt";
        // 添加条件，`key`键为对象名称
        policy.addEqualsCondition("key", objectName);

        // 添加'Content-Type.设置前缀为“application”
        policy.addStartsWithCondition("Content-Type", "application/");

        // 添加内容长度范围，20Bit - 1MiB，不在这个区间的文件上传失败
        policy.addContentLengthRangeCondition(20, 1024*1024);

        Map<String, String> formData = minioClient.getPresignedPostFormData(policy);
        // 以表单的形式上传文本
        MultipartBody.Builder multipartBuilder = new MultipartBody.Builder();
        multipartBuilder.setType(MultipartBody.FORM);
        for (Map.Entry<String, String> entry : formData.entrySet()) {
            multipartBuilder.addFormDataPart(entry.getKey(), entry.getValue());
        }
        multipartBuilder.addFormDataPart("key", objectName);
        multipartBuilder.addFormDataPart("Content-Type",ContentType.OCTET_STREAM.getValue());
        String fileName = objectName.substring(objectName.lastIndexOf("/") + 1);
        File uploadFile = MinioFileUtils.getUploadFile(fileName);
        // "file" 必须最后添加
        multipartBuilder.addFormDataPart(
                "file", objectName, RequestBody.create(uploadFile, null));
        System.out.println(url+"/"+objectName);
        Request request =
                new Request.Builder()
                        // minio服务器路径
                        .url(url +"/"+buckName)
                        .post(multipartBuilder.build())
                        .build();
        OkHttpClient httpClient = new OkHttpClient().newBuilder().build();
        try (Response response = httpClient.newCall(request).execute();) {
            if (response.isSuccessful()) {
                System.out.println(fileName + " is uploaded successfully using POST object");
            } else {
                System.out.println("Failed to upload " + fileName);
            }
        }
        System.out.println("***********************************************************************************");
    }


    @SneakyThrows
    public void listObjects() {
        System.out.println("************************************listObjects操作***********************************************");
        uploadObject("listMethod", "listObject1.txt");
        uploadObject("listMethod", "listObject2.txt");
        uploadObject("listMethod", "listObject3.txt");
        System.out.println("===================第一种List==================");
        ListObjectsArgs listObjectsArgs = ListObjectsArgs.builder().bucket(buckName).build();
        Iterable<Result<Item>> listObjects = minioClient.listObjects(listObjectsArgs);
        print(listObjects);

        System.out.println("===================第二种List==================");
        listObjectsArgs = ListObjectsArgs.builder()
                .bucket(buckName)
                .recursive(true)
                .build();
        print(minioClient.listObjects(listObjectsArgs));

        System.out.println("===================第三种List==================");
        listObjectsArgs = ListObjectsArgs.builder()
                .bucket(buckName)
                .recursive(true)
                .prefix("listMethod")
                .build();
        print(minioClient.listObjects(listObjectsArgs));

        System.out.println("===================第四种List==================");
        listObjectsArgs = ListObjectsArgs.builder()
                .bucket(buckName)
                .recursive(true)
                .prefix("listMethod")
                .startAfter("listMethod/listObject2.txt")
                .build();
        print(minioClient.listObjects(listObjectsArgs));
        // ......
        System.out.println("***********************************************************************************");
    }

    private void print(Iterable<Result<Item>> listObjects) throws Exception {
        for (Result<Item> listObject : listObjects) {
            System.out.println(listObject.get().objectName());
        }
    }


    @SneakyThrows
    public void objectRetentionOperation() {
        System.out.println("************************************对象保留策略操作***********************************************");
        System.err.println("创建bucket需要设置objectLock为true");
        uploadObject("retentionMethod", "objectRetention.txt");
        GetObjectRetentionArgs getObjectRetentionArgs = GetObjectRetentionArgs.builder().bucket(buckName).object("retentionMethod/objectRetention.txt")
                .build();
        Retention objectRetention = minioClient.getObjectRetention(getObjectRetentionArgs);
        if (objectRetention == null) {
            System.out.println("修改策略前配置: null");
        } else {
            System.out.println("修改策略前配置: " + objectRetention.mode().toString() + " " + objectRetention.retainUntilDate().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")));
        }
        Retention retention = new Retention(RetentionMode.COMPLIANCE, ZonedDateTime.now().plusSeconds(40));
        SetObjectRetentionArgs setObjectRetentionArgs = SetObjectRetentionArgs.builder()
                .bucket(buckName)
                .object("retentionMethod/objectRetention.txt")
                .config(retention)
                // ByPass Mode（略过模式或旁路模式），泛指在一个系统的正常流程中，
                // 有一堆检核机制，而“ByPass Mode”就是当检核机制发生异常，
                // 无法在短期间内排除时，使系统作业能绕过这些检核机制，
                // 使系统能够继续运行的作业模式
                .bypassGovernanceMode(true)
                .build();

        minioClient.setObjectRetention(setObjectRetentionArgs);
        objectRetention = minioClient.getObjectRetention(getObjectRetentionArgs);
        System.out.println("修改策略后配置：" + objectRetention.mode().toString() + " " + objectRetention.retainUntilDate().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")));

        System.out.println("***********************************************************************************");

    }


    @SneakyThrows
    public void objectLegalHoldOperation() {
        System.out.println("************************************对象合法保留操作***********************************************");
        System.err.println("创建bucket需要设置objectLock为true");
        uploadObject("legalHoldMethod", "objectLegalHoldMethod.txt");

        IsObjectLegalHoldEnabledArgs isObjectLegalHoldEnabledArgs = IsObjectLegalHoldEnabledArgs.builder()
                .bucket(buckName)
                .object("legalHoldMethod/objectLegalHoldMethod.txt")
                .build();

        boolean objectLegalHoldEnabled = minioClient.isObjectLegalHoldEnabled(isObjectLegalHoldEnabledArgs);
        System.out.println("执行isObjectLegalHoldEnabledArgs：" + objectLegalHoldEnabled);

        EnableObjectLegalHoldArgs enableObjectLegalHoldArgs = EnableObjectLegalHoldArgs.builder()
                .bucket(buckName)
                .object("legalHoldMethod/objectLegalHoldMethod.txt")
                .build();
        minioClient.enableObjectLegalHold(enableObjectLegalHoldArgs);

        objectLegalHoldEnabled = minioClient.isObjectLegalHoldEnabled(isObjectLegalHoldEnabledArgs);
        System.out.println("执行enableObjectLegalHold：" + objectLegalHoldEnabled);

        DisableObjectLegalHoldArgs disableObjectLegalHoldArgs = DisableObjectLegalHoldArgs.builder()
                .bucket(buckName)
                .object("legalHoldMethod/objectLegalHoldMethod.txt")
                .build();

        minioClient.disableObjectLegalHold(disableObjectLegalHoldArgs);

        objectLegalHoldEnabled = minioClient.isObjectLegalHoldEnabled(isObjectLegalHoldEnabledArgs);
        System.out.println("执行disableObjectLegalHold：" + objectLegalHoldEnabled);

        System.out.println("***********************************************************************************");

    }


    @SneakyThrows
    public void objectTagOperation() {
        System.out.println("************************************对象标签操作***********************************************");
        uploadObject("tagMethod", "objectTags.txt");
        // 与给bucket添加标签的方法是相似的
        SetObjectTagsArgs setObjectTagsArgs = SetObjectTagsArgs.builder()
                .bucket(buckName)
                .object("tagMethod/objectTags.txt")
                .tags(new HashMap<String, String>(1) {{
                    this.put("blessing", "star");
                }})
                .build();
        minioClient.setObjectTags(setObjectTagsArgs);


        GetObjectTagsArgs getObjectTagsArgs = GetObjectTagsArgs.builder()
                .bucket(buckName)
                .object("tagMethod/objectTags.txt")
                .build();
        Tags objectTags = minioClient.getObjectTags(getObjectTagsArgs);
        System.out.println("设置后的标签：" + objectTags.get());
        DeleteObjectTagsArgs deleteObjectTagsArgs = DeleteObjectTagsArgs.builder()
                .bucket(buckName)
                .object("tagMethod/objectTags.txt")
                .build();

        minioClient.deleteObjectTags(deleteObjectTagsArgs);

        objectTags = minioClient.getObjectTags(getObjectTagsArgs);
        System.out.println("删除后的标签对象：" + objectTags.get());
        System.out.println("***********************************************************************************");
    }


//    @SneakyThrows
//    public void selectObjectContent() {
//        uploadObject("selectMethod", "selectObjectContent1.txt");
//        uploadObject("selectMethod", "selectObjectContent2.txt");
//        uploadObject("selectMethod", "selectObjectContent3.txt");
//        String sqlExpression = "select * from S3Object";
//        InputSerialization is = new InputSerialization(null, false, null, null, FileHeaderInfo.USE, null, null, null);
//        OutputSerialization os = new OutputSerialization(null, null, null, QuoteFields.ASNEEDED, null);
//        SelectResponseStream stream =
//                minioClient.selectObjectContent(
//                        SelectObjectContentArgs.builder()
//                                .bucket(buckName)
//                                .object("selectMethod/selectObjectContent3.txt")
//                                .sqlExpression(sqlExpression)
//                                .inputSerialization(is)
//                                .outputSerialization(os)
//                                .requestProgress(true)
//                                .build());
//
//        byte[] buf = new byte[512];
//        int bytesRead = stream.read(buf, 0, buf.length);
//        System.out.println(new String(buf, 0, bytesRead, StandardCharsets.UTF_8));
//
//        Stats stats = stream.stats();
//        System.out.println("bytes scanned: " + stats.bytesScanned());
//        System.out.println("bytes processed: " + stats.bytesProcessed());
//        System.out.println("bytes returned: " + stats.bytesReturned());
//
//        stream.close();
//    }


    ///  The operation is not valid for the current state of the object.不知道还需要什么状态
//    @SneakyThrows
//    public void restoreObject() {
//        System.out.println("************************************restoreObject操作***********************************************");
//        uploadObject("restoreMethod", "restoreObject.txt");
////        minioClient.removeObject(RemoveObjectArgs.builder().bucket(buckName).object("restoreMethod/restoreObject.txt").build());
//
//        RestoreObjectArgs restoreObjectArgs = RestoreObjectArgs.builder()
//                .bucket(buckName)
//                .object("restoreMethod/restoreObject.txt")
//                .request(new RestoreRequest(null, null, null, null, null, null))
//                .build();
//        minioClient.restoreObject(restoreObjectArgs);
//
//        System.out.println("***********************************************************************************");
//
//    }

    @SneakyThrows
    public void uploadObject(String folder, String fileName) {
        byte[] contentBytes = MinioFileUtils.generateContent(fileName).getBytes(StandardCharsets.UTF_8);
        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(contentBytes);
        PutObjectArgs thirdObjectArgs = PutObjectArgs.builder()
                .bucket(buckName)
                // 将创建一个名为putMethod文件夹
                .object(folder + "/" + fileName)
                // 两者不能同时为-1
                .stream(byteArrayInputStream, -1, PutObjectArgs.MIN_MULTIPART_SIZE)
                .build();
        ObjectWriteResponse objectWriteResponse = minioClient.putObject(thirdObjectArgs);
    }

    @SneakyThrows
    private void putObjectByPartSize(String folder, String fileName) {
        String content = MinioFileUtils.generateContentPartSize(fileName, PutObjectArgs.MIN_MULTIPART_SIZE);
        byte[] contentBytes = content.getBytes(StandardCharsets.UTF_8);
        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(contentBytes);
        PutObjectArgs thirdObjectArgs = PutObjectArgs.builder()
                .bucket(buckName)
                // 将创建一个名为putMethod文件夹
                .object(folder + "/" + fileName)
                // 两者不能同时为-1
                .stream(byteArrayInputStream, -1, PutObjectArgs.MIN_MULTIPART_SIZE)
                .build();
        minioClient.putObject(thirdObjectArgs);
    }


}
启动类
package com.example;

import io.minio.*;
import io.minio.messages.*;
import com.example.config.MinioProperties;
import com.example.template.MinioBucketApi;
import com.example.template.MinioObjectApi;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.boot.WebApplicationType;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.context.ConfigurableApplicationContext;

import javax.annotation.Resource;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

/**
 * Description:
 *
 * @author DawnStar
 * Date: 2023/6/16 19:24
 */
@SpringBootApplication
public class MinioApplication implements ApplicationRunner {
    public static void main(String[] args) throws Exception {
        ConfigurableApplicationContext context = new SpringApplicationBuilder(MinioApplication.class).web(WebApplicationType.NONE).run(args);
        MinioBucketApi bean = context.getBean(MinioBucketApi.class);
        invokeMethod(bean);
//        bean.objectLockConfigurationOperation();

        MinioObjectApi objectApiBean = context.getBean(MinioObjectApi.class);
        objectApiBean.getPresignedPostFormData();
        invokeMethod(objectApiBean);
        System.out.println("结束");
//        objectApiBean.objectLegalHoldOperation();
//        SpringApplication.run(MinioApplication.class, args);
    }

    @Resource
    private MinioProperties minioProperties;

    @Override
    public void run(ApplicationArguments args) throws Exception {
        if (minioProperties.getInitBuckets().length == 0) {
            return;
        }
        // 初始化桶
        MinioClient minioClient = MinioClient.builder().credentials(minioProperties.getAccessKey(), minioProperties.getSecretKey())
                .endpoint(minioProperties.getEndpoint())
                .build();
        // 获取所有桶
        List<Bucket> buckets = minioClient.listBuckets();
        List<String> collect = buckets.stream().map(Bucket::name).collect(Collectors.toList());
        List<String> targetBucketNames = Arrays.stream(minioProperties.getInitBuckets()).filter(s -> !collect.contains(s)).collect(Collectors.toList());
        for (String targetBucketName : targetBucketNames) {
            MakeBucketArgs makeBucketArgs = MakeBucketArgs.builder().bucket(targetBucketName).build();
            minioClient.makeBucket(makeBucketArgs);
        }
        String operationBucket = minioProperties.getOperationBucket();
        boolean bucketExists = minioClient.bucketExists(BucketExistsArgs.builder().bucket(operationBucket).build());
        if (!bucketExists) {
            // 只能在创建时候才能设置对象锁，之后都无效
            minioClient.makeBucket(MakeBucketArgs.builder().bucket(operationBucket).objectLock(true).build());
        }
        // 防止bucket被设置了对象锁，无法修改minio服务器的对象
        minioClient.deleteObjectLockConfiguration(DeleteObjectLockConfigurationArgs.builder().bucket(operationBucket).build());
    }


    private static void invokeMethod(Object o) {
        Method[] declaredMethods = o.getClass().getDeclaredMethods();

        for (Method declaredMethod : declaredMethods) {
            if (declaredMethod.getParameters().length > 0) {
                continue;
            }
            try {

                if (declaredMethod.getReturnType().equals(void.class) && !declaredMethod.getName().contains("lambda")) {
                    declaredMethod.invoke(o);
                }
            } catch (Exception e) {
                e.printStackTrace();
            }
            System.out.println("\n\n");
        }
    }
}

注意事项

关于Nginx代理问题

通过nginx代理，https下，通过getPresignedObjectUrl方法获取文件URL可能无法访问指定的文件。 以下是官方给出的一种方案，详情请访问Minio Nginx代理

 location / {
      proxy_set_header Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;

      proxy_connect_timeout 300;
      # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
      proxy_http_version 1.1;
      proxy_set_header Connection "";
      chunked_transfer_encoding off;

      proxy_pass https://minio:9000/; # This uses the upstream directive definition to load balance
   }

   location /minio {
      proxy_set_header Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-NginX-Proxy true;

      # This is necessary to pass the correct IP to be hashed
      real_ip_header X-Real-IP;

      proxy_connect_timeout 300;

      # To support websockets in MinIO versions released after January 2023
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";

      chunked_transfer_encoding off;

      proxy_pass https://minio:9001/; # This uses the upstream directive definition to load balance and assumes a static Console port of 9001
   }

关于秒传，断点续传，分片续传问题

目前(2023/06/17) Minio Java API没有提供断点续传功能，所以如果想实现上述功能，可以借助Redis,和继承MinioAsyncClient类并将其重写开放部分方法的访问权限来实现相关功能。可以参考这篇博客blog.csdn.net/Alan_ran/ar… 不过他的版本是低的，所以有些方法过时了如 listParts方法便是过时的。

image.png 当然官方API也给予相关的方法是listPartsAsync方法，可以看出listParts方法中也是调用了listPartsAsync方法。更多具体细节请查看S3Base类以及其子类MinioAsyncClient,在8.5.3中MinioClinet其实是作为一个工具类，其内部封装了MinioAsyncClient类。

参考文档

Minio Java API

Minio Docker部署

blog.csdn.net/qq_43437874…

avatar
文章
阅读
粉丝