如何结合 Minio 实现一个简单的可嵌入的 Spring Boot Starter 文件服务

5,452 阅读6分钟

Quiet 项目简介:juejin.cn/post/717122…

上一篇:

作为一个后端 Java 开发,为何、如何自己实现一个 Markdown 编辑器

前言

在上一篇文章中,Markdown 编辑器还没有实现图片上传的功能,要实现图片上传,那么后端服务就需要支持文件上传,文件上传是很多后端服务需要的功能,可以实现一个 Spring Boot Starter 来支持文件的相关功能,比如文件上传、预览、下载、删除等。

实现原理:Creating Your Own Auto-configuration

开源组件版本

Spring Boot:2.6.3

Minio:8.2.1

安装 Minio

Minio 简介:min.io/

# 拉取 minio 镜像
docker pull minio/minio
# 运行 minio 服务
docker run -n minio-dev -p 7000:9000 -p 7001:9001 minio/minio server /data --console-address ":9001"

启动完成后,本地访问:http://localhost:7001/

项目依赖

因为是嵌入一个文件服务,在平常的 Spring Boot 项目中可以查看项目的健康状况,那么 Minio 服务的状态也添加进项目健康状况中,这样就能监控 Minio 的服务状况了,所以需要添加依赖 org.springframework.boot:spring-boot-starter-actuator,还需要支持文件相关接口,需要添加org.springframework.boot:spring-boot-starter-webio.minio:minio:8.2.1 依赖。

Starter 实现

自定义 Spring Boot Starter 是 Spring Boot 一个比较常用的扩展点,可以利用这个扩展点为应用提供默认配置或者自定义功能等。

准备

新建一个 quiet-minio-spring-boot-starter 项目,同时新建一个 spring.factories 文件。

spring.factories 文件在 Spring Boot 2.7.0 版本已过期,该版本以上的请自行适配。

image.png

Config Properties

classifications:图片分类,一个项目中大部分不止一个地方用到文件上传,为了更好管理文件资源,后端可以限制可以上传的文件分类,以 / 分割可以在 Minio 中创建文件夹,实现分文件夹管理文件。

objectPrefix:访问文件时的前缀,不同的服务,它有不同的 URL,不同的端口号,这个不是必须的,但是在后端统一配置更方便统一管理,这个可以根据团队的规范自行决定是否使用。

/**
 * @author <a href="mailto:lin-mt@outlook.com">lin-mt</a>
 */
@Slf4j
@Getter
@Setter
@ConfigurationProperties(prefix = "quiet.minio")
public class MinioConfigurationProperties implements InitializingBean {

  private String url = "http://localhost:7000";

  private String accessKey;

  private String secretKey;

  private String bucketName;

  private Set<String> classifications;

  private String objectPrefix;

  private Duration connectTimeout = Duration.ofSeconds(10);

  private Duration writeTimeout = Duration.ofSeconds(60);

  private Duration readTimeout = Duration.ofSeconds(10);

  private boolean checkBucket = true;

  private boolean createBucketIfNotExist = true;

  @Override
  public void afterPropertiesSet() {
    Assert.hasText(accessKey, "accessKey must not be empty.");
    Assert.hasText(secretKey, "secretKey must not be empty.");
    Assert.hasText(bucketName, "bucketName must not be empty.");
    Assert.hasText(objectPrefix, "objectPrefix must not be empty.");
  }
}

Configuration

  1. 创建一个配置类,在这个类中,使用我们上一步提供的配置信息,注入一个 Bean 实例 MinioClient,所有文件的相关操作都可以通过这个 Bean 实现。
  2. spring.factory 文件中需要添加:org.springframework.boot.autoconfigure.EnableAutoConfiguration=com.gitee.quiet.minio.config.QuietMinioConfiguration

该步骤是实现项目嵌入 Minio 服务的关键,具体的原理可以看源码 org.springframework.boot.autoconfigure.AutoConfigurationImportSelectororg.springframework.core.io.support.SpringFactoriesLoader

/**
 * @author <a href="mailto:lin-mt@outlook.com">lin-mt</a>
 */
@Slf4j
@Configuration
@AllArgsConstructor
@ComponentScan("com.gitee.quiet.minio")
@EnableConfigurationProperties(MinioConfigurationProperties.class)
public class QuietMinioConfiguration {

  private final MinioConfigurationProperties properties;

  @Bean
  public MinioClient minioClient()
      throws ServerException, InsufficientDataException, ErrorResponseException, IOException,
          NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException,
          XmlParserException, InternalException {
    MinioClient minioClient =
        MinioClient.builder()
            .endpoint(properties.getUrl())
            .credentials(properties.getAccessKey(), properties.getSecretKey())
            .build();
    minioClient.setTimeout(
        properties.getConnectTimeout().toMillis(),
        properties.getWriteTimeout().toMillis(),
        properties.getReadTimeout().toMillis());
    if (properties.isCheckBucket()) {
      String bucketName = properties.getBucketName();
      BucketExistsArgs existsArgs = BucketExistsArgs.builder().bucket(bucketName).build();
      boolean bucketExists = minioClient.bucketExists(existsArgs);
      if (!bucketExists) {
        if (properties.isCreateBucketIfNotExist()) {
          MakeBucketArgs makeBucketArgs = MakeBucketArgs.builder().bucket(bucketName).build();
          minioClient.makeBucket(makeBucketArgs);
        } else {
          throw new IllegalStateException("Bucket does not exist: " + bucketName);
        }
      }
    }
    return minioClient;
  }
}

Controller

提供文件相关操作的接口,比如文件上传、下载、删除、预览等。

/**
 * @author <a href="mailto:lin-mt@outlook.com">lin-mt</a>
 */
@Slf4j
@RestController
@AllArgsConstructor
@RequestMapping("/minio")
public class MinioController {

  private final MinioService minioService;
  private final MinioConfigurationProperties properties;
  private final Optional<MinioHandler> minioHandler;

  private String getFileName(String object) {
    if (StringUtils.isBlank(object)) {
      return UUID.randomUUID().toString().replace("-", "");
    }
    if (!object.contains("/") || object.endsWith("/")) {
      return object;
    }
    return object.substring(object.lastIndexOf("/") + 1);
  }

  private FileResponse buildFileResponse(StatObjectResponse metadata, Tags tags) {
    FileResponse.FileResponseBuilder builder = FileResponse.builder();
    String object = metadata.object();
    String objectPrefix = properties.getObjectPrefix();
    if (!objectPrefix.endsWith("/")) {
      objectPrefix = objectPrefix + "/";
    }
    objectPrefix = objectPrefix + "minio/";
    builder
        .object(object)
        .detailPath(objectPrefix + "detail/" + object)
        .viewPath(objectPrefix + "view/" + object)
        .downloadPath(objectPrefix + "download/" + object)
        .deletePath(objectPrefix + "delete/" + object)
        .lastModified(metadata.lastModified().toLocalDateTime())
        .fileSize(metadata.size())
        .filename(getFileName(metadata.object()))
        .contentType(metadata.contentType())
        .userMetadata(metadata.userMetadata())
        .headers(metadata.headers());
    if (tags != null) {
      builder.tags(tags.get());
    }
    return builder.build();
  }

  @SneakyThrows
  @PostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
  @ResponseStatus(HttpStatus.CREATED)
  public ResponseEntity<List<FileResponse>> fileUpload(
      @RequestParam("classification") String classification,
      @RequestPart("files") List<MultipartFile> files) {
    minioHandler.ifPresent(handler -> handler.beforeUpload(classification, files));
    if (CollectionUtils.isEmpty(files)) {
      return ResponseEntity.status(HttpStatus.NO_CONTENT).build();
    }
    if (!properties.getClassifications().contains(classification)) {
      throw new IllegalArgumentException("classification is not config.");
    }
    List<FileResponse> responses = new ArrayList<>(files.size());
    for (MultipartFile file : files) {
      String fileId = UUID.randomUUID().toString().replace("-", "");
      String originalFilename = file.getOriginalFilename();
      if (originalFilename == null) {
        originalFilename = fileId;
      }
      StringBuilder fileName = new StringBuilder(fileId);
      if (originalFilename.contains(".")) {
        fileName.append(originalFilename.substring(originalFilename.lastIndexOf(".")));
      }
      Path source = Path.of(classification, fileName.toString());
      Multimap<String, String> userMetadata = ArrayListMultimap.create(1, 1);
      userMetadata.put("original_file_name", originalFilename);
      minioService.upload(source, file.getInputStream(), null, userMetadata);
      responses.add(
          buildFileResponse(minioService.getMetadata(source), minioService.getTags(source)));
    }
    AtomicReference<List<FileResponse>> reference = new AtomicReference<>(responses);
    minioHandler.ifPresent(handler -> reference.set(handler.afterUpload(responses)));
    return ResponseEntity.status(HttpStatus.CREATED)
        .contentType(MediaType.APPLICATION_JSON)
        .body(reference.get());
  }

  @GetMapping("/view/**")
  @ResponseStatus(HttpStatus.OK)
  public ResponseEntity<InputStreamResource> viewFile(HttpServletRequest request) {
    String object = request.getRequestURL().toString().split("/view/")[1];
    minioHandler.ifPresent(handler -> handler.beforeView(object));
    Path objectPath = Path.of(object);
    InputStream inputStream = minioService.get(objectPath);
    StatObjectResponse metadata = minioService.getMetadata(objectPath);
    return ResponseEntity.ok()
        .contentType(MediaType.parseMediaType(metadata.contentType()))
        .contentLength(metadata.size())
        .header("Content-disposition", "attachment; filename=" + getFileName(metadata.object()))
        .body(new InputStreamResource(inputStream));
  }

  @GetMapping("/download/**")
  @ResponseStatus(HttpStatus.OK)
  public ResponseEntity<InputStreamResource> downloadFile(HttpServletRequest request) {
    String object = request.getRequestURL().toString().split("/download/")[1];
    minioHandler.ifPresent(handler -> handler.beforeDownloadGetObject(object));
    Path objectPath = Path.of(object);
    InputStream inputStream = minioService.get(objectPath);
    StatObjectResponse metadata = minioService.getMetadata(objectPath);
    AtomicReference<StatObjectResponse> ref = new AtomicReference<>(metadata);
    minioHandler.ifPresent(
        handler -> {
          StatObjectResponse response = handler.beforeDownload(metadata);
          if (response == null) {
            log.warn("response can not be null.");
          } else {
            ref.set(response);
          }
        });
    return ResponseEntity.ok()
        .contentType(MediaType.APPLICATION_OCTET_STREAM)
        .contentLength(ref.get().size())
        .header("Content-disposition", "attachment; filename=" + getFileName(ref.get().object()))
        .body(new InputStreamResource(inputStream));
  }

  @DeleteMapping("/delete/**")
  @ResponseStatus(HttpStatus.NO_CONTENT)
  public ResponseEntity<Object> removeFile(HttpServletRequest request) {
    String object = request.getRequestURL().toString().split("/delete/")[1];
    minioHandler.ifPresent(handler -> handler.beforeDelete(object));
    Path objectPath = Path.of(object);
    minioService.remove(objectPath);
    minioHandler.ifPresent(handler -> handler.afterDelete(object));
    return ResponseEntity.noContent().build();
  }

  @GetMapping("/detail/**")
  @ResponseStatus(HttpStatus.OK)
  public ResponseEntity<FileResponse> getFileDetail(HttpServletRequest request) {
    String object = request.getRequestURL().toString().split("/detail/")[1];
    minioHandler.ifPresent(handler -> handler.beforeGetDetail(object));
    Path objectPath = Path.of(object);
    StatObjectResponse metadata = minioService.getMetadata(objectPath);
    FileResponse response = buildFileResponse(metadata, minioService.getTags(objectPath));
    AtomicReference<FileResponse> reference = new AtomicReference<>(response);
    minioHandler.ifPresent(handler -> reference.set(handler.afterGetDetail(response)));
    return ResponseEntity.ok().contentType(MediaType.APPLICATION_JSON).body(reference.get());
  }
}

MinioHandler

因为这是一个嵌入式的文件服务,在进行文件操作的时候,不同的项目可能需要做一些自定义操作,那么我们需要提供一些扩展点,这也是软件设计的原则之一:对扩展开放,对修改关闭。当然,这个扩展点可提供也可不提供,具体实现可以根据自己的团队规范进行设计。

/**
 * @author <a href="mailto:lin-mt@outlook.com">lin-mt</a>
 */
public interface MinioHandler {

  default void beforeUpload(String classification, List<MultipartFile> files) {}

  default List<FileResponse> afterUpload(List<FileResponse> responses) {
    return responses;
  }

  default void beforeView(String object) {}

  default void beforeDownloadGetObject(String object) {}

  default StatObjectResponse beforeDownload(StatObjectResponse response) {
    return response;
  }

  default void beforeDelete(String object) {}

  default void afterDelete(String object) {}

  default void beforeGetDetail(String object) {}

  default FileResponse afterGetDetail(FileResponse response) {
    return response;
  }
}

MinioService

MinioService 主要是对 MinioClient 的常用 API 的简单封装。

/**
 * @author <a href="mailto:lin-mt@outlook.com">lin-mt</a>
 */
@Service
@AllArgsConstructor
@ConditionalOnBean({MinioClient.class, MinioProperties.class})
public class MinioService {

  private final MinioClient minioClient;
  private final MinioProperties properties;

  @SneakyThrows
  public Tags getTags(Path path) {
    GetObjectTagsArgs args =
        GetObjectTagsArgs.builder()
            .bucket(properties.getBucketName())
            .object(path.toString())
            .build();
    return minioClient.getObjectTags(args);
  }

  @SneakyThrows
  public InputStream get(Path path) {
    GetObjectArgs args =
        GetObjectArgs.builder().bucket(properties.getBucketName()).object(path.toString()).build();
    return minioClient.getObject(args);
  }

  @SneakyThrows
  public StatObjectResponse getMetadata(Path path) {
    StatObjectArgs args =
        StatObjectArgs.builder().bucket(properties.getBucketName()).object(path.toString()).build();
    return minioClient.statObject(args);
  }

  @SneakyThrows
  public void upload(
      Path source,
      InputStream file,
      String contentType,
      Multimap<String, String> userMetadata,
      Multimap<String, String> headers,
      Map<String, String> tags) {
    PutObjectArgs.Builder builder =
        PutObjectArgs.builder().bucket(properties.getBucketName()).object(source.toString()).stream(
            file, file.available(), -1);
    if (userMetadata != null) {
      builder.userMetadata(userMetadata);
    }
    if (headers != null) {
      builder.headers(headers);
    }
    if (tags != null) {
      builder.tags(tags);
    }
    if (StringUtils.isNotBlank(contentType)) {
      builder.contentType(contentType);
    }
    minioClient.putObject(builder.build());
  }

  public void upload(
      Path source,
      InputStream file,
      String contentType,
      Multimap<String, String> userMetadata,
      Multimap<String, String> headers) {
    upload(source, file, contentType, userMetadata, headers, null);
  }

  public void upload(
      Path source, InputStream file, String contentType, Multimap<String, String> userMetadata) {
    upload(source, file, contentType, userMetadata, null);
  }

  public void upload(Path source, InputStream file, String contentType) {
    upload(source, file, contentType, null);
  }

  public void upload(Path source, InputStream file) {
    upload(source, file, null);
  }

  @SneakyThrows
  public void remove(Path source) {
    RemoveObjectArgs args =
        RemoveObjectArgs.builder()
            .bucket(properties.getBucketName())
            .object(source.toString())
            .build();
    minioClient.removeObject(args);
  }
}

健康状态检查

这个 Starter 提供了一个文件上传的服务,我们需要提供监控该服务的健康状态的信息,这部分可以自己增加健康状态的详细信息。

/**
 * @author <a href="mailto:lin-mt@outlook.com">lin-mt</a>
 */
@Component
@AllArgsConstructor
@ConditionalOnClass(ManagementContextAutoConfiguration.class)
public class MinioHealthIndicator implements HealthIndicator {

  private final MinioClient minioClient;
  private final MinioConfigurationProperties properties;

  @Override
  public Health health() {
    if (minioClient == null) {
      return Health.down().build();
    }
    String bucketName = properties.getBucketName();
    try {
      BucketExistsArgs args = BucketExistsArgs.builder().bucket(properties.getBucketName()).build();
      if (minioClient.bucketExists(args)) {
        return Health.up().withDetail("bucketName", bucketName).build();
      } else {
        return Health.down().withDetail("bucketName", bucketName).build();
      }
    } catch (Exception e) {
      return Health.down(e).withDetail("bucketName", bucketName).build();
    }
  }
}

至此,一个简易的开箱即用的文件服务插件就完成了。

示例

创建 accessKey

image.png

项目中引入 Starter

api project(path: ":quiet-spring-boot-starters:quiet-minio-spring-boot-starter", configuration: "default")

application.yml 配置 Minio

quiet:
  minio:
    bucket-name: ${spring.application.name}
    access-key: 65mtumFyO3xMpUyP
    secret-key: sXBTjKmCtWf8iwOiy8Uw3fCOhe8ibuGV
    object-prefix: http://localhost:8080/doc
    classifications:
      - api/remark

效果图

文件上传

image.png

服务状态

image.png

github.com/lin-mt/quie…

源码:quiet-spring-boot-starters/quiet-minio-spring-boot-starter

示例项目:quiet-doc

下一篇:

实现在 Markdown 编辑器中图片的上传和缩放