Kubernetes configmap 热更新spring-boot应用

2,399 阅读6分钟

一. 背景

在生产环境中,一个app服务处于工作中的状态,此时如果需要临时替换某些配置信息, 如数据库连接池大小,链接等,一般都需要在源码端修改配置信息,然后重新部署,可想而知,这样做会影响用户或者其他与其对接的app服务。有没有可以在不重启应用的情况下就可以修改配置信息呢?答案是肯定。 达到热更新的方案还是很多的,如果是java体系,spring-cloud已经为我们提供的一套方案,即spring-cloud-config,本文就不做介绍了。 spring-cloud-kubernetes是springcloud官方推出的开源项目,用于将Spring Cloud和Spring Boot应用运行在kubernetes环境,并且提供了通用的接口来调用kubernetes服务,最终是借用了kubernetes自己的服务发现功能,当configmap发生变动时可通知相关服务更新配置。

二. 实践

1.准备

1.1 应用源码

准备一个spring-boot应用,目录如下:

image.png

(1)添加maven依赖

通过maven创建名为springcloudk8sreloadconfigdemo的springboot工程,pom.xml内容如下,要注意的是新增了依赖spring-cloud-starter-kubernetes-config、spring-boot-actuator、spring-boot-actuator-autoconfigure,

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.7.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.ftlcloud</groupId>
    <artifactId>demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>demo</name>
    <description>Demo project for Spring Boot</description>


    <properties>
        <spring-boot.version>2.2.7.RELEASE</spring-boot.version>
        <spring-cloud.version>Hoxton.SR4</spring-cloud.version>
        <maven-checkstyle-plugin.failsOnError>false</maven-checkstyle-plugin.failsOnError>
        <maven-checkstyle-plugin.failsOnViolation>false</maven-checkstyle-plugin.failsOnViolation>
        <maven-checkstyle-plugin.includeTestSourceDirectory>false</maven-checkstyle-plugin.includeTestSourceDirectory>
        <maven-compiler-plugin.version>3.5</maven-compiler-plugin.version>
        <maven-deploy-plugin.version>2.8.2</maven-deploy-plugin.version>
        <maven-failsafe-plugin.version>2.18.1</maven-failsafe-plugin.version>
        <maven-surefire-plugin.version>2.21.0</maven-surefire-plugin.version>
        <fabric8.maven.plugin.version>3.5.37</fabric8.maven.plugin.version>
        <springcloud.kubernetes.version>1.0.1.RELEASE</springcloud.kubernetes.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-actuator-autoconfigure</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-kubernetes-config</artifactId>
            <version>${springcloud.kubernetes.version}</version>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>


        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <version>${spring-boot.version}</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <!--skip deploy -->
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-deploy-plugin</artifactId>
                <version>${maven-deploy-plugin.version}</version>
                <configuration>
                    <skip>true</skip>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${maven-surefire-plugin.version}</version>
                <configuration>
                    <skipTests>true</skipTests>
                    <!-- Workaround for https://issues.apache.org/jira/browse/SUREFIRE-1588 -->
                    <useSystemClassLoader>false</useSystemClassLoader>
                </configuration>
            </plugin>
            <plugin>
                <groupId>io.fabric8</groupId>
                <artifactId>fabric8-maven-plugin</artifactId>
                <version>${fabric8.maven.plugin.version}</version>
                <executions>
                    <execution>
                        <id>fmp</id>
                        <goals>
                            <goal>resource</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <profiles>
        <profile>
            <id>kubernetes</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>io.fabric8</groupId>
                        <artifactId>fabric8-maven-plugin</artifactId>
                        <version>${fabric8.maven.plugin.version}</version>
                        <executions>
                            <execution>
                                <id>fmp</id>
                                <goals>
                                    <goal>resource</goal>
                                    <goal>build</goal>
                                </goals>
                            </execution>
                        </executions>
                        <configuration>
                            <enricher>
                                <config>
                                    <fmp-service>
                                        <type>NodePort</type>
                                    </fmp-service>
                                </config>
                            </enricher>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>
</project>

(2)src\main\resources创建名为bootstrap.yml的文件

如下:

server:
  port: 8080
  
management:
  endpoint:
    restart:
      enabled: true
    health:
      enabled: true
    info:
      enabled: true
      
spring:
  application:
    name: ftl-cloud-app-demo
  profiles:
    active: staging
  cloud:
    kubernetes:
      reload:
        #自动更新配置的开关设置为打开
        enabled: true
        #更新配置信息的模式是主动拉取
        mode: polling
        #主动拉取的间隔时间是500毫秒
        period: 1000
      config:
        sources:
          - name: ${spring.application.name}
            namespace: import-staging

可见新增了配置项spring.cloud.kubernetes.reload和spring.cloud.kubernetes.config,前者用于开启自动更新配置,执行更新模式为500毫秒拉取一次,后者指定配置来源于kubernetes的哪个namespace下的哪个configmap。

(3)java源码

需要应用加载的信息如下

## kafka topic相关配置
kafka:
  topic:
    group-id: receiver-group
    topic-name:
      - topic1
      - topic2
      - topic3

KafkaTopicProperties.java :

@RefreshScope
@Data
@Component
@ConfigurationProperties("kafka.topic")
public class KafkaTopicProperties {

    private String groupId;
    private String[] topicName;

    public String getGroupId() {
        return groupId;
    }

    public void setGroupId(String groupId) {
        this.groupId = groupId;
    }

    public String[] getTopicName() {
        return topicName;
    }

    public void setTopicName(String[] topicName) {
        this.topicName = topicName;
    }
}

TestController.java


@RestController
public class TestController {

    @Autowired
    private KafkaTopicProperties properties;

    @GetMapping("/get")
    @ResponseBody
    public Object test(){
        return  properties.getTopicName();
    }

}

DemoApplication.java


@SpringBootApplication
public class DemoApplication {
    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }
}

1.2 configmap创建

即application.yml内容,本文以kafka主题简单配置为例,如下:

## kafka topic相关配置
kafka:
  topic:
    group-id: receiver-group
    topic-name:
      - topic1
      - topic2
      - topic3

(1)通过rancher创建configmap

如果k8s通过rancher管理的话,可以通过rancher创建此configmap. 首先,点击资源-配置映射如下:

image.png
添加一个配置映射,名称就取我们的demo名称ftl-cloud-app-demo,注意选择命名空间。
image.png
点击保存即可创建成功一个名称为ftl-cloud-app-demo的configmap

(2)通过kubectl创建configmap

1)在kubernetes环境新建名为ftl-cloud-app-demo.yml的文件,内容如下:

apiVersion: v1
data:
  application.yml: |-
    ## kafka topic相关配置
    kafka:
      topic:
        group-id: receiver-group
        topic-name:
          - topic1
          - topic2
          - topic3
kind: ConfigMap
metadata:
  name: ftl-cloud-app-demo
  namespace: import-staging

保存后执行:

kubectl apply -f ftl-cloud-app-demo.yml

即可生成名称为ftl-cloud-app-demo的configmap。

2.授权

2.1 角色的创建

在命令行执行:

kubectl create role role-configmap-reader --verb=get,list,watch --resource=pods,configmaps --dry-run -o yaml > role-configmap-reader.yml

执行后,会生成role-configmap-reader.yml的文件,编辑使之如下:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: import-staging
  name: role-configmap-reader
rules:
  - apiGroups: [""]
    resources: ["pods","configmaps"]
    verbs: ["get", "watch", "list"]

保存,执行

kubectl apply -f role-configmap-reader.yml

即可创建一个拥有"get", "watch", "list" -> "pods","configmaps"权限的角色

2.2 创建ServiceAccount

在命令行执行:

kubectl create serviceaccount qianxunke  -o yaml > user-qianxunke.yml

在当前目录下会生成user-qianxunke.yml的文件,我们需要编辑它,改变默认命名空间(default),如下:

image.png
修改好命名空间后,保存执行

kubectl apply -f user-qianxunke.yml

2.3 绑定Role和ServiceAccount

kubectl create rolebinding rolebinding-pods-configmap-reade- --role=pods-reader --user=qianxunke --dry-run -o yaml > rolebinding-pods-configmap-reader.yaml

执行成功会在当前目录生成rolebinding-pods-configmap-reader.yaml,编辑其,使之如下:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: rolebinding-pods-configmap-reader
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: role-configmap-reader
subjects:
  - kind: ServiceAccount
    name: qianxunke
    namespace: import-staging

保存,执行:

kubectl apply -f rolebinding-pods-configmap-reader.yaml

3.验证

3.1项目部署

这个根据自己的环境,基本步骤就是:编译镜像,推送镜像到镜像仓库,在k8s中创建模版,执行。 这里我们创建一个无状态的deployment来部署: 内容如下:

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: ftl-cloud-app-demo
spec:
  selector:
    matchLabels:
      app: ftl-cloud-app-demo
  replicas: 2 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: ftl-cloud-app-demo
    spec:
      containers:
      - name: ftl-cloud-app-demo
        image: 填写镜像地址
        ports:
        - containerPort: 8080
      serviceAccount: qianxunke
      serviceAccountName: qianxunke

注意:serviceAccount,serviceAccountName的值为上文授权的用户名 保存以上内容在ftl-cloud-app-demo-deployment.yml,然后执行

kubectl apply -f ./ftl-cloud-app-demo-deployment.yml

不出意外,程序运行正常。 如果结合rancher部署应用,只需修改已有的deployment,在相应位置添加serviceAccount,serviceAccountName即可。

3.2 验证

在postman输入: http://自己的域名或IP/get 输入如下:

image.png
修改configmap

kubectl edit configmap ftl-cloud-app-demo

image.png
再在postman点击send,即可马得到如下结果:
image.png

4.补充说明

之前的bootstrap.yml中和同步配置相关的参数,如下图红框所示:

image.png
polling是定时拉取的模式,间隔时间太大会影响实时性,太小又导致请求过于密集,所以spring-cloud-kubernetes框架还给出了另一种模式:事件通知,对应的值是event; 设置事件通知模式的步骤:先将mode的值从polling改为event,再将period参数注释掉(该参数只在mode等于polling时有效),修改后如下:
image.png