现在，我们将尝试创建一个复制的statefulset应用程序。作为创建这篇博客的良好参考，我想归功于Kubernetes文档以及阿里云的一个例子。
配置复制
为了复制MariaDB应用程序，我们将创建一个由一个初始容器和一个应用程序容器组成的statefulset。这两个容器都是基于MariaDB镜像的。
init容器将在应用容器之前运行，我们将使用该容器来处理与复制有关的所有设置，主要是为特定的容器类型（主容器或副本）在特定的目录中加载特定文件的卷。在这个过程中使用的文件将从配置图中获得（为此我们需要挂载一个额外的卷），该配置图将容纳必要的主卷和副本的配置文件，这些文件将被挂载在/etc/mysql/conf.d/ ，配置文件的全局目录中，以及带有SQL语句的SQL文件将被挂载在docker-entrypoint-initdb.d ，这些语句将在容器的首次启动中执行。
应用MariaDB容器要使用那些带有微调配置的卷来进行复制，以及一个用于数据目录的持久卷。
配置文件看起来像这样。
# ConfigMap holding information about configuration files for primary/secondary and dockerinit
apiVersion: v1
kind: ConfigMap
metadata:
name: mariadb-configmap
data:
primary.cnf: |
[mariadb]
log-bin # enable binary logging
log-basename=my-mariadb # used to be independent of hostname changes (otherwise is in datadir/mysql)
replica.cnf: |
[mariadb]
log-basename=my-mariadb # used to be independent of hostname changes (otherwise is in datadir/mysql)
primary.sql: |
CREATE USER 'repluser'@'%' IDENTIFIED BY 'replsecret';
GRANT REPLICATION REPLICA ON *.* TO 'repluser'@'%';
CREATE DATABASE primary_db;
secondary.sql: |
# We have to know name of sts (`mariadb-sts`) and
# service `mariadb-service` in advance as an FQDN.
# No need to use master_port
CHANGE MASTER TO
MASTER_HOST='mariadb-sts-0.mariadb-service.default.svc.cluster.local',
MASTER_USER='repluser',
MASTER_PASSWORD='replsecret',
MASTER_CONNECT_RETRY=10;
# Secret holds information about root password
---
apiVersion: v1
kind: Secret
metadata:
name: mariadb-secret
type: Opaque
data:
mariadb-root-password: c2VjcmV0 # echo -n 'secret'|base64
# Headless service
---
apiVersion: v1
kind: Service
metadata:
name: mariadb-service
labels:
app: mariadb
spec:
ports:
- port: 3306
name: mariadb-port
clusterIP: None
selector:
app: mariadb
# Statefulset
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mariadb-sts
spec:
serviceName: "mariadb-service"
replicas: 3
selector:
matchLabels:
app: mariadb
template:
metadata:
labels:
app: mariadb
spec:
initContainers:
- name: init-mariadb
image: mariadb
imagePullPolicy: Always
command:
- bash
- "-c"
- |
set -ex
echo 'Starting init-mariadb';
# Check config map to directory that already exists
# (but must be used as a volume for main container)
ls /mnt/config-map
# Statefulset has sticky identity, number should be last
[[ `hostname` =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
# Copy appropriate conf.d files from config-map to
# mariadb-config volume (emptyDir) depending on pod number
if [[ $ordinal -eq 0 ]]; then
# This file holds SQL for connecting to primary
cp /mnt/config-map/primary.cnf /etc/mysql/conf.d/server-id.cnf
# Create the users needed for replication on primary on a volume
# initdb (emptyDir)
cp /mnt/config-map/primary.sql /docker-entrypoint-initdb.d
else
# This file holds SQL for connecting to secondary
cp /mnt/config-map/replica.cnf /etc/mysql/conf.d/server-id.cnf
# On replicas use secondary configuration on initdb volume
cp /mnt/config-map/secondary.sql /docker-entrypoint-initdb.d
fi
# Add an offset to avoid reserved server-id=0 value.
echo server-id=$((3000 + $ordinal)) >> etc/mysql/conf.d/server-id.cnf
ls /etc/mysql/conf.d/
cat /etc/mysql/conf.d/server-id.cnf
volumeMounts:
- name: mariadb-config-map
mountPath: /mnt/config-map
- name: mariadb-config
mountPath: /etc/mysql/conf.d/
- name: initdb
mountPath: /docker-entrypoint-initdb.d
restartPolicy: Always
containers:
- name: mariadb
image: mariadb
ports:
- containerPort: 3306
name: mariadb-port
env:
# Using Secrets
- name: MARIADB_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mariadb-secret
key: mariadb-root-password
- name: MYSQL_INITDB_SKIP_TZINFO
value: "1"
# Mount volume from persistent volume claim
volumeMounts:
- name: datadir
mountPath: /var/lib/mysql/
- name: mariadb-config
mountPath: /etc/mysql/conf.d/
- name: initdb
mountPath: /docker-entrypoint-initdb.d
volumes:
- name: mariadb-config-map
configMap:
name: mariadb-configmap
#defaultMode: 0544
- name: mariadb-config
emptyDir: {}
- name: initdb
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: datadir
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 300M
测试复制
应用配置文件并观察pod的创建。
$ kubectl apply -f mariadb-sts-replication.yaml
configmap/mariadb-configmap created
secret/mariadb-secret created
service/mariadb-service created
statefulset.apps/mariadb-sts created
$ kubectl get pod -w
NAME READY STATUS RESTARTS AGE
mariadb-sts-0 1/1 Running 0 14s
mariadb-sts-1 1/1 Running 0 8s
mariadb-sts-2 0/1 Init:0/1 0 2s
mariadb-sts-2 0/1 PodInitializing 0 4s
mariadb-sts-2 1/1 Running 0 6s
要调试特定的pod/container，使用以下命令。
$ kubectl describe pod mariadb-sts-0
$ kubectl logs mariadb-sts-0 -c init-mariadb
在主目录上创建数据
$ kubectl exec -it mariadb-sts-0 -- mariadb -uroot -psecret
Defaulted container "mariadb" out of: mariadb, init-mariadb (init)
MariaDB [primary_db]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| primary_db |
| sys |
+--------------------+
5 rows in set (0.000 sec)
MariaDB [primary_db]> create table my_table (t int); insert into my_table values (5),(15),(25);
Query OK, 0 rows affected (0.031 sec)
Query OK, 3 rows affected (0.004 sec)
Records: 3 Duplicates: 0 Warnings: 0
检查复制体上的数据
$ kubectl exec -it mariadb-sts-2 -- mariadb -uroot -psecret
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| primary_db |
| sys |
+--------------------+
MariaDB [(none)]> use primary_db;
Database changed
MariaDB [primary_db]> show tables;
+----------------------+
| Tables_in_primary_db |
+----------------------+
| my_table |
+----------------------+
1 row in set (0.000 sec)
MariaDB [primary_db]> select * from my_table;
+------+
| t |
+------+
| 5 |
| 15 |
| 25 |
+------+
3 rows in set (0.000 sec)
扩大规模
$ kubectl scale sts mariadb-sts --replicas=4
statefulset.apps/mariadb-sts scaled
$ kubectl get pod -w
NAME READY STATUS RESTARTS AGE
mariadb-sts-0 1/1 Running 0 2m52s
mariadb-sts-1 1/1 Running 0 2m46s
mariadb-sts-2 1/1 Running 0 2m40s
mariadb-sts-3 0/1 Pending 0 0s
mariadb-sts-3 0/1 Pending 0 0s
mariadb-sts-3 0/1 Pending 0 2s
mariadb-sts-3 0/1 Init:0/1 0 2s
mariadb-sts-3 0/1 PodInitializing 0 5s
mariadb-sts-3 1/1 Running 0 7s
检查新创建的副本
$ kubectl exec -it mariadb-sts-3 -- mariadb -uroot -psecret
MariaDB [(none)]> use primary_db;
MariaDB [primary_db]> show tables;
+----------------------+
| Tables_in_primary_db |
+----------------------+
| my_table |
+----------------------+
1 row in set (0.000 sec)
MariaDB [primary_db]> select * from my_table;
+------+
| t |
+------+
| 5 |
| 15 |
| 25 |
+------+
3 rows in set (0.000 sec)
尝试在主服务器上插入新数据
MariaDB [primary_db]> insert into my_table values (40),(45);
检查副本mariadb-sts-3
MariaDB [primary_db]> select * from my_table;
+------+
| t |
+------+
| 5 |
| 15 |
| 25 |
| 40 |
| 45 |
+------+
5 rows in set (0.000 sec)
我们也可以检查PVC
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
datadir-mariadb-sts-0 Bound pvc-c1676027-6c75-473b-9b46-3d9a7d370fdc 300M RWO standard 31m
datadir-mariadb-sts-1 Bound pvc-7f969265-3d8f-4677-950b-271ea670321e 300M RWO standard 30m
datadir-mariadb-sts-2 Bound pvc-d116e494-5078-46ec-abcb-864fa8ae6b59 300M RWO standard 30m
datadir-mariadb-sts-3 Bound pvc-00aca08a-7a5e-459a-8ae1-a854c4171a27 300M RWO standard 28m
缩小规模
$ kubectl scale sts mariadb-sts --replicas=2
statefulset.apps/mariadb-sts scaled
$ kubectl get pod -w
mariadb-sts-3 1/1 Terminating 0 78s
mariadb-sts-3 0/1 Terminating 0 79s
mariadb-sts-3 0/1 Terminating 0 79s
mariadb-sts-3 0/1 Terminating 0 79s
mariadb-sts-2 1/1 Terminating 0 4m4s
mariadb-sts-2 0/1 Terminating 0 4m5s
mariadb-sts-2 0/1 Terminating 0 4m5s
mariadb-sts-2 0/1 Terminating 0 4m5s
请注意，缩减规模不会删除PVC。
总结
该博客展示了如何在K8s中执行MariaDB复制。欢迎大家在Zulip上讨论这个问题。
