Confd改造

ForKazmodon
1,328 阅读4分钟

问题

  • 一次版本发布会修改若干个key，导致confd的watchProcessor触发多次的文件更新。
  • 配置中心的配置发布会更新多个key（除了配置内容的key外，还包含配置内容签名、版本号等于配置相关的key）。如果配置发布以watch版本号的变更为触发条件的话，万一版本号更新后，配置内容没有更新拉取的配置将会是老配置。
  • 目前容器化的部署是一个node部署一个consul client，node部署的每个confd pod都会去连接这个consul client作为后台存储。这里的鉴权如何去做。

配置中心介绍

用户需要提供两类配置文件给confd

1、confd的系统配置文件：

func init() {
	flag.StringVar(&authToken, "auth-token", "", "Auth bearer token to use")
	flag.StringVar(&backend, "backend", "etcd", "backend to use")
	flag.BoolVar(&basicAuth, "basic-auth", false, "Use Basic Auth to authenticate (only used with -backend=etcd)")
	flag.StringVar(&clientCaKeys, "client-ca-keys", "", "client ca keys")
	flag.StringVar(&clientCert, "client-cert", "", "the client cert")
	flag.StringVar(&clientKey, "client-key", "", "the client key")
	flag.StringVar(&confdir, "confdir", "/etc/confd", "confd conf directory")
	flag.StringVar(&configFile, "config-file", "", "the confd config file")
	flag.IntVar(&interval, "interval", 600, "backend polling interval")
	flag.BoolVar(&keepStageFile, "keep-stage-file", false, "keep staged files")
	flag.StringVar(&logLevel, "log-level", "", "level which confd should log messages")
	flag.Var(&nodes, "node", "list of backend nodes")
	flag.BoolVar(&noop, "noop", false, "only show pending changes")
	flag.BoolVar(&onetime, "onetime", false, "run once and exit")
	flag.StringVar(&prefix, "prefix", "", "key path prefix")
	flag.BoolVar(&printVersion, "version", false, "print version and exit")
	flag.StringVar(&scheme, "scheme", "http", "the backend URI scheme for nodes retrieved from DNS SRV records (http or https)")
	flag.StringVar(&srvDomain, "srv-domain", "", "the name of the resource record")
	flag.StringVar(&srvRecord, "srv-record", "", "the SRV record to search for backends nodes. Example: _etcd-client._tcp.example.com")
	flag.BoolVar(&syncOnly, "sync-only", false, "sync without check_cmd and reload_cmd")
	flag.StringVar(&authType, "auth-type", "", "Vault auth backend type to use (only used with -backend=vault)")
	flag.StringVar(&appID, "app-id", "", "Vault app-id to use with the app-id backend (only used with -backend=vault and auth-type=app-id)")
	flag.StringVar(&userID, "user-id", "", "Vault user-id to use with the app-id backend (only used with -backend=value and auth-type=app-id)")
	flag.StringVar(&table, "table", "", "the name of the DynamoDB table (only used with -backend=dynamodb)")
	flag.StringVar(&username, "username", "", "the username to authenticate as (only used with vault and etcd backends)")
	flag.StringVar(&password, "password", "", "the password to authenticate with (only used with vault and etcd backends)")
	flag.StringVar(&destDir, "dest-dir", "", "write config file dir")
	flag.BoolVar(&watch, "watch", false, "enable watch support")
	flag.StringVar(&consulToken, "consul-token", "", "consul-token")
}

config.conf交代了confd的运行内容，主要的配置项：

  • backend：存储后端的类型，例如：consul
  • nodes：后端节点的地址
  • prefix：存储的前缀。在获取变量时需要perfix+key才能找到存储在后端的变量
  • confdir：告诉需要被更新的template的存放位置

2、template配置文件

[template]
prefix = "/myapp"
src = "nginx.tmpl"
dest = "/tmp/myapp.conf"
owner = "nginx"
mode = "0644"
keys = [
  "/subdomain",
  "/upstream",
]
check_cmd = "/usr/sbin/nginx -t -c {{.src}}"
reload_cmd = "/usr/sbin/service nginx reload"

template.toml配置模板文件交代了confd如何更新配置文件：

  • prefix：后端存储变量的前缀
  • src：tmpl文件，配置文件的模板文件（用占位符表达配置文件的内容，最后用拉取的k/v替换占位符，生成应用程序使用的配置文件）
  • dest：替换完成后的配置文件存放的路径，即应用程序读取配置的路径。
  • check_cmd：除了文件更新，confd支持一些shell命令的执行
  • reload_cmd：

配置中心的实现有两种方式，好多大厂都用以变量为粒度来维护配置。在代码中用application.yml.tmpl文件描述应用的配置文件，占位符表述的值会被存储到像consul、etcd等k/v存储中，并提供可以添加、修改、删除配置变量的管理端。每次发布会拉去最新的key值进行替换占位符得到应用程序使用的配置文件application.yml。

另一种是以配置文件为粒度来维护配置，开发需要将配置文件的整个内容发布到配置中心，k/v存储存储的是配置文件整个内容作为一个value，更新配置文件可以用value完成更新。

原生的配置对现有的场景不太满足。一般一个应用程序会有多个配置文件，而且发布/上线会修改多个配置文件。还有为了保证拉取最新的配置文件或是为了防止配置被篡改，存储后端会存储对应配置的签名。通过为配置添加多个key(这个key可以理解为配置文件的描述信息，包括：配置的发布版本、回滚、服务的节点、发布结果钉钉通知地址、签名)

[template]
keys = [
  "config",
  "template",
  "md5",
  "release",
  "deploy_nodes",
  "notify"
]
dest_dir = "/home/work/www/app/conf"
prefix = "group/app"
back_prefix = "/home/work/confd/data/.bak/app/{getValue release}/"
deploy_nodes = "{getValue deploy_nodes}"
module = "group:app"
notify = "{getValue notify}"
health_check_interval = "30s"
health_check_script = "ps -ef | grep 'confd' | grep -v 'grep'"
release = "{getValue release}"

[[files]]
key="config"
dst=""
avatar
文章
阅读
粉丝