Vearch源码阅读——部署

318 阅读1分钟

好奇vearch是怎么部署的,部署流程参考这篇文章

部署相关的代码在 tools/deployment/batch_deployment.go下面

首先我们来看几个工具函数

工具函数

  • runCommand:在其他机器远程执行命令

main函数

部署的命令如下, 分为deploy,start,status

#开始部署 并检查部署状态
readonly cur_dir=`dirname $(readlink -f "$0")`
./batch_deployment -dir=$cur_dir vearch deploy
./batch_deployment -dir=$cur_dir vearch start
./batch_deployment -dir=$cur_dir vearch status

这里需要注意,以上这些命令都是在一台机器上执行一遍即可,我们就称之为运维机器吧。

解析配置

第一步是解析配置, conf.json文件长这样:

{
  "user": "root",
  "password": "123456",
  "dir": "/home/vearch/deploy",
  "master": ["ip1", "ip2", "ip3"],
  "ps": ["ip4", "ip5", "ip6"],
  "router": ["ip7", "ip8", "ip9"],
  "port":"22",
  "copy":["lib"]
}

main函数读取conf.json文件,解析进DepConf类型的变量conf

bytes, err := ioutil.ReadFile(confDir + "/conf.json")
conf = DepConf{}
err = json.Unmarshal(bytes, &conf)

DepConf类型长这样:

type DepConf struct {
    User     string   `json:"user"`
    Password string   `json:"password"`
    Port     string   `json:"port"`
    Dir      string   `json:"dir"`
    Master   []string `json:"master"`
    Ps       []string `json:"ps"`
    Router   []string `json:"router"`
    Copy     []string `json:"copy"`
}

可以看到,里面有master,ps,router,的ip数组,main函数用一个map把这些ip为key的value都设为了true

ipMap := make(map[string]bool)
for _, ip := range conf.Master {
        ipMap[ip] = true
}

for _, ip := range conf.Ps {
        ipMap[ip] = true
}

for _, ip := range conf.Router {
        ipMap[ip] = true
}

我们先看deploy

./batch_deployment -dir=$cur_dir vearch deploy

因为args[1]是deploy,所以进入deploee函数

deploy(ipMap, config, binPath)

deploy()

func deploy(ipMap map[string]bool, config *ssh.ClientConfig, binPath string) {
    now := time.Now()
    wg := sync.WaitGroup{}
    for ip, _ := range ipMap {
        wg.Add(1)
        go func(ip string) {
            client := createClient(ip, conf, config)
            defer client.Close()
            copyFile(client, confDir+"/config.toml", conf.Dir+"/config.toml")
            copyFile(client, confDir+"/start.sh", conf.Dir+"/start.sh")
            copyFile(client, confDir+"/stop.sh", conf.Dir+"/stop.sh")
            copyFile(client, binPath, conf.Dir+"/vearch")
            for i := range conf.Copy {
                    copyDir(client, conf.Copy[i], path.Join(conf.Dir, "copy"))
            }

            runCommand(client, "chmod u+x "+conf.Dir+"/start.sh")
            runCommand(client, "chmod u+x "+conf.Dir+"/stop.sh")
            runCommand(client, "chmod u+x "+conf.Dir+"/vearch")
            wg.Done()
        }(ip)
    }
    wg.Wait()

    log.Println("deploy all ok use time ", time.Now().Sub(now))
}

函数不长,所谓部署其实就是遍历所有ipMap里的ip,把conf.tomlstart.shstop.sh以及vearch的bin包拷贝到这些机器上

start()

经过deploy(),配置文件、运行脚本、以及bin包都被拷贝到了对应的机器上,接下来就是启动了,启动是在运维机器上执行下面这句

./batch_deployment -dir=$cur_dir vearch start

main()函数解析参数执行start()函数

func start(config *ssh.ClientConfig, binPath string) {

    for _, ip := range conf.Master {
            client := createClient(ip, conf, config)
            runCommand(client, "cd "+conf.Dir+"; ./start.sh master \n")
            client.Close()
    }

    for _, ip := range conf.Router {
            client := createClient(ip, conf, config)
            runCommand(client, "cd "+conf.Dir+"; ./start.sh router \n")
            client.Close()
    }

    for _, ip := range conf.Ps {
            client := createClient(ip, conf, config)
            runCommand(client, "cd "+conf.Dir+"; ./start.sh ps \n")
            client.Close()
    }
}

函数很短,就是根据conf中对应的 master、router、ps 列表,分别远程执行启动命令

sh start.sh master
sh start.sh router
sh start.sh ps

start.sh的内容可以在项目里搜索看到,核心就是通过下面这句执行bin包

nohup $BasePath/vearch -conf $BasePath/config.toml $1

翻译一下,master、router、ps分别执行下面三句

nohup $BasePath/vearch -conf $BasePath/config.toml master
nohup $BasePath/vearch -conf $BasePath/config.toml router
nohup $BasePath/vearch -conf $BasePath/config.toml ps

全文完,下一篇研究研究这三者启动都做了什么吧!