查看开始的Ceph集群状态
可以看到当前有三个mon
[root@node1 ~]# ceph status
cluster:
id: 9e7b59a6-c3ee-43d4-9baf-60d5bb05484a
health: HEALTH_OK
services:
mon: 3 daemons, quorum node3,node4,node1 (age 6h)
mgr: node1(active, since 6h), standbys: node4, node3
osd: 3 osds: 3 up (since 6h), 3 in (since 6h)
data:
pools: 2 pools, 33 pgs
objects: 2 objects, 769 KiB
usage: 82 MiB used, 477 GiB / 477 GiB avail
pgs: 33 active+clean
在另外两个需要部署mon的节点上操作
因为mon一般为奇数，因此我们再加两个mon
安装软件包
# 安装epel源
dnf install epel-release
# 安装ceph相关软件包,在node2，和node5上添加mon
ceph-deploy install --no-adjust-repos --nogpgcheck node2 node5
添加mon
在开始执行ceph-deploy的节点上操作添加
# 拷贝下密钥
ceph-deploy admin node2 node5
# 部署mon
ceph-deploy mon add node2
ceph-deploy mon add node5
查看当前ceph集群状态
[root@node1 cephcluster]# ceph status
cluster:
id: 9e7b59a6-c3ee-43d4-9baf-60d5bb05484a
health: HEALTH_OK
services:
mon: 5 daemons, quorum node3,node4,node1,node2,node5 (age 0.561433s)
mgr: node1(active, since 7h), standbys: node4, node3
osd: 3 osds: 3 up (since 7h), 3 in (since 7h)
data:
pools: 2 pools, 33 pgs
objects: 2 objects, 769 KiB
usage: 82 MiB used, 477 GiB / 477 GiB avail
pgs: 33 active+clean
验证ceph-mon状态
ceph quorum_status --format json-pretty
遇到的错误
错误一
[root@node1 cephcluster]# ceph-deploy mon add node2
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (2.1.0): /usr/local/bin/ceph-deploy mon add node2
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] subcommand : add
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf object at 0x7f8f5d7b98e0>
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] func : <function mon at 0x7f8f5d7b4c10>
[ceph_deploy.cli][INFO ] address : None
[ceph_deploy.cli][INFO ] mon : ['node2']
[ceph_deploy.mon][INFO ] ensuring configuration of new mon host: node2
[ceph_deploy.admin][DEBUG ] Pushing admin keys and conf to node2
[node2][DEBUG ] connected to host: node2
[ceph_deploy.mon][DEBUG ] Adding mon to cluster ceph, host node2
[ceph_deploy.mon][DEBUG ] using mon address by resolving host: 192.168.0.101
[ceph_deploy.mon][DEBUG ] detecting platform for host node2 ...
[node2][DEBUG ] connected to host: node2
[ceph_deploy.mon][INFO ] distro info: rocky 9.2 blue onyx
[node2][DEBUG ] determining if provided host has same hostname in remote
[node2][DEBUG ] adding mon to node2
[node2][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-node2/done
[node2][DEBUG ] done path does not exist: /var/lib/ceph/mon/ceph-node2/done
[node2][INFO ] creating keyring file: /var/lib/ceph/tmp/ceph-node2.mon.keyring
[node2][INFO ] Running command: ceph --cluster ceph mon getmap -o /var/lib/ceph/tmp/ceph.node2.monmap
[node2][WARNIN] got monmap epoch 1
[node2][INFO ] Running command: ceph-mon --cluster ceph --mkfs -i node2 --monmap /var/lib/ceph/tmp/ceph.node2.monmap --keyring /var/lib/ceph/tmp/ceph-node2.mon.keyring --setuser 167 --setgroup 167
[node2][INFO ] unlinking keyring file /var/lib/ceph/tmp/ceph-node2.mon.keyring
[node2][INFO ] Running command: systemctl enable ceph.target
[node2][INFO ] Running command: systemctl enable ceph-mon@node2
[node2][WARNIN] Created symlink /etc/systemd/system/ceph-mon.target.wants/ceph-mon@node2.service → /usr/lib/systemd/system/ceph-mon@.service.
[node2][INFO ] Running command: systemctl start ceph-mon@node2
[node2][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.node2.asok mon_status
[node2][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[node2][WARNIN] node2 is not defined in `mon initial members`
[node2][WARNIN] monitor node2 does not exist in monmap
[node2][WARNIN] neither `public_addr` nor `public_network` keys are defined for monitors
[node2][WARNIN] monitors may not be able to form quorum
[node2][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.node2.asok mon_status
[node2][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[node2][WARNIN] monitor: mon.node2, might not be running yet
解决办法
编辑ceph.conf,内容如下
再执行下面的命令
[root@node1 cephcluster]# ceph-deploy --overwrite-conf config push node2 node5
