一、背景
-
管理工具
名字： cluster manager for Apache doris
简介：Cluster Manager for Apache Doris（以下简称 Doris Manager）是 SelectDB 推出的管理运维 Apache Doris、SelectDB Doris 集群的工具。
地址：
http://172.16.70.88:8004/3/operation/overview
账号
root / 123456
文档
docs.selectdb.com/docs/enterp…
agent包
暂时无法在飞书文档外展示此内容
agent安装
# 传输
# 解压
tar zxvf /opt/doris/manager-agent-24.1.3-x64-bin.tar.gz
# 启动
sh /opt/doris/manager-agent-24.1.3-x64-bin/bin/start.sh
# agent进程常驻机制是否生效
crontab -e
# 解压
tar zxvf /data/doris/manager-agent-24.1.3-x64-bin.tar.gz
# 启动
sh ~/manager-agent-24.1.3-x64-bin/bin/start.sh
# agent进程常驻机制是否生效
crontab -e
2. ## 测试机器
# 徐迟的3台主机
172.20.28.71
172.20.28.72
172.20.28.73
# ELK池子的三台主机用于测试
172.21.27.30
172.21.27.31
172.21.27.29
# 已经部署的一个单点doris
mysql -uroot -P9030 -h172.21.240.67
MySQL [demo]> show frontends;
+-----------------------------------------+---------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
| Name | Host | EditLogPort | HttpPort | QueryPort | RpcPort | ArrowFlightSqlPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastStartTime | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------------+---------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
| fe_7f3b1df7_93b8_4d39_af68_8b87925fe3b3 | 172.21.240.67 | 9010 | 8030 | 9030 | 9020 | -1 | FOLLOWER | true | 278090002 | true | true | 188010 | 2024-12-23 11:54:49 | 2024-12-27 15:55:21 | true | | doris-3.0.3-rc04-62a58bff4c | Yes |
+-----------------------------------------+---------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
1 row in set (0.02 sec)
MySQL [demo]> show backends;
+-----------+---------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
| BackendId | Host | HeartbeatPort | BePort | HttpPort | BrpcPort | ArrowFlightSqlPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | TabletNum | DataUsedCapacity | TrashUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag | ErrMsg | Version | Status | HeartbeatFailureCounter | NodeRole | CpuCores | Memory |
+-----------+---------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
| 10101 | 172.21.240.67 | 9050 | 9060 | 8040 | 8060 | -1 | 2024-12-23 11:58:15 | 2024-12-27 15:55:21 | true | false | 65 | 0.000 | 0.000 | 2.777 TB | 2.928 TB | 5.17 % | 5.17 % | 0.000 | {"location" : "default"} | | doris-3.0.3-rc04-62a58bff4c | {"lastSuccessReportTabletsTime":"2024-12-27 15:55:18","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false,"isActive":true,"currentFragmentNum":0,"lastFragmentUpdateTime":1735286072757} | 0 | mix | 16 | 62.76 GB |
+-----------+---------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
【徐迟的3台测试机】数据盘全在 /opt 下
二、功能测试项
-
FE/BE 进程宕机，自动拉齐
-
巡检能力
-
FE节点扩缩容
-
BE节点扩缩容
-
集群接管
-
集群部署
-
FE/BE 进程的日志查看
-
基础监控
-
集群信息摘要
-
doris版本升级
三、集群接管
#
root密码 Yumchina@2024
- 接管
- 填入信息
- 集群接管
- 执行中
- 观察
四、集群部署
- 主机注册
- FE配置
- be节点配置
- 执行部署
- 部署完成
# 复制连接信息
[{"type":"Doris Manager","address":"172.16.73.225:8004","user":"root","password":"Yumchina@2024","connectionStr":""},
{"type":"MySQL 协议","address":"172.21.27.30:9030","user":"admin","password":"JpSBz4_3ppYVP)U_p","connectionStr":"mysql -h172.21.27.30 -P9030 -u<user_name> -p<password>"},
{"type":"Http 协议","address":"172.21.27.30:8030","user":"admin","password":"JpSBz4_3ppYVP)U_p","connectionStr":"http://172.21.27.30:8030"}]
6. 集群信息
[root@xfzcvmmw185023 ~]# mysql -h172.21.27.30 -P9030 -uroot -p'Yumchina@2024'
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 41
Server version: 5.7.99 Doris version doris-3.0.3-rc04-62a58bff4c
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [(none)]> show frontends;
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
| Name | Host | EditLogPort | HttpPort | QueryPort | RpcPort | ArrowFlightSqlPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastStartTime | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
| fe_425ecbe3_b398_4fb7_b3be_7572082e8322 | 172.21.27.30 | 9010 | 8030 | 9030 | 9020 | -1 | FOLLOWER | true | 56337720 | true | true | 74 | 2024-12-27 16:09:49 | 2024-12-27 16:14:09 | true | | doris-3.0.3-rc04-62a58bff4c | Yes |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
1 row in set (0.02 sec)
MySQL [(none)]> show backends;
+-----------+--------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
| BackendId | Host | HeartbeatPort | BePort | HttpPort | BrpcPort | ArrowFlightSqlPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | TabletNum | DataUsedCapacity | TrashUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag | ErrMsg | Version | Status | HeartbeatFailureCounter | NodeRole | CpuCores | Memory |
+-----------+--------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
| 10003 | 172.21.27.30 | 9050 | 9060 | 8040 | 8060 | -1 | 2024-12-27 16:10:01 | 2024-12-27 16:14:09 | true | false | 22 | 0.000 | 0.000 | 919.884 GB | 984.176 GB | 6.53 % | 6.53 % | 0.000 | {"location" : "default"} | | doris-3.0.3-rc04-62a58bff4c | {"lastSuccessReportTabletsTime":"2024-12-27 16:13:24","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false,"isActive":true,"currentFragmentNum":0,"lastFragmentUpdateTime":1735287242808} | 0 | mix | 16 | 31.26 GB |
+-----------+--------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
1 row in set (0.00 sec)
MySQL [(none)]>
五、FE扩容
- fe扩容
- 添加节点
- 新增fe节点
- 执行
- 检查fe集群信息
MySQL [(none)]> show frontends;
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
| Name | Host | EditLogPort | HttpPort | QueryPort | RpcPort | ArrowFlightSqlPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastStartTime | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
| fe_d2b26d83_64f4_48fe_994b_5082a0f6ed25 | 172.21.27.31 | 9010 | 8030 | 9030 | 9020 | -1 | FOLLOWER | false | 56337720 | true | true | 492 | 2024-12-27 16:44:00 | 2024-12-27 16:45:09 | true | | doris-3.0.3-rc04-62a58bff4c | No |
| fe_425ecbe3_b398_4fb7_b3be_7572082e8322 | 172.21.27.30 | 9010 | 8030 | 9030 | 9020 | -1 | FOLLOWER | true | 56337720 | true | true | 493 | 2024-12-27 16:09:49 | 2024-12-27 16:45:09 | true | | doris-3.0.3-rc04-62a58bff4c | Yes |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
2 rows in set (0.02 sec)
6. 完成
六、FE缩容
- 选择FE缩容
- 选择节点
- 执行
- 验证
MySQL [(none)]> show frontends;
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
| Name | Host | EditLogPort | HttpPort | QueryPort | RpcPort | ArrowFlightSqlPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastStartTime | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
| fe_425ecbe3_b398_4fb7_b3be_7572082e8322 | 172.21.27.30 | 9010 | 8030 | 9030 | 9020 | -1 | FOLLOWER | true | 56337720 | true | true | 603 | 2024-12-27 16:09:49 | 2024-12-27 16:51:29 | true | | doris-3.0.3-rc04-62a58bff4c | Yes |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
1 row in set (0.00 sec)
MySQL [(none)]>
5. 完成
七、BE扩容
- BE 扩容
- 添加节点
- 新增BE节点
- 执行中
- 检查be集群信息
MySQL [(none)]> show frontends;
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
| Name | Host | EditLogPort | HttpPort | QueryPort | RpcPort | ArrowFlightSqlPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastStartTime | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
| fe_425ecbe3_b398_4fb7_b3be_7572082e8322 | 172.21.27.30 | 9010 | 8030 | 9030 | 9020 | -1 | FOLLOWER | true | 56337720 | true | true | 248 | 2024-12-27 16:09:49 | 2024-12-27 16:24:49 | true | | doris-3.0.3-rc04-62a58bff4c | Yes |
+-----------------------------------------+--------------+-------------+----------+-----------+---------+--------------------+----------+----------+-----------+------+-------+-------------------+---------------------+---------------------+----------+--------+-----------------------------+------------------+
1 row in set (0.02 sec)
MySQL [(none)]> show backends;
+-----------+--------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
| BackendId | Host | HeartbeatPort | BePort | HttpPort | BrpcPort | ArrowFlightSqlPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | TabletNum | DataUsedCapacity | TrashUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag | ErrMsg | Version | Status | HeartbeatFailureCounter | NodeRole | CpuCores | Memory |
+-----------+--------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
| 10003 | 172.21.27.30 | 9050 | 9060 | 8040 | 8060 | -1 | 2024-12-27 16:10:01 | 2024-12-27 16:24:29 | true | false | 18 | 404.000 B | 0.000 | 919.884 GB | 984.176 GB | 6.53 % | 6.53 % | 0.000 | {"location" : "default"} | | doris-3.0.3-rc04-62a58bff4c | {"lastSuccessReportTabletsTime":"2024-12-27 16:23:54","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false,"isActive":true,"currentFragmentNum":0,"lastFragmentUpdateTime":1735287801641} | 0 | mix | 16 | 31.26 GB |
| 10196 | 172.21.27.31 | 9050 | 9060 | 8040 | 8060 | -1 | 2024-12-27 16:22:57 | 2024-12-27 16:24:29 | true | false | 14 | 8.793 KB | 0.000 | 920.888 GB | 984.176 GB | 6.43 % | 6.43 % | 0.000 | {"location" : "default"} | | doris-3.0.3-rc04-62a58bff4c | {"lastSuccessReportTabletsTime":"2024-12-27 16:24:02","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false,"isActive":true,"currentFragmentNum":0,"lastFragmentUpdateTime":1735287801645} | 0 | mix | 16 | 31.26 GB |
+-----------+--------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
2 rows in set (0.01 sec)
- 完成
八、BE缩容
- 选择BE节点
- 点击缩容
- 执行
- 检查
MySQL [(none)]> show backends;
+-----------+--------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
| BackendId | Host | HeartbeatPort | BePort | HttpPort | BrpcPort | ArrowFlightSqlPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | TabletNum | DataUsedCapacity | TrashUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag | ErrMsg | Version | Status | HeartbeatFailureCounter | NodeRole | CpuCores | Memory |
+-----------+--------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
| 10003 | 172.21.27.30 | 9050 | 9060 | 8040 | 8060 | -1 | 2024-12-27 16:10:01 | 2024-12-27 16:49:09 | true | false | 32 | 9.188 KB | 0.000 | 919.883 GB | 984.176 GB | 6.53 % | 6.53 % | 0.000 | {"location" : "default"} | | doris-3.0.3-rc04-62a58bff4c | {"lastSuccessReportTabletsTime":"2024-12-27 16:49:03","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false,"isActive":true,"currentFragmentNum":0,"lastFragmentUpdateTime":1735289283974} | 0 | mix | 16 | 31.26 GB |
+-----------+--------------+---------------+--------+----------+----------+--------------------+---------------------+---------------------+-------+----------------------+-----------+------------------+-------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+----------+----------+
1 row in set (0.02 sec)
- 完成
九、doris进程故障与自动重启
FE的故障
-
集群FE状态
- 模拟FE进程故障
- 观察
BE的故障
- 集群BE状态
- 模拟BE进程故障
- 观察
十、巡检
十一、doris服务日志查看
FE 和 BE 的日志均可以看到
十二、FE节点的滚动重启
十三、BE节点的滚动重启
十四、doris版本升级
十五、数据库管理
-
SQL编辑器
-
检索分析
-
会话管理
-
查询审计
需要安装插件后开启
-
负载管理
-
数据操作
-
用户管理
-
导入
