MongoDB分片集群搭建

LFly
766 阅读6分钟

1.环境

服务器规划

  • Centos Linux release 7.3.1611 (Core)       (查看系统版本:cat /etc/redhat-release)
  • MongoDB 4.0.6      (下载地址:www.mongodb.com/download-ce…)
  • shard分片主机
    • shard1: IP: 172.29.x.31 端口: 27001
    • shard2: IP: 172.29.x.33 端口: 27002
    • shard3: IP: 172.29.x.35 端口: 27003
    • shard4: IP: 172.29.x.37 端口: 27004
    • shard5: IP: 172.29.x.39 端口: 27005
  • configsrv主机：
    • IP: 172.29.x.35 端口: 21000
    • IP: 172.29.x.37 端口: 21000
    • IP: 172.29.x.39 端口: 21000
  • mongos主机：
    • IP: 172.29.5.35 端口: 20000
    • IP: 172.29.5.37 端口: 20000
    • IP: 172.29.5.39 端口: 20000

  • 虚机登录用户名/密码：

    • root/root
    • mongodb/mongodb

  • mongodb Root用户 mongos登录 分片登录

    • admin/admin
    • shard1/shard1
    • shard2/shard2
    • shard3/shard3
    • shard4/shard4
    • shard5/shard5

  • mongodb监控账号

    • monitor_mongodb_mongos/Mmongos
    • monitor_mongodb_shard1/Mshard1
    • monitor_mongodb_shard2/Mshard2
    • monitor_mongodb_shard3/Mshard3
    • monitor_mongodb_shard4/Mshard4
    • monitor_mongodb_shard5/Mshard5

2.查看挂载盘

  • 查看系统盘 df -h
  • 磁盘规划 P/A/S(/config/mongos)

3.建立文件目录

根据服务器规范，分别在对应服务器上建立conf、mongos、config、shard1、shard2、shard3、shard4、shard5目录， 因为mongos不存储数据，只需要建立日志文件目录即可

创建命令

mkdir -p {conf,mongos/log,config/data,config/log,shard1/data,shard1/log,shard2/data,shard2/log,shard3/data,shard3/log,shard4/data,shard4/log,shard5/data,shard5/log}

4.安装mongodb

可选

上传下载文件：yum -y install lrzsz
看查树形目录：yum -y install tree
杀进程：yum install -y psmisc

解压

tar -xzvf mongodb-linux-x86_64-4.0.6.tgz -C /usr/local/

改名

mv mongodb-linux-x86_64-4.0.6 mongodb

配置环境变量

vim /etc/profile

export MONGODB_HOME=/usr/local/mongodb
export PATH=$MONGODB_HOME/bin:$PATH

使立即生效

source /etc/profile

生成key并复制至所有主机示例

openssl rand -base64 512 > /data1/mongodb/conf/key
(注: security key has length 1368, must be between 6 and 1024 chars)
scp -r 172.29.5.31:/data1/mongodb/conf/key ./
chmod 600 key

5.配置config server

configsvr的配置文件

systemLog:
  destination: file
  logAppend: true
  path: /data2/mongodb/config/log/configsrv.log

storage:
  dbPath: /data2/mongodb/config/data/
  journal:
    enabled: true
  wiredTiger:
    engineConfig:
      directoryForIndexes: true

processManagement:
  fork: true
  pidFilePath: /var/run/mongodb/configsrv.pid
  timeZoneInfo: /usr/share/zoneinfo

net:
  port: 21000
  bindIp: 0.0.0.0
  maxIncomingConnections: 20000

security:
  keyFile: /data1/mongodb/conf/key
  authorization: enabled

replication:
  replSetName: configs
sharding:
  clusterRole: configsvr

启动

35   mongod  -f  /data1/mongodb/conf/configsvc.conf   
37   mongod  -f  /data1/mongodb/conf/configsvc.conf   
39   mongod  -f  /data1/mongodb/conf/configsvc.conf

初始化configsrv集群

mongo --port 21000

rs.initiate(
{
  _id: "configs",
  members: [
    {_id: 0,host: "172.29.x.35:21000",},
    {_id: 1,host: "172.29.x.37:21000",},
    {_id: 2,host: "172.29.x.39:21000",}
  ],
})

6.配置分片副本集

mongod.conf配置文件:

systemLog:
  destination: file
  logAppend: true
  path: /data2/mongodb/shard1/log/shard1.log

storage:
  dbPath: /data2/mongodb/shard1/data/
  journal:
    enabled: true
  wiredTiger:
engineConfig:
  cacheSizeGB: 10
      directoryForIndexes: true     

processManagement:
  fork: true 
  pidFilePath: /var/run/mongodb/shard1.pid
  timeZoneInfo: /usr/share/zoneinfo

net:
  port: 27001
  bindIp: 0.0.0.0 
  maxIncomingConnections: 500
  
security:
  keyFile: /data1/mongodb/conf/key
  authorization: enabled

replication:
  replSetName: shard1
sharding:
  clusterRole: shardsvr
31 33 35 37 39
P1 P2 P3 P4 P5
S5 S1 S2 S3 S4
A4 A5 A1 A2 A3

启动

31
mongod  -f  /data1/mongodb/conf/shard1-p.conf
mongod  -f  /data1/mongodb/conf/shard5-s.conf
mongod  -f  /data1/mongodb/conf/shard4-a.conf
33
mongod  -f  /data1/mongodb/conf/shard2-p.conf
mongod  -f  /data1/mongodb/conf/shard1-s.conf
mongod  -f  /data1/mongodb/conf/shard5-a.conf
35 
mongod  -f  /data1/mongodb/conf/shard3-p.conf
mongod  -f  /data1/mongodb/conf/shard2-s.conf
mongod  -f  /data1/mongodb/conf/shard1-a.conf
37
mongod  -f  /data1/mongodb/conf/shard4-p.conf
mongod  -f  /data1/mongodb/conf/shard3-s.conf
mongod  -f  /data1/mongodb/conf/shard2-a.conf
39
mongod  -f  /data1/mongodb/conf/shard5-p.conf
mongod  -f  /data1/mongodb/conf/shard4-s.conf
mongod  -f  /data1/mongodb/conf/shard3-a.conf

初始化副本集 注意需要分别设置5个副本，修改对应host和其他对应值

mongo --port 27001
rs.initiate(
{
  _id: "shard1",
  members: [
    {_id: 0,host: "172.29.5.31:27001",},
    {_id: 1,host: "172.29.5.33:27001",},
    {_id: 2,host: "172.29.5.35:27001",arbiterOnly: true,}
  ],
})

mongo --port 27002
rs.initiate(
{
  _id: "shard2",
  members: [
    {_id: 0,host: "172.29.5.33:27002",},
    {_id: 1,host: "172.29.5.35:27002",},
    {_id: 2,host: "172.29.5.37:27002",arbiterOnly: true,}
  ],
})

mongo --port 27003
rs.initiate(
{
  _id: "shard3",
  members: [
    {_id: 0,host: "172.29.5.35:27003",},
    {_id: 1,host: "172.29.5.37:27003",},
    {_id: 2,host: "172.29.5.39:27003",arbiterOnly: true,}
  ],
})

mongo --port 27004
rs.initiate(
{
  _id: "shard4",
  members: [
    {_id: 0,host: "172.29.5.37:27004",},
    {_id: 1,host: "172.29.5.39:27004",},
    {_id: 2,host: "172.29.5.31:27004",arbiterOnly: true,}
  ],
})

mongo --port 27005
rs.initiate(
{
  _id: "shard5",
  members: [
    {_id: 0,host: "172.29.5.39:27005",},
    {_id: 1,host: "172.29.5.31:27005",},
    {_id: 2,host: "172.29.5.33:27005",arbiterOnly: true,}
  ],
})

查看shard副本集状态

rs.status()

7.配置路由服务器mongos

mongos.conf配置文件:

systemLog:
  destination: file
  logAppend: true
  path: /data2/mongodb/mongos/log/mongos.log

processManagement:
  fork: true 
  pidFilePath: /var/run/mongodb/mongos.pid
  timeZoneInfo: /usr/share/zoneinfo

net:
  port: 20000
  bindIp: 0.0.0.0
  maxIncomingConnections: 500
 

security:
  keyFile: /data1/mongodb/conf/key

sharding:
  configDB: configs/172.29.x.35:21000,172.29.x.37:21000,172.29.x.39:21000

启动 mongos

35
mongos  -f  /data1/mongodb/conf/mongos.conf
37
mongos  -f  /data1/mongodb/conf/mongos.conf
39
mongos  -f  /data1/mongodb/conf/mongos.conf

设置分片

mongo --port 20000
use admin
#串联路由服务器与分配副本集
sh.addShard("shard1/172.29.x.31:27001,172.29.x.33:27001,172.29.x.35:27001")
sh.addShard("shard2/172.29.x.33:27002,172.29.x.35:27002,172.29.x.37:27002")
sh.addShard("shard3/172.29.x.35:27003,172.29.x.37:27003,172.29.x.39:27003")
sh.addShard("shard4/172.29.x.37:27004,172.29.x.39:27004,172.29.x.31:27004")
sh.addShard("shard5/172.29.x.39:27005,172.29.x.31:27005,172.29.x.33:27005")

查看集群状态

8.启用分片

为test数据库开启分片

选择一个片键并指定一个集合user对其进行hash分片

sh.enableSharding("test")
db.createCollection("text.user")
sh.shardCollection("test.user",{"_id":"hashed"})

测试分片，写入数据到数据库中

use test
for (i = 1; i <= 10000; i++) 
db.user.insert({age:(i%100), name:"user"+i, address:i+", West Street , Xuanwumen,beijing", country:"China"})

9.创建用户

mongos用户

db.createUser({user: "admin",pwd:"admin",roles:[{role:"root",db:"admin"}]})

副本集建立用户

db.createUser({user: "shard1",pwd:"shard1",roles:[{role:"root",db:"admin"}]})
db.createUser({user: "shard2",pwd:"shard2",roles:[{role:"root",db:"admin"}]})
db.createUser({user: "shard3",pwd:"shard3",roles:[{role:"root",db:"admin"}]})
db.createUser({user: "shard4",pwd:"shard4",roles:[{role:"root",db:"admin"}]})
db.createUser({user: "shard5",pwd:"shard5",roles:[{role:"root",db:"admin"}]})

管理员账户 admin/admin

监控账号

db.createUser({user: "monitor_mongodb_mongos",pwd:"Mmongos",roles:[{role:"readAnyDatabase",db:"admin"}]})

其他账号如下：

  • monitor_mongodb_mongos/Mmongos
  • monitor_mongodb_shard1/Mshard1
  • monitor_mongodb_shard2/Mshard2
  • monitor_mongodb_shard3/Mshard3
  • monitor_mongodb_shard4/Mshard4
  • monitor_mongodb_shard5/Mshard5

Read：允许用户读取指定数据库
readWrite：允许用户读写指定数据库
dbAdmin：允许用户在指定数据库中执行管理函数，如索引创建、删除，查看统计或访问system.profile
userAdmin：允许用户向system.users集合写入，可以找指定数据库里创建、删除和管理用户
clusterAdmin：只在admin数据库中可用，赋予用户所有分片和复制集相关函数的管理权限
readAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的读权限
readWriteAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的读写权限
userAdminAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的userAdmin权限
dbAdminAnyDatabase：只在admin数据库中可用，赋予用户所有数据库的dbAdmin权限
root：只在admin数据库中可用。超级账号，超级权限

10.启动关闭

关闭时，直接killall杀掉所有进程

ps -ef | grep mongo
killall mongod
killall mongos

11.脚本

空

12.容器化部署

空

13.常用命令

help查看命令提示

Help
db.help()
db.your_collection.help()
db.your_collection.find().help()
rs.help()

切换数据库

use your_database

查询所有数据库

show dbs

删除当前使用数据库

db.dropDatabase()

从指定主机上克隆数据库

db.cloneDatabase(“127.0.0.1”)

从指定机器上复制指定数据库数据到某一个数据库

db.copyDatabase(“mydb”,”temp”,”127.0.0.1”)

查看当前使用的数据库/数据库状态/数据库版本/数据库链接机器地址

db.getName()
db.status()
db.version()
db.getMongo()

用户相关

show users db.dropUser("userName");

查看oplog的信息,分片副本需要在分片上查询

db.printReplicationInfo()

增加权限

db.grantRolesToUser("monitor_mongodb_shard5", [{role:"clusterMonitor", db:"admin"}])

取消权限

db.revokeRolesFromUser( "myuser" , [ { role: "readWrite", db: "mydb" } ])

修改用户账户

db.changeUserPassword(username, password)

mongo shell 连接

mongo ip:port/database --username "username" --password "password" "database"
例：mongo --port 20000 -uadmin -padmin admin

14.参数优化配置

空

15.PMM监控注册

建cluster, 35\37\39

pmm-admin add mongodb:metrics --cluster mongodb_mongos35 --uri mongodb://monitor_mongodb_mongos:Mmongos@172.29.x.35:20000

注册分片,此处只给出一个实例

pmm-admin add mongodb:metrics  mongodb_mongos35_shard1_31  --cluster  mongodb_mongos35  --uri mongodb://monitor_mongodb_shard1:monitor@172.29.5.31:27001
pmm-admin add mongodb:metrics  mongodb_mongos37_shard1_31  --cluster  mongodb_mongos37  --uri mongodb://monitor_mongodb_shard1:monitor@172.29.5.31:27001
pmm-admin add mongodb:metrics  mongodb_mongos39_shard1_31  --cluster a_tc_yz_mongodb_mongos39  --uri mongodb://monitor_mongodb_shard1:monitor@172.29.5.31:27001
pmm-admin add mongodb:metrics  mongodb_mongos35_shard5_31  --cluster  mongodb_mongos35  --uri mongodb://monitor_mongodb_shard5:monitor@172.29.5.31:27005
pmm-admin add mongodb:metrics  mongodb_mongos37_shard5_31  --cluster  mongodb_mongos37  --uri mongodb://monitor_mongodb_shard5:monitor@172.29.5.31:27005
pmm-admin add mongodb:metrics  mongodb_mongos39_shard5_31  --cluster  mongodb_mongos39  --uri mongodb://monitor_mongodb_shard5:monitor@172.29.5.31:27005

参考

[1] MongoDB官网手册 docs.mongodb.com/master/tuto…
[2] MongoDB 4.0.1分片集群搭建https://blog.51cto.com/bigboss/2160311
[3] mongodb 3.4 集群搭建:分片+副本集https://www.cnblogs.com/ityouknow/p/7344005.html
[4] mongodb 3.4 集群搭建升级版五台集群—windows blog.csdn.net/DilemmaVF/a…
[5] Linux系统上传下载命令rz和sz blog.51cto.com/oldboy/5885…
[6] Centos7 中tree命令的使用 www.jianshu.com/p/d52a183c9…
[7] mongodb 配置文件 www.cnblogs.com/phpandmysql…
[8] CentOS7利用systemctl添加自定义系统服务https://mp.weixin.qq.com/s?__biz=MzA4NzQzMzU4Mg==&mid=2652923832&idx=1&sn=c8214873c1191baf1a404e51f7699573&chksm=8bed4fb9bc9ac6afcf2f2b473eeed76ba492b43762d0ca8d6b06359f92aca0a2e8967f5d95ff&scene=0&xtrack=1#rd
[9] MongoDB学习笔记https://blog.csdn.net/qq_33206732/column/info/21212

avatar
文章
阅读
粉丝