【Mongo DB】万字详解，Mongo DB的简介到实战使用

小郭的技术笔记
3,924 阅读4分钟

持续创作，加速成长！这是我参与「掘金日新计划 · 6 月更文挑战」的第10天，点击查看活动详情

前言

大家好，我是小郭，今天主要为大家提供一条龙服务，从Mongo DB的简介到实战使用，使我们面对技术选型的时候可以得心应手。

Mongo DB简介

image-20220413220900191.png MongoDB是一个文档数据库（以Json为数据模型）,旨在简化应用程序开发和扩展，介于关系型数据库和非关系型之间。MongoDB中的记录是一个文档，它是一个由字段和值对组成的数据结构。

image-20220414122915794.png

特点

  • 原生高可用 高可用.png

  • 横向扩展能力 横向扩展.png

  • 无缝扩展

  • 应用全透明

  • 多种数据分布策略

应用场景

  • 共享单车，存储位置信息
  • 大文本JSON，记录应用服务器的日志记录

MongoDB 安装

MongoDB为我们提供多种的方案，灵活选择安装方式

Docker方式安装

docker pull mongo:5.0

docker run  \
--name mongo \
-p 27017:27017  \
-v /Users/xxx/soft/mongodb/configdb:/data/configdb/ \
-v /Users/xxx/soft/mongodb/db/:/data/db/ \
-d mongo:5.0 --auth	
 
# 进入容器
docker exec -it 6b594bf79827  bash
# 连接mongo
mongo

image-20220414123707375.png

MongoDB Atlas 搭建集群

可以参考下mongodb为我们提供的方案

account.mongodb.com/account/reg…

搭建完之后可以看到整个集群的状态

Altas.png

MongoDB基础概念和操作

SQL概念MongoDB概念
databasedatabase
tablecollection
rowdocument
columnfield
indexindex
primary keyprimary key（MongoDB自动将_id设置为主键）

数据库和集群

数据库操作

show dbs # 查看所有库

use myDB # 如果没有这个数据库会新建一个

db.dropDatabase() # 删除当前数据库

Collection

MongoDB将文档存储在集合中。集合类似于关系数据库中的表。

Collection.png

# 查看集合
show collections
# 创建集合
db.createCollection("myNewCollection1")
# 创建集合如果集合不存在
db.myNewCollection2.insertOne( { x: 1 } )
# 删除集合
db.myNewCollection1.drop()
# 集合详细信息
db.getCollectionInfos()

Document

  • 单个文档
db.inventory.insertOne(
   { item: "canvas", qty: 100, tags: ["cotton"], size: { h: 28, w: 35.5, uom: "cm" } }
)

插入单个文档.png _id是MongoDB默认指定的一个主键，我们插入的时候也可以自己指定id的值

手动设置_id.png

  • 多个文档

    db.inventory.insertMany([
   { item: "journal", qty: 25, tags: ["blank", "red"], size: { h: 14, w: 21, uom: "cm" } },
   { item: "mat", qty: 85, tags: ["gray"], size: { h: 27.9, w: 35.5, uom: "cm" } },
   { item: "mousepad", qty: 25, tags: ["gel", "blue"], size: { h: 19, w: 22.85, uom: "cm" } }
])

常用查询

MongoDBSQL
db.inventory.find( {} )SELECT * FROM inventory
db.inventory.find( { status: "D" } )SELECT * FROM inventory WHERE status = "D"
db.inventory.find( { status: { $in: [ "A", "D" ] } } )SELECT * FROM inventory WHERE status in ("A", "D")
db.inventory.find( { status: "A", qty: { $lt: 30 } } )SELECT * FROM inventory WHERE status = "A" AND qty < 30
db.inventory.find( { or: [ { status: "A" }, { qty: { lt: 30 } } ] } )SELECT * FROM inventory WHERE status = "A" OR qty < 30
db.inventory.find( {
status: "A",
or: [ { qty: { lt: 30 } }, { item: /^p/ } ]
} )		SELECT * FROM inventory WHERE status = "A" AND ( qty < 30 OR item LIKE "p%")

查询条件对照表

SQLMQL
a = 1{a:1}
a > 1{a:{$gt:1}
a >= 1{a:{$gte:1}
a < 1{a:{$lt:1}
a <= 1{a:{$lte:1}
a != 1{a:{$ne:1}
a = 1 and b = 1{a: 1, b: 1}或{$and: [{a: 1}, {b: 1}]}
a = 1 or b =1{$or: [{a: 1}, {b: 1}]}
a is null{a: {$exists: false}}
a in (1,2,3){a: {$in: [1, 2, 3]}}
a not in (1,2,3){a:{$nin:[1,2,3]}}
a like %xx%{a:/xx/}
a like %x{a:/x$/}
a like x%{a:/^x/}



嵌套查询

db.inventory.insertMany( [
   { item: "journal", qty: 25, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
   { item: "notebook", qty: 50, size: { h: 8.5, w: 11, uom: "in" }, status: "A" },
   { item: "paper", qty: 100, size: { h: 8.5, w: 11, uom: "in" }, status: "D" },
   { item: "planner", qty: 75, size: { h: 22.85, w: 30, uom: "cm" }, status: "D" },
   { item: "postcard", qty: 45, size: { h: 10, w: 15.25, uom: "cm" }, status: "A" }
]);

整个文档的查询需要整个完档都匹配上包括顺序

db.inventory.find( { size: { h: 14, w: 21, uom: "cm" } } )  -- 这个可以匹配上
db.inventory.find(  { size: { w: 21, h: 14, uom: "cm" } }  )  -- 这个不行

嵌套匹配.png

嵌套字段查询

db.inventory.find( { "size.uom": "in" } )

db.inventory.find( { "size.h": { $lt: 15 } } )

db.inventory.find( { "size.h": { $lt: 15 }, "size.uom": "in", status: "D" } )

数组查询


db.inventory.insertMany([
   { item: "journal", qty: 25, tags: ["blank", "red"], dim_cm: [ 14, 21 ] },
   { item: "notebook", qty: 50, tags: ["red", "blank"], dim_cm: [ 14, 21 ] },
   { item: "paper", qty: 100, tags: ["red", "blank", "plain"], dim_cm: [ 14, 21 ] },
   { item: "planner", qty: 75, tags: ["blank", "red"], dim_cm: [ 22.85, 30 ] },
   { item: "postcard", qty: 45, tags: ["blue"], dim_cm: [ 10, 15.25 ] }
]);

db.inventory.find( { tags: ["red", "blank"] } )


db.inventory.find( { tags: { $all: ["red", "blank"] } } ) -- 查询存在red和blank的document

列表查询.png

查询列表中的元素

db.inventory.find( { tags: "red" } )

查询返回字段

db.inventory.insertMany( [
  { item: "journal", status: "A", size: { h: 14, w: 21, uom: "cm" }, instock: [ { warehouse: "A", qty: 5 } ] },
  { item: "notebook", status: "A",  size: { h: 8.5, w: 11, uom: "in" }, instock: [ { warehouse: "C", qty: 5 } ] },
  { item: "paper", status: "D", size: { h: 8.5, w: 11, uom: "in" }, instock: [ { warehouse: "A", qty: 60 } ] },
  { item: "planner", status: "D", size: { h: 22.85, w: 30, uom: "cm" }, instock: [ { warehouse: "A", qty: 40 } ] },
  { item: "postcard", status: "A", size: { h: 10, w: 15.25, uom: "cm" }, instock: [ { warehouse: "B", qty: 15 }, { warehouse: "C", qty: 35 } ] }
]);


db.inventory.find( { status: "A" } ) -- 返回所有字段
db.inventory.find( { status: "A" }, { item: 1, status: 1 } )  -- 返回_id,item,status 类似SELECT _id, item, status from inventory WHERE status = "A"
db.inventory.find( { status: "A" }, { item: 1, status: 1, _id: 0 } ) -- 返回item,status


db.inventory.find( { status: "A" }, { status: 0, instock: 0 } ) -- 不返回status,instock


db.inventory.find(
   { status: "A" },
   { item: 1, status: 1, "size.uom": 1 }
)


db.inventory.find(
   { status: "A" },
   { "size.uom": 0 } -- 不返回size.uom其他都返回
)

db.inventory.find( { status: "A" }, { item: 1, status: 1, instock: { $slice: -1 } } ) -- 返回instock最后一个元素

查询NULL数据

db.inventory.insertMany([
   { _id: 1, item: null },
   { _id: 2 }
])

db.inventory.find( { item: null } )


db.inventory.find( { item : { $type: 10 } } )

db.inventory.find( { item : { $exists: false } } )

BSON TYPE可以看：

www.mongodb.com/docs/manual…

查询NULL.png

更新文档

  • db.collection.updateOne(,,)
  • db.collection.updateMany(,,)
db.inventory.insertMany( [
   { item: "canvas", qty: 100, size: { h: 28, w: 35.5, uom: "cm" }, status: "A" },
   { item: "journal", qty: 25, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
   { item: "mat", qty: 85, size: { h: 27.9, w: 35.5, uom: "cm" }, status: "A" },
   { item: "mousepad", qty: 25, size: { h: 19, w: 22.85, uom: "cm" }, status: "P" },
   { item: "notebook", qty: 50, size: { h: 8.5, w: 11, uom: "in" }, status: "P" },
   { item: "paper", qty: 100, size: { h: 8.5, w: 11, uom: "in" }, status: "D" },
   { item: "planner", qty: 75, size: { h: 22.85, w: 30, uom: "cm" }, status: "D" },
   { item: "postcard", qty: 45, size: { h: 10, w: 15.25, uom: "cm" }, status: "A" },
   { item: "sketchbook", qty: 80, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
   { item: "sketch pad", qty: 95, size: { h: 22.85, w: 30.5, uom: "cm" }, status: "A" }
] );


-- 更新第一条item=paper的数据 
db.inventory.updateOne(
   { item: "paper" },
   {
     $set: { "size.uom": "cm", status: "P" },
     $currentDate: { lastModified: true }
   }
)

db.inventory.updateMany(
   { "qty": { $lt: 50 } },
   {
     $set: { "size.uom": "in", status: "P" },
     $currentDate: { lastModified: true }
   }
)

$currentDate将更新lastModified为最新的时间，如果没有lastModified将会创建一个

document更新单条.png

删除文档

db.inventory.insertMany( [
   { item: "journal", qty: 25, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
   { item: "notebook", qty: 50, size: { h: 8.5, w: 11, uom: "in" }, status: "P" },
   { item: "paper", qty: 100, size: { h: 8.5, w: 11, uom: "in" }, status: "D" },
   { item: "planner", qty: 75, size: { h: 22.85, w: 30, uom: "cm" }, status: "D" },
   { item: "postcard", qty: 45, size: { h: 10, w: 15.25, uom: "cm" }, status: "A" },
] );


db.inventory.deleteMany({})

db.inventory.deleteMany({ status : "A" })

db.inventory.deleteOne( { status: "D" } )

聚合操作

MongoDB 中聚合(aggregate)主要用于处理数据(诸如统计平均值，求和等)，并返回计算后的数据结果。

管道命令

命令描述
$match将文档进行分组，统计结果
$group过滤数据
$sort文档排序
$limit限制文档返回数量
$skip跳过指定文档数量
$project修改输入文档的结构。可以用来重命名、增加或删除域，也可以用于创建计算结果以及嵌套文档。

表达式

命令描述
$sum计算总和
$avg计算平均值
$min获取最小值
$max获取最大值
$push在结果⽂档中插⼊值到⼀个数组中
$multiply将数字相乘并返回结果
db.orders.insertMany( [
   { _id: 0, name: "Pepperoni", size: "small", price: 19,
     quantity: 10, date: ISODate( "2021-03-13T08:14:30Z" ) },
   { _id: 1, name: "Pepperoni", size: "medium", price: 20,
     quantity: 20, date : ISODate( "2021-03-13T09:13:24Z" ) },
   { _id: 2, name: "Pepperoni", size: "large", price: 21,
     quantity: 30, date : ISODate( "2021-03-17T09:22:12Z" ) },
   { _id: 3, name: "Cheese", size: "small", price: 12,
     quantity: 15, date : ISODate( "2021-03-13T11:21:39.736Z" ) },
   { _id: 4, name: "Cheese", size: "medium", price: 13,
     quantity:50, date : ISODate( "2022-01-12T21:23:13.331Z" ) },
   { _id: 5, name: "Cheese", size: "large", price: 14,
     quantity: 10, date : ISODate( "2022-01-12T05:08:13Z" ) },
   { _id: 6, name: "Vegan", size: "small", price: 17,
     quantity: 10, date : ISODate( "2021-01-13T05:08:13Z" ) },
   { _id: 7, name: "Vegan", size: "medium", price: 18,
     quantity: 10, date : ISODate( "2021-01-13T05:10:13Z" ) }
] )


db.orders.aggregate({$match:{size:"medium"}});
-- select * from orders where size='medium';

db.orders.aggregate([
    {
        $match:{size:{$in:["small","large"]}}
    },
  	{
        $group:{_id:"$name",total:{$sum:"$quantity"}}
    }
]);

-- select name,sum(quantity)  total from orders where size in ('small','large') group by name

group.png

db.orders.aggregate( [
   {
      $match:
      {
         "date": { $gte: new ISODate( "2020-01-30" ), $lt: new ISODate( "2022-01-30" ) }
      }
   },
   {
      $group:
      {
         _id: { $dateToString: { format: "%Y-%m-%d", date: "$date" } },
         totalOrderValue: { $sum: { $multiply: [ "$price", "$quantity" ] } },
         averageOrderQuantity: { $avg: "$quantity" }
      }
   },
   {
      $sort: { totalOrderValue: -1 }
   }
 ] );

// 
db.orders.aggregate(
    {
        $match:{name:"Vegan"}
    },

    {
        $project: {_id:0,name:1,date:1}
    }
);

// select name,data from orders where name = 'Vegon';

orders.png

分页查询

db.students.find().skip(10).limit(10)

分页.png

索引

MongoDB 的索引是基于 B-tree 数据结构及对应算法形成的。_id默认会创建索引

index.png

索引类型

单列索引

​ 除了 MongoDB 定义的 _id 索引之外，MongoDB 还支持在文档的单个字段上创建用户定义的升序/降序索引。

single index.png

>db.collection.createlndex ( { source: 1 } )  //1 为升序，-1 为降序

复合索引

​ MongoDB 也支持复合索引，并且复合索引的规则和MySQL基本一致。复合索引中列出的字段顺序具有重要意义。例如，如果复合索引由 { userid: 1, score: -1 } 组成，则索引首先按userid正序排序，然后在每个userid的值内，再在按score倒序排序。

compound index .png

>db.collection.createIndex ({ "score": -1, "userid": 1 })

多键索引

若要为包含数组的字段建立索引，MongoDB 会为数组中的每个元素创建索引键。这些多键值索引支持对数组字段的高效查询

Multikey index.png

索引新增删除

  • 创建索引
db.collection.createIndex( { name: -1 } )

db.user.getIndexes()
[
  { v: 2, key: { _id: 1 }, name: '_id_' },
  { v: 2, key: { name: 1 }, name: 'name_1' }
]

Atlas atlas-119lob-shard-0 [primary] test> db.user.createIndex({name: 1},{name:"idx_name"})
idx_name
Atlas atlas-119lob-shard-0 [primary] test> db.user.getIndexes()
[
  { v: 2, key: { _id: 1 }, name: '_id_' },
  { v: 2, key: { name: 1 }, name: 'idx_name' }
]
  • 删除索引
db.user.dropIndexes("name_1")

执行计划

explain有三种模式，分别是：queryPlanner、executionStats、allPlansExecution。

  • queryPlanner：输出索引的候选索引，包括最优索引及其执行stage过程(winningPlan)+其他非最优候选索引及其执行stage过程
  • executionStats：相比queryPlanner参数，executionStats会记录查询优化器根据所选最优索引执行SQL的整个过程信息，会真正执行整个SQL。
  • allPlansExecution：和executionStats类似，只是多了所有候选索引的执行过程

新增10000条测试数据

for (let i = 1; i <= 500000; i++) db.item.insertOne({_id:i, item_id: "10010_" + i, item_order: "P1HNHN" + i, test:"test_"+i+"TES"})

Atlas atlas-119lob-shard-0 [primary] order> db.item.find({item_id:"10010_38976"}).explain("executionStats")
{
  explainVersion: '1',
  queryPlanner: {
    namespace: 'order.item',
    indexFilterSet: false,
    parsedQuery: { item_id: { '$eq': '10010_38976' } },
    maxIndexedOrSolutionsReached: false,
    maxIndexedAndSolutionsReached: false,
    maxScansToExplodeReached: false,
    winningPlan: {
      stage: 'COLLSCAN',
      filter: { item_id: { '$eq': '10010_38976' } },
      direction: 'forward'
    },
    rejectedPlans: []
  },
  executionStats: {
    executionSuccess: true,
    nReturned: 1,
    executionTimeMillis: 23,
    totalKeysExamined: 0,
    totalDocsExamined: 50000,
    executionStages: {
      stage: 'COLLSCAN',
      filter: { item_id: { '$eq': '10010_38976' } },
      nReturned: 1,
      executionTimeMillisEstimate: 2,
      works: 50002,
      advanced: 1,
      needTime: 50000,
      needYield: 0,
      saveState: 50,
      restoreState: 50,
      isEOF: 1,
      direction: 'forward',
      docsExamined: 50000
    }
  },
  command: { find: 'item', filter: { item_id: '10010_38976' }, '$db': 'order' },
  serverInfo: {
    host: 'cluster0-shard-00-02.fdv5c.mongodb.net',
    port: 27017,
    version: '5.0.8',
    gitVersion: 'c87e1c23421bf79614baf500fda6622bd90f674e'
  },
  serverParameters: {
    internalQueryFacetBufferSizeBytes: 104857600,
    internalQueryFacetMaxOutputDocSizeBytes: 104857600,
    internalLookupStageIntermediateDocumentMaxSizeBytes: 16793600,
    internalDocumentSourceGroupMaxMemoryBytes: 104857600,
    internalQueryMaxBlockingSortMemoryUsageBytes: 33554432,
    internalQueryProhibitBlockingMergeOnMongoS: 0,
    internalQueryMaxAddToSetBytes: 104857600,
    internalDocumentSourceSetWindowFieldsMaxMemoryBytes: 104857600
  },
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp(1, 1651821606),
    signature: {
      hash: Binary(Buffer.from("9250e56618dba0aacb504c846045c8e534adb68f", "hex"), 0),
      keyId: Long("7066952799589761026")
    }
  },
  operationTime: Timestamp(1, 1651821606)
}

在item_id上创建索引

db.item.createIndex ({ "item_id": 1},{"name":"idx_item_id" })

Atlas atlas-119lob-shard-0 [primary] order> db.item.find({item_id:"10010_38976"}).explain("executionStats")
{
  explainVersion: '1',
  queryPlanner: {
    namespace: 'order.item',
    indexFilterSet: false,
    parsedQuery: { item_id: { '$eq': '10010_38976' } },
    maxIndexedOrSolutionsReached: false,
    maxIndexedAndSolutionsReached: false,
    maxScansToExplodeReached: false,
    winningPlan: {
      stage: 'FETCH',
      inputStage: {
        stage: 'IXSCAN',
        keyPattern: { item_id: 1 },
        indexName: 'idx_item_id',
        isMultiKey: false,
        multiKeyPaths: { item_id: [] },
        isUnique: false,
        isSparse: false,
        isPartial: false,
        indexVersion: 2,
        direction: 'forward',
        indexBounds: { item_id: [ '["10010_38976", "10010_38976"]' ] }
      }
    },
    rejectedPlans: []
  },
  executionStats: {
    executionSuccess: true,
    nReturned: 1,
    executionTimeMillis: 0,
    totalKeysExamined: 1,
    totalDocsExamined: 1,
    executionStages: {
      stage: 'FETCH',
      nReturned: 1,
      executionTimeMillisEstimate: 0,
      works: 2,
      advanced: 1,
      needTime: 0,
      needYield: 0,
      saveState: 0,
      restoreState: 0,
      isEOF: 1,
      docsExamined: 1,
      alreadyHasObj: 0,
      inputStage: {
        stage: 'IXSCAN',
        nReturned: 1,
        executionTimeMillisEstimate: 0,
        works: 2,
        advanced: 1,
        needTime: 0,
        needYield: 0,
        saveState: 0,
        restoreState: 0,
        isEOF: 1,
        keyPattern: { item_id: 1 },
        indexName: 'idx_item_id',
        isMultiKey: false,
        multiKeyPaths: { item_id: [] },
        isUnique: false,
        isSparse: false,
        isPartial: false,
        indexVersion: 2,
        direction: 'forward',
        indexBounds: { item_id: [ '["10010_38976", "10010_38976"]' ] },
        keysExamined: 1,
        seeks: 1,
        dupsTested: 0,
        dupsDropped: 0
      }
    }
  },
  command: { find: 'item', filter: { item_id: '10010_38976' }, '$db': 'order' },
  serverInfo: {
    host: 'cluster0-shard-00-02.fdv5c.mongodb.net',
    port: 27017,
    version: '5.0.8',
    gitVersion: 'c87e1c23421bf79614baf500fda6622bd90f674e'
  },
  serverParameters: {
    internalQueryFacetBufferSizeBytes: 104857600,
    internalQueryFacetMaxOutputDocSizeBytes: 104857600,
    internalLookupStageIntermediateDocumentMaxSizeBytes: 16793600,
    internalDocumentSourceGroupMaxMemoryBytes: 104857600,
    internalQueryMaxBlockingSortMemoryUsageBytes: 33554432,
    internalQueryProhibitBlockingMergeOnMongoS: 0,
    internalQueryMaxAddToSetBytes: 104857600,
    internalDocumentSourceSetWindowFieldsMaxMemoryBytes: 104857600
  },
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp(2, 1651821695),
    signature: {
      hash: Binary(Buffer.from("035c24b6cc83e5d1d5bca97befb43a2c2fc30605", "hex"), 0),
      keyId: Long("7066952799589761026")
    }
  },
  operationTime: Timestamp(2, 1651821695)
}

事务

什么是writeConcern？

writeConcern 决定一个写操作落到多少个节点上才算成功。writeConcern 的取值包括：

  • 0：发起写操作，不关心是否成功；
  • 1~集群最大数据节点数：写操作需要被复制到指定节点数才算成功；
  • majority：写操作需要被复制到大多数节点上才算成功。

发起写操作的程序将阻塞到写操作到达指定的节点数为止

writeConcern=1.png majority

writeConcern=majority.png

副本集

MongoDB的副本集主要用于实现服务的高可用。

  • 数据写入时将数据迅速复制到其他节点上
  • 在Primary节点出现故障后自动选举一个Secondary节点作为Primary

副本结构.png

Golang操作MongoDB

var client *mongo.Client

type Item struct {
	ItemId string `bson:"item_id"`
	ItemOrder string `bson:"item_order"`
	Test string `bson:"test"`
	UpdateTime string `bson:"update_time"`

}

func main() {
	initDB()
	collection := client.Database("order").Collection("item")
	result, err := collection.InsertOne(context.TODO(), &Item{
		ItemId:     "test_item_id",
		ItemOrder:  "test_item_order",
		Test:       "test_test",
		UpdateTime: time.Now().Format("2006-01-02 15:04:05"),
	})
	if err != nil{
		log.Fatal(err)
	}
	fmt.Println(result.InsertedID)
	// find()

}

func find(){
	initDB()
	collection := client.Database("order").Collection("item")
	cursor, err := collection.Find(context.TODO(), bson.D{{"itemid", "test_item_id"}})
	if err != nil {
		log.Fatal(err)
	}
	defer cursor.Close(context.TODO())
	for cursor.Next(context.TODO()) {
		item := &Item{}
		err := cursor.Decode(item)
		if err != nil {
			log.Fatal(err)
		}
		fmt.Println(item)
	}

}

func initDB() {
	clientOptions := options.Client().ApplyURI(url)
	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel()
	c, err := mongo.Connect(ctx, clientOptions)
	if err != nil {
		log.Fatal(err)
	}
	err3 := c.Ping(ctx, nil)
	if err3 != nil {
		log.Fatal(err3)
	}
	fmt.Println("mongodb connected")
	client = c
}

官方文档：www.mongodb.com/docs/manual…

avatar
文章
阅读
粉丝