MongoDB数据的导出导入及日志分析
2023-02-19 12:21:39 时间
一、远程连接导出报错超时
mongodump -h 10.110.63.150:27017 -u'admin' -p'passwd!' --authenticationDatabase flowtest --db flowtest -o /home/mongod/bak > mongodump.log 2>&1 &
导出报错:
2022-11-17T18:58:54.663+0800 [###############.........] flowtest.processDataObj 68313/106962 (63.9%)
2022-11-17T18:58:56.506+0800 Failed: error writing data for collection `flowtest.processDataObj` to disk: error reading collection: connection pool for 10.110.63.150:27017 was cleared because another operation failed with: connection(10.110.63.150:27017[-2]) incomplete read of message header: read tcp 7.7.7.11:53305->10.110.63.150:27017: i/o timeout
二、本地导出不会超时
bin/mongodump -u'admin' -p'passwd!' --authenticationDatabase flowtest --db flowtest -o /home/nrms/thirdparty/mongodb/mongodb-linux-x86_64-3.6.13/bak > mongodump.log 2>&1 &
同时可见,导出不压缩,数据量比源目录大了很多,大约2倍,侧面说明了mongodb库本身的内部数据压缩效果不错:
1.4G data
2.8G bak
233M bak.tar
三、导入1个集合 history_task 到 testdb 库
mongorestore -h node1:20000 -u'liking' -p'passwd!2#' --authenticationDatabase testdb --db testdb --collection history_task /home/mongod/bak/flowtest/history_task.bson > mongorestore.log 2>&1 &
四、导入全部
mongorestore -h node1:20000 -u'admin' -p'passwd!2#' --authenticationDatabase admin --db flowtest /home/mongod/bak/flowtest > mongorestore.log 2>&1 &
五、导入日志分析
确定还原的集合列表,并读取各个集合的元数据:
2022-11-18T09:59:51.909+0800 The --db and --collection flags are deprecated for this use-case; please use --nsInclude instead, i.e. with --nsInclude=${DATABASE}.${COLLECTION}
2022-11-18T09:59:51.910+0800 building a list of collections to restore from /home/mongod/bak/flowtest dir
2022-11-18T09:59:51.911+0800 reading metadata for flowtest.history_task from metadata
2022-11-18T09:59:51.912+0800 reading metadata for flowtest.processConfiguration from metadata
2022-11-18T09:59:51.925+0800 reading metadata for flowtest.processDataObj from metadata
2022-11-18T09:59:51.930+0800 reading metadata for flowtest.processDataObjInit from metadata
2022-11-18T09:59:51.931+0800 reading metadata for flowtest.snapshotResource from metadata
2022-11-18T09:59:51.932+0800 reading metadata for flowtest.activityConfiguration from metadata
2022-11-18T09:59:51.945+0800 reading metadata for flowtest.processDataObjDraft from metadata
2022-11-18T09:59:51.945+0800 reading metadata for flowtest.processsDataObjInit from metadata
2022-11-18T09:59:51.946+0800 reading metadata for flowtest.role from metadata
同时开启4个并发 restoring:
2022-11-18T09:59:52.290+0800 restoring flowtest.snapshotResource
2022-11-18T09:59:52.335+0800 restoring flowtest.history_task
2022-11-18T09:59:52.369+0800 restoring flowtest.processDataObj
2022-11-18T09:59:52.396+0800 restoring flowtest.processDataObjInit
每 finished 1个,即再开始 restoring 另1个:
2022-11-18T09:59:52.824+0800 finished restoring flowtest.snapshotResource (790 documents, 0 failures)
2022-11-18T09:59:52.878+0800 restoring flowtest.processDataObjDraft
2022-11-18T09:59:53.096+0800 finished restoring flowtest.processDataObjDraft (123 documents, 0 failures)
2022-11-18T09:59:53.159+0800 restoring flowtest.activityConfiguration
每 finished 2个,即再开始 restoring 另2个:
2022-11-18T09:59:53.335+0800 finished restoring flowtest.activityConfiguration (1308 documents, 0 failures)
2022-11-18T09:59:53.382+0800 finished restoring flowtest.history_task (3629 documents, 0 failures)
2022-11-18T09:59:53.464+0800 restoring flowtest.processConfiguration
2022-11-18T09:59:53.544+0800 restoring flowtest.processsDataObjInit
2022-11-18T09:59:53.577+0800 finished restoring flowtest.processConfiguration (168 documents, 0 failures)
2022-11-18T09:59:53.622+0800 restoring flowtest.role from /home/mongod/bak/flowtest/role.bson
2022-11-18T09:59:55.108+0800 finished restoring flowtest.role (10 documents, 0 failures)
2022-11-18T09:59:55.108+0800 finished restoring flowtest.processsDataObjInit (4 documents, 0 failures)
2022-11-18T10:00:28.471+0800 finished restoring flowtest.processDataObjInit (96008 documents, 0 failures)
2022-11-18T10:00:45.768+0800 finished restoring flowtest.processDataObj (106968 documents, 0 failures)
开始恢复 indexes:
2022-11-18T10:00:45.781+0800 restoring indexes for collection flowtest.processDataObjInit from metadata
2022-11-18T10:00:45.799+0800 index: &idx.IndexDocument{Options:primitive.M{"name":"flowId_1_activityConfiguration.activityNameEn_1", "ns":"flowtest.processDataObjInit", "v":2}, Key:primitive.D{primitive.E{Key:"flowId", Value:1}, primitive.E{Key:"activityConfiguration.activityNameEn", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-11-18T10:00:45.799+0800 no indexes to restore for collection flowtest.snapshotResource
2022-11-18T10:00:45.799+0800 no indexes to restore for collection flowtest.activityConfiguration
2022-11-18T10:00:45.799+0800 no indexes to restore for collection flowtest.history_task
2022-11-18T10:00:45.799+0800 restoring indexes for collection flowtest.processDataObj from metadata
2022-11-18T10:00:45.799+0800 index: &idx.IndexDocument{Options:primitive.M{"name":"flowId_1_activityConfiguration.activityNameEn_1", "ns":"flowtest.processDataObj", "v":2}, Key:primitive.D{primitive.E{Key:"flowId", Value:1}, primitive.E{Key:"activityConfiguration.activityNameEn", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-11-18T10:00:45.799+0800 index: &idx.IndexDocument{Options:primitive.M{"name":"flowNo_1", "ns":"flowtest.processDataObj", "v":2}, Key:primitive.D{primitive.E{Key:"flowNo", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-11-18T10:00:45.799+0800 no indexes to restore for collection flowtest.processsDataObjInit
2022-11-18T10:00:45.799+0800 no indexes to restore for collection flowtest.processConfiguration
2022-11-18T10:00:45.799+0800 no indexes to restore for collection flowtest.processDataObjDraft
2022-11-18T10:00:45.800+0800 index: &idx.IndexDocument{Options:primitive.M{"name":"flowNo_1", "ns":"flowtest.processDataObjInit", "v":2}, Key:primitive.D{primitive.E{Key:"flowNo", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-11-18T10:00:45.800+0800 no indexes to restore for collection flowtest.role
最后的总结:恢复了 209008 document(s)
2022-11-18T10:00:48.031+0800 209008 document(s) restored successfully. 0 document(s) failed to restore.
相关文章
- 学生数据库管理系统
- SpringDataJpa 用MySQL语句怎么分页,spring全家桶SpringDataJpa 用MySQL语句怎么分页
- Docker创建MySQL容器模板命令
- Elasticsearch对应MySQL的对应关系
- 使用SpringDataJpa保存(save)报错误:SQL Error: 1062, SQLState: 23000 控制台会报:Duplicate entry ‘数‘ for key ‘PRIMA
- Navicat Premium 连接sqlserver数据库时提示安装Client失败,解决方案
- Mysql查询当前用户所有数据库语句(SHOW DATABASES)
- MySQL语句-查看当前数据库有哪些表(SHOW TABLES)
- MySQL5.0版本以上新增的 information_schema 数据库是什么?
- MariaDB数据库备份之逻辑备份
- MariaDB数据库创建用户
- MariaDB数据库给用户授权
- MariaDB数据库刷新权限表命令
- MariaDB数据库删除用户命令
- PhpStudy 2016搭建-sqli-libs靶场
- MySQL手动注入步骤
- Pikachu靶场-SQL注入-数字型注入(post)过关步骤
- Pikachu靶场-SQL注入-字符型注入(get)过关步骤
- 利用SQL注入漏洞实现MySQL数据库读写文件
- Kali-工具-sqlmap常见用法