Performing Data Write Operations with MongoDB

This article discusses the steps involved in performing data write operations with MongoDB, focusing on the roles of the Journal and Oplog applications.

Introduction

This article discusses the steps involved in performing data write operations with MongoDB, focusing on the roles of the Journal and Oplog applications. Journal is a concept on the MongoDB storage engine layer while plog is a capped collection on the MongoDB master-slave replication layer.

MongoDB Journal

All data read and write operations in MongoDB require calling the interface on the storage engine layer to store and read data. The journal is an auxiliary mechanism for the storage engine to store data. Currently, MongoDB supports MMAPv1, WiredTiger, MongoRocks, and other storage engines, and all of them support the configuration of the journal.

To illustrate this, consider how WiredTiger functions. WiredTiger does not immediately store data written to it unless the configuration of the journal is complete. Instead, it performs a full-data checkpoint (storage.syncPeriodSecs configuration item) once every minute by default to make all the data persistent. If the server goes down in the middle of the process, data restoration is possible for data dating back to the most recent checkpoint.

It is often said that the enablement of journal is imperative. Upon enabling the journal, each write operation (reconstruction of the written data in the journal) is recorded in an operation log. As a result, if a fault occurs on the server after starting WiredTiger, WiredTiger can restore data from the most recent checkpoint, and the subsequent journal operation logs will be played back to restore the remaining data.

Two parameters control the actions of the journal in MongoDB. The storage.journal.enabled parameter determines whether to enable the journal and the storage.journal.commitInternalMs parameter determines the interval of the journal flushing to the disk, which has a default value of 100 ms. You can set the writeConcern to {j: true} during writing to ensure that journal flushes the disk at every write.

MongoDB Oplog

Through oplog, you can synchronize data between nodes in the replication set. The client writes data to the primary node, and the primary node records an oplog after writing the data. The secondary node pulls the oplog from the primary node (or other secondary nodes) to ensure each node in the replication set stores the same data. For the storage engine, oplog is part of the ordinary data.

One-Time Write with MongoDB

When writing a document to the MongoDB replication set, perform the following steps:

Write the document data to the corresponding set Update the sets index information Write an oplog for synchronization

The steps above must succeed completely, or fail completely, to avoid the following instances:

If data write is successful but the index write fails, some data may be readable in full-table scans but unreadable through indexes. If data write and index write are successful but the oplog write fails, the synchronization of the write operation to the secondary node will not be possible. This leads to data inconsistency between the master and slave nodes.

When MongoDB writes data, it puts the above three operations into a WiredTiger transaction to ensure the atomicity of the operations.

beginTransaction();

writeDataToColleciton();

writeCollectionIndex();

writeOplog();

commitTransaction();

Performing a transaction with WiredTiger initializes all application changes, with all the operations written to a journal operation log. The background will frequently set checkpoints to make the changes persistent and remove useless journals.

In terms of the data layout, the relationship between oplog and journal is as follows:

Conclusion

In this article, we discussed how MongoDB performs data write operations, specifically looking at the roles of oplog and journal in the process. Oplog and journal are concepts that represent the different layers of MongoDB. Since oplog is a common set in MongoDB, oplog writes and common set writes are identical. One write will change the corresponding data, index, and oplog, and these changes correspond to a journal operation log.

【安装MongoDB报错】mkdir: /data/db: Read-only file system 在安装MongoDB时，需要创建一个/data/db文件夹用来作为默认数据库目录。 但是因为Mac电脑默认是开启安全模式的，不能在根目录下面随便创建删除文件夹。所以我们创建的时候，会报这个错误mkdir: /data/db: Read-only file system，
Read-only dynamic data lwn文章翻译，原文[链接](https://lwn.net/Articles/750215/) ## 简介 本文主要讲述的是一种动态内存的只读保护机制。 ## 原文 内核开发者可以对想保护的数据设置为read-only权限，借助于MMU来避免恶意攻击者的篡改。kernel目前已经支持只读内存保护，但这些内存必须在操作系统自举完成前被初始化，所以局限性很大。Igor Stoppa的
MongoDB Data Synchronization MongoDB replica set (V3.0) synchronizes member status information through heartbeat information. Each node periodically sends heartbeat information to other members in the replica set.
db file sequential read 详解 db file sequential read (本文由thomaswoo_dba翻译,转载请注明出处) db file sequential read 事件有三个参数:file#,first block#, block count, 在oracle 10g里,此等待事件在归于 User I/O wait class 下面的.
MongoDB Write Concern，简称MongoDB写入安全机制，是一种客户端设置，用于控制写入安全的级别。
dellemc-streaming_data_platform_final_compressed 立即下载