Perform Two Phase Commits

Synopsis

This document provides a pattern for doing multi-document updates or “transactions” using a two-phase commit approach for writing data to multiple documents. Additionally, you can extend this process to provide a rollback like functionality.

Background

Operations on a single document are always atomic with MongoDB databases; however, operations that involve multiple documents, which are often referred to as “transactions,” are not atomic. Since documents can be fairly complex and contain multiple “nested” documents, single-document atomicity provides necessary support for many practical use cases.

因此,不注意的话,数据库操作的成功或失败就不可能是“全部或无”,没有对多文档事务的支持,一项操作就有可能在某些时候成功,但在其他时候却失败。在执行由数项序列操作组成的事务时,会出现以下问题:

  • Atomicity: if one operation fails, the previous operation within the transaction must “rollback” to the previous state (i.e. the “nothing,” in “all or nothing.”)
  • 隔离:与事务操作集同时执行的操作必须通过事务过程“看到”数据的一致视图。
  • 一致性:如果重大故障(如网络、硬件)中断了事务,数据库必须能够还原为一致的状态。

Despite the power of single-document atomic operations, there are cases that require multi-document transactions. For these situations, you can use a two-phase commit, to provide support for these kinds of multi-document updates.

Because documents can represent both pending data and states, you can use a two-phase commit to ensure that data is consistent, and that in the case of an error, the state that preceded the transaction is recoverable.

注意

Because only single-document operations are atomic with MongoDB, two-phase commits can only offer transaction-like semantics. It’s possible for applications to return intermediate data at intermediate points during the two-phase commit or rollback.

Pattern

Overview

The most common example of transaction is to transfer funds from account A to B in a reliable way, and this pattern uses this operation as an example. In a relational database system, this operation would encapsulate subtracting funds from the source (A) account and adding them to the destination (B) within a single atomic transaction. For MongoDB, you can use a two-phase commit in these situations to achieve a compatible response.

All of the examples in this document use the mongo shell to interact with the database, and assume that you have two collections: First, a collection named accounts that will store data about accounts with one account per document, and a collection named transactions which will store the transactions themselves.

Begin by creating two accounts named A and B, with the following command:

db.accounts.save({name: "A", balance: 1000, pendingTransactions: []})
db.accounts.save({name: "B", balance: 1000, pendingTransactions: []})

To verify that these operations succeeded, use find():

db.accounts.find()

mongo will return two documents that resemble the following:

{ "_id" : ObjectId("4d7bc66cb8a04f512696151f"), "name" : "A", "balance" : 1000, "pendingTransactions" : [ ] }
{ "_id" : ObjectId("4d7bc67bb8a04f5126961520"), "name" : "B", "balance" : 1000, "pendingTransactions" : [ ] }

事务描述

Set Transaction State to Initial

Create the transaction collection by inserting the following document. The transaction document holds the source and destination, which refer to the name fields of the accounts collection, as well as the value field that represents the amount of data change to the balance field. Finally, the state field reflects the current state of the transaction.

db.transactions.save({source: "A", destination: "B", value: 100, state: "initial"})

To verify that these operations succeeded, use find():

db.transactions.find()

这将返回一个类似于以下内容的文档:

{ "_id" : ObjectId("4d7bc7a8b8a04f5126961522"), "source" : "A", "destination" : "B", "value" : 100, "state" : "initial" }

Switch Transaction State to Pending

Before modifying either records in the accounts collection, set the transaction state to pending from initial.

Set the local variable t in your shell session, to the transaction document using findOne():

t = db.transactions.findOne({state: "initial"})

After assigning this variable t, the shell will return the value of t, you will see the following output:

{
     "_id" : ObjectId("4d7bc7a8b8a04f5126961522"),
     "source" : "A",
     "destination" : "B",
     "value" : 100,
     "state" : "initial"
}

Use update() to change the value of state to pending:

db.transactions.update({_id: t._id}, {$set: {state: "pending"}})
db.transactions.find()

The find() operation will return the contents of the transactions collection, which should resemble the following:

{ "_id" : ObjectId("4d7bc7a8b8a04f5126961522"), "source" : "A", "destination" : "B", "value" : 100, "state" : "pending" }

Apply Transaction to Both Accounts

Continue by applying the transaction to both accounts. The update() query will prevent you from applying the transaction if the transaction is not already pending. Use the following update() operation:

db.accounts.update({name: t.source, pendingTransactions: {$ne: t._id}}, {$inc: {balance: -t.value}, $push: {pendingTransactions: t._id}})
db.accounts.update({name: t.destination, pendingTransactions: {$ne: t._id}}, {$inc: {balance: t.value}, $push: {pendingTransactions: t._id}})
db.accounts.find()

The find() operation will return the contents of the accounts collection, which should now resemble the following:

{ "_id" : ObjectId("4d7bc97fb8a04f5126961523"), "balance" : 900, "name" : "A", "pendingTransactions" : [ ObjectId("4d7bc7a8b8a04f5126961522") ] }
{ "_id" : ObjectId("4d7bc984b8a04f5126961524"), "balance" : 1100, "name" : "B", "pendingTransactions" : [ ObjectId("4d7bc7a8b8a04f5126961522") ] }

Set Transaction State to Committed

Use the following update() operation to set the transaction’s state to committed:

db.transactions.update({_id: t._id}, {$set: {state: "committed"}})
db.transactions.find()

The find() operation will return the contents of the transactions collection, which should now resemble the following:

{ "_id" : ObjectId("4d7bc7a8b8a04f5126961522"), "destination" : "B", "source" : "A", "state" : "committed", "value" : 100 }

Remove Pending Transaction

Use the following update() operation to set remove the pending transaction from the documents in the accounts collection:

db.accounts.update({name: t.source}, {$pull: {pendingTransactions: t._id}})
db.accounts.update({name: t.destination}, {$pull: {pendingTransactions: t._id}})
db.accounts.find()

The find() operation will return the contents of the accounts collection, which should now resemble the following:

{ "_id" : ObjectId("4d7bc97fb8a04f5126961523"), "balance" : 900, "name" : "A", "pendingTransactions" : [ ] }
{ "_id" : ObjectId("4d7bc984b8a04f5126961524"), "balance" : 1100, "name" : "B", "pendingTransactions" : [ ] }

Set Transaction State to Done

Complete the transaction by setting the state of the transaction document to done:

db.transactions.update({_id: t._id}, {$set: {state: "done"}})
db.transactions.find()

The find() operation will return the contents of the transactions collection, which should now resemble the following:

{ "_id" : ObjectId("4d7bc7a8b8a04f5126961522"), "destination" : "B", "source" : "A", "state" : "done", "value" : 100 }

从故障情景中还原

The most important part of the transaction procedure is not, the prototypical example above, but rather the possibility for recovering from the various failure scenarios when transactions do not complete as intended. This section will provide an overview of possible failures and provide methods to recover from these kinds of events.

共有两类故障:

Thus, the application will always be able to resume the transaction and eventually arrive at a consistent state. Run the following recovery operations every time the application starts to catch any unfinished transactions. You may also wish run the recovery operation at regular intervals to ensure that your data remains consistent.

达到一致状态所需的时间取决于应用程序还原各事务所需的时长。

Rollback

In some cases you may need to “rollback” or undo a transaction when the application needs to “cancel” the transaction, or because it can never recover as in cases where one of the accounts doesn’t exist, or stops existing during the transaction.

共有两种可能的回滚操作:

  1. After you apply the transaction (i.e. the third step,) you have fully committed the transaction and you should not roll back the transaction. Instead, create a new transaction and switch the values in the source and destination fields.
  2. After you create the transaction (i.e. the first step,) but before you apply the transaction (i.e the third step,) use the following process:
Set Transaction State to Canceling

Begin by setting the transaction’s state to canceling using the following update() operation:

db.transactions.update({_id: t._id}, {$set: {state: "canceling"}})
Undo the Transaction

使用下面的一系列操作,从两个帐户撤消事务的操作:

db.accounts.update({name: t.source, pendingTransactions: t._id}, {$inc: {balance: t.value}, $pull: {pendingTransactions: t._id}})
db.accounts.update({name: t.destination, pendingTransactions: t._id}, {$inc: {balance: -t.value}, $pull: {pendingTransactions: t._id}})
db.accounts.find()

The find() operation will return the contents of the accounts collection, which should resemble the following:

{ "_id" : ObjectId("4d7bc97fb8a04f5126961523"), "balance" : 1000, "name" : "A", "pendingTransactions" : [ ] }
{ "_id" : ObjectId("4d7bc984b8a04f5126961524"), "balance" : 1000, "name" : "B", "pendingTransactions" : [ ] }
Set Transaction State to Canceled

Finally, use the following update() operation to set the transaction’s state to canceled:

Step 3: set the transaction’s state to “canceled”:

db.transactions.update({_id: t._id}, {$set: {state: "canceled"}})

Multiple Applications

事务部分存在,因此多个应用程序可创建并同时执行操作,不会导致数据不一致或冲突。因此,在任何时间点上,只有一个应用程序可处理给定事务非常重要。

Consider the following example, with a single transaction (i.e. T1) and two applications (i.e. A1 and A2). If both applications begin processing the transaction which is still in the initial state (i.e. step 1), then:

  • A1 can apply the entire whole transaction before A2 starts.
  • A2 will then apply T1 for the second time, because the transaction does not appear as pending in the accounts documents.

To handle multiple applications, create a marker in the transaction document itself to identify the application that is handling the transaction. Use findAndModify() method to modify the transaction:

t = db.transactions.findAndModify({query: {state: "initial", application: {$exists: 0}},
                                   update: {$set: {state: "pending", application: "A1"}},
                                   new: true})

When you modify and reassign the local shell variable t, the mongo shell will return the t object, which should resemble the following:

{
     "_id" : ObjectId("4d7be8af2c10315c0847fc85"),
     "application" : "A1",
     "destination" : "B",
     "source" : "A",
     "state" : "pending",
     "value" : 150
}

Amend the transaction operations to ensure that only applications that match the identifier in the value of the application field before applying the transaction.

If the application A1 fails during transaction execution, you can use the recovery procedures, but applications should ensure that they “owns” the transaction before applying the transaction. For example to resume pending jobs, use a query that resembles the following:

db.transactions.find({application: "A1", state: "pending"})

This will (or may) return a document from the transactions document that resembles the following:

{ "_id" : ObjectId("4d7be8af2c10315c0847fc85"), "application" : "A1", "destination" : "B", "source" : "A", "state" : "pending", "value" : 150 }

Using Two-Phase Commits in Production Applications

以上示例事务故意简化。例如,它假设:

  • 始终可能回滚操作一个帐户。
  • 帐户余额可以为负值。

Production implementations would likely be more complex. Typically accounts need to information about current balance, pending credits, pending debits. Then:

  • when your application switches the transaction state to pending (i.e. step 2) it would also make sure that the accounts has sufficient funds for the transaction. During this update operation, the application would also modify the values of the credits and debits as well as adding the transaction as pending.
  • when your application removes the pending transaction (i.e. step 4) the application would apply the transaction on balance, modify the credits and debits as well as removing the transaction from the pending field., all in one update.

Because all of the changes in the above two operations occur within a single update() operation, these changes are all atomic.

此外,对于大多数重要的事务,请确保:

  • the database interface (i.e. client library or driver) has a reasonable write concern configured to ensure that operations return a response on the success or failure of a write operation.
  • your mongod instance has journaling enabled to ensure that your data is always in a recoverable state, in the event of an unclean mongod shutdown.