Batch insert using transactions

.NET SDK
,
1

Hi everyone,

I’m facing an issue while inserting simultaneously multiple documents in couchbase, using transactions.
What I have read on the couchbase documentation (batching paragraph) is to instantiate a Task for each insertion and wait for the completion of all of them (as documented here)

If I try to execute many inserts in a batch (by instantiating multiple tasks) without using transactions everythings works fine, but when I call the InsertAsync method from the AttemptContext object, an exeception will randomly be thrown.
Here I’ve created a repository where you can reproduce the error.

You can find the stack trace listed below:

Couchbase.Transactions.Error.TransactionFailedException: Transaction failed.
—> System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source, Func`2 predicate)
at Couchbase.Transactions.AttemptContext.InsertAsync(ICouchbaseCollection collection, String id, Object content)
at TransactionBulkInsert.Program.<>c__DisplayClass7_1.<b__0>d.MoveNext() in C:\Users\user\source\repos\TransactionBulkInsert\Program.cs:line 73
— End of stack trace from previous location where exception was thrown —
at Couchbase.Transactions.Transactions.ExecuteApplicationLambda(Func`2 transactionLogic, TransactionContext overallContext, ILoggerFactory loggerFactory, TransactionResult result)
— End of inner exception stack trace —
at Couchbase.Transactions.Transactions.RunAsync(Func`2 transactionLogic, PerTransactionConfig perConfig)
at TransactionBulkInsert.Program.SaveBatch() in C:\Users\user\source\repos\TransactionBulkInsert\Program.cs:line 64
-----------------------Context Info---------------------------
null

Examining the source code I suppose is something related to _stagedMutations property of AttempContext class. In particular seems that the exception was thrown at line 424 when searching for staged mutations while another task may have changed the same collection.

Currently I’m using these nuget packages:

  • CouchbaseNetClient: v3.1.7
  • Couchbase.Transactions: v1.0.0

and Couchbase Server Community v.6.6.0

Do you have any suggestion?

2

It looks to me like the internal implementation of Transactions doesn’t support multi-threading, it’s using collection types that are not thread-safe.

@Richard_Ponton thoughts?

3

Indeed, @btburnett3, the 1.0.0 version of transactions in .NET appears to not be thread-safe in this way.

There is internal state management and order of operations matters when doing things like multiple operations on the same doc in the same transaction. I thought I had guarded against this sort of thing with locking, which wouldn’t have helped performance when trying to be parallel like this, but apparently I missed a case.

@Alessandro_Gargiulo , please await each task inside the transaction lambda, for now. Parallel operations inside a transaction are something we’ve been talking about and will be working on.