Receiving a "Transaction Operation Failed Exception" when attempting an Insert

.NET SDK
1

When using the Couchbase.Transactions library, v1.0.0-beta.2, I am getting the following exception when calling the InsertAsync method:

Couchbase.Transactions.Error.External.TransactionOperationFailedException: Exception of type 'Couchbase.Transactions.Error.External.TransactionOperationFailedException' was thrown.
   at Couchbase.Transactions.AttemptContext.<SetAtrPending>b__47_0()
   at Couchbase.Transactions.AttemptContext.<>c__DisplayClass79_0.<<RepeatUntilSuccessOrThrow>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Couchbase.Transactions.AttemptContext.RepeatUntilSuccessOrThrow[T](Func`1 func, Int32 retryLimit, String caller)
   at Couchbase.Transactions.AttemptContext.SetAtrPending()
   at Couchbase.Transactions.AttemptContext.SetAtrPendingIfFirstMutation(ICouchbaseCollection collection)
   at Couchbase.Transactions.AttemptContext.InsertAsync(ICouchbaseCollection collection, String id, Object content)
   at MyLibrary.MyOtherMethod() in /Users/mlearner/work/repos/.../myOtherClass.cs:line 73
-----------------------Context Info---------------------------
null

This exception repeats many times because of retries, but is never successful. I am not able to find any documentation regarding why this exception may be raised.
I know I am able to connect to my (new and empty) bucket as it has created an empty _txn:client-record document there.
Before receiving the exception message listed above, I receive this exception message, which I expect and handle:

Couchbase.Core.Exceptions.KeyValue.DocumentNotFoundException: Exception of type 'Couchbase.Core.Exceptions.KeyValue.DocumentNotFoundException' was thrown.
   at Couchbase.Core.ClusterNode.ExecuteOp(Func`4 sender, IOperation op, Object state, CancellationTokenPair tokenPair)
   at Couchbase.CouchbaseBucket.SendAsync(IOperation op, CancellationTokenPair tokenPair)
   at Couchbase.Core.Retry.RetryOrchestrator.RetryAsync(BucketBase bucket, IOperation operation, CancellationTokenPair tokenPair)
   at Couchbase.KeyValue.CouchbaseCollection.GetAsync(String id, GetOptions options)
   at MyLibrary.MyMethod() in /Users/mlearner/work/repos/.../someClass.cs:line 264
-----------------------Context Info---------------------------
{"DispatchedFrom":null,"DispatchedTo":null,"DocumentKey":"MyDocumentKey","ClientContextId":"19","Cas":0,"Status":1,"BucketName":"myBucket","CollectionName":"_default","ScopeName":"_default","Message":"KV Error: {Name=\"KEY_ENOENT\", Description=\"Not Found\", Attributes=\"item-only\"}"}

I do see that the “Context Info” in this initial message is not null, whereas it is null in the subsequent message. I’m not sure what to do with that information however.
The AttemptContext, on which I am calling InsertAsync is not null, of course. Rather it is just the “Context Info” found in the exception message itself that is showing “null”.
What might cause the TransactionOperationFailedException?

  • Couchb aseNetClient, 3.1.3
  • Couchbase.Transactions, 1.0.0-beta.2
  • net5.0
2

Hi malclear.

SetAtrPending is attempting to create a metadata record on the same collection/scope as the first document in the transaction. Are you able to do a regular insert of a dummy document on that collection?

The exception information could definitely be more useful… In the meantime, can you configure logging to try and get more info? Reference the NuGet package Microsoft.Extensions.Logging.Console,

using var loggerFactory = LoggerFactory.Create(builder =>
{
   builder.AddConsole();
});
var txnConfigBuilder = TransactionConfigBuilder.Create().LoggerFactory(loggerFactory);
var txn = Transactions.Create(cluster, txnConfigBuilder.Build());
1 Like
3

Thank you, you helped.
After properly configuring the logger, I saw that I was receiving a DurabilityImpossibleException. I followed instructions here to set the number of bucket replicas to zero. I also removed the call to the TransactionConfigBuilder’s DurabilityLevel method, where I was passing in the DurabilityLevel.PersistToMajority value.

However, this wasn’t enough. I had to, finally, replace the DurabilityLevel.PersistToMajority with DurabilityLevel.None . This resolved my issue.

If there is no other workaround, this does mean that I’ll need to pass the appropriate DurabilityLevel to my infrastructure code after the current operating environment (development or server-deployed) is identified in the DI configuration.

4

You should be able to use any DurabilityLevel supported by your cluster for transactions.

If you can get it working with a regular Insert, a transaction should be able to use the same DurabilityLevel. Keep in mind that it needs to be available on every collection used by every document modification in the transaction.

5

I have gotten it working and I thank you for your help with that.
The point I was making is that it seems the durability level that is appropriate for the cluster must be supplied to the TransactionConfigBuilder at runtime. On the developer’s machine, DurabilityLevel.None will have to be supplied, but when deployed to a server we will probably use DurabilityLevel.PersistToMajority as the clusters will be different in those two places. This seems a little awkward to me. It seems like the DurabilityLevel configured for the bucket should be the default for the TransactionConfig when it is built.

But, we can manage that.

6

Hi @malclear
So this is a common stumbling block that we’re aiming to improve in future server releases. A few things (and you’ve already figured most of this out, I’m just listing them as a reference for others):

  • It is possible to use the same durability level on both a single node and multi node cluster.
  • The default number of replicas on a newly created bucket is 1, which is incompatible with a single node cluster when using durable operations and results in the DurabilityImpossibleException. It needs to be set to 0 on a single node cluster.
  • Some bucket settings, when being changed on an existing bucket, need a rebalance to take effect - and this num-replicas setting is one of them. (I believe this is the change you need.)

This is documented in various places, but clearly not well enough (especially the need for a rebalance). I’ll open a ticket to improve it (DOC-8414).

1 Like
7

Graham, you were exactly correct. The rebalance was the missing piece.
My need to produce this PoC and my need to thoroughly read all the docs have been in competition with each other.
Thanks again to you both for the help.