Couchbase transactions require a degree of co-operation from an application. Specifically, the application should ensure that non-transactional writes are never done concurrently with transactional writes, on the same document.
It does not elaborate further on how this can happen (1) and what is an effective way to prevent this (2).
I tried to make one of the writes to be overwritten by the other but I didn’t succeed. Is it an edge case that happens under very specific circumstances ?
ensure that non-transactional writes are never done concurrently with transactional writes, on the same document
It can happen any time a write is done without using transactions. So if you have an existing application that does not use transactions, or someone using the WebUI that is updating the same document - those could result in concurrent updates even though you are using transactions elsewhere.
Don’t do it. Perhaps by always using transactions.
how to handle concurrent writes.
relying on the CAS to detect concurrent modifications is faster and eaiser. Unless you have something that specifically requires transactions - such as modifications to multiple documents - I would recommend just using the CAS mechanism.
I tried to make one of the writes to be overwritten by the other but I didn’t succeed.
I will try to describe this, I don’t guarantee that this will produce the overwrite behavior. But something like this …You’ll likely need to use synchronization between the transactional and non-transactional updates. The gist of it is that outside of transactions is oblivious to what is going on inside transactions and inside transactions is oblivious to what is going on outside transactions.
O → outside tx thread
I → inside tx thread
O.1) block until the doc has been read inside the tx.
I.1) start the transaction
I.2) read the document, unblock O
I.3) block until the doc has been read outside the tx.
O.2) read and update the document
O.3) unblock I
I.4) update the document in the transaction
I.5) complete the transaction
The transaction will succeed (without complaining about the concurrent update from O.2) and the update from O.2 will have been overwritten.
Of course, but I’m interested in concurrent writes with non-tx vs tx, specifically.
I thought about that, but it is impractical.
Yes, but again, I’m interested specifically in transactions.
I’ve already tried that exact sequence.
The result is : the tx throws and the document content is the one from O.
Which makes sense, because the tx uses the cas, so the replace fails.
I think it’s more likely that an overwrite happens during the unstaging phase.
The unstaging probably doesn’t check for cas when swapping the content from xattr to the document body. The tx cannot be rollback at this point anyway, so checking for the cas would be quite irrelevant. WDYT ?
Hi @JesusTheHun . We do try to detect if a non-transactional concurrent write has been made to a document at the point it is unstaged, using techniques including CAS. As you say we cannot fix the situation (we are committed here, and cannot rollback the transaction) - but we can warn the user this is going on.
Each SDK will differ in how it raises these issues to the user. The Java SDK for instance will both log them, and raise events you can subscribe on - see these docs for more.
It’s best-effort though, as there are some situations where we do not have enough information to reliably know if this has occurred.
That’s somewhat correct, though more technically it’s between the point a doc is staged and unstaged
If we detect the document has changed in-between read and staging, we can easily detect that and make the transaction rollback and retry. And once we have staged a document, it is safe from writes from other transactions. It is non-transactional writes between the staging and unstaging points that the application needs to protect against.
This was a decision taken in order to continue providing the high KV performance that Couchbase users are used to: it was a design goal of transactions that they do not impact on non-transactional KV ops in any way.
Thanks for the detailed explanation.
Is there a recommended way to prevent concurrent kv write over tx write ?
A lock wouldn’t do it because as soon as the tx mutate the doc, the lock would be released.
Or maybe a specific document that is locked and unlocked after the tx ? That seems cumbersome though.