Mutation details

Kafka Connector
1

Is it possible to acquire mutation details as a delta from the old document?
Alternatively, is it possible to get access to the old document?

Thanks,
Igor S

2

Hi Igor,

The Couchbase DCP protocol used by the Kafka connector does not provide deltas or any information about the previous document content. And once the document has been modified, there’s no way to retrieve the previous state.

However, with a bit of work in your application, it’s possible to build your own document versioning strategy. Here’s a third-party article on Document Versioning in Couchbase that describes one such technique.

Thanks,
David

3

Hi David,

Thanks for quick response and the reference.

Regards,
Igor

4

You’re welcome!

I don’t want to trivialize the amount of effort this would take. If you decide to try this, you’d need to write a custom SourceHandler for the Kafka Connector to look up the previous version of the document using your application’s document versioning scheme. This would require using the Couchbase Client SDK from within the connector. That’s not a use case we’ve handled before, so there might be some hurdles, especially around passing custom configuration to your SourceHandler. Let me know about any issues you encounter, and we can try to address them in a way that benefits everyone using the connector.

5

I am, planning, to use Kafka connector and Kafka Streams