Hi everyone,
To address the needs of users who want to customize the kafka-connect-couchbase Sink connector, in version 4.1 we’re planning to add a SinkHandler extension point, similar to the existing SourceHandler in the source connector.
The commit that adds the SinkHandler interface is linked below. (It also incidentally banishes some rare race conditions in the SUBDOCUMENT and N1QL modes).
If this sounds interesting, we’d love to hear your feedback on the new extension point (ideally within the next few weeks while it’s still easy to change).
Thanks,
committed 01:42AM - 24 Jun 21 UTC
KAFKAC-261 Sink: Missed updates when using subdoc mode with append/prepend
KAFKA… C-262 Sink: Missed updates when using N1QL mode
Motivation
----------
Users want to customize the sink behavior.
Modifications
-------------
Add a SinkHandler interface for users to implement,
with concurrency controlled by ConcurrencyHints.
Add an CustomSinkHandler example.
Refactor the configurable sink behavior from CouchbaseSinkTask
into 3 new built-in handlers:
* UpsertSinkHandler (the default connector behavior)
* N1qlSinkHandler (`couchbase.document.mode = N1QL`)
* SubDocumentSinkHandler (`couchbase.document.mode = SUBDOC`)
Add a new `couchbase.sink.handler` config property for selecting
the handler.
Deprecate the `couchbase.document.mode` config property
in favor of `couchbase.sink.handler`.
Group the handler-specific config properties (for N1qlSinkHandler
and SubDocumentSinkHandler) under separate documentation headings.
Split the `couchbase.create.document` config property into
`couchbase.n1ql.create.document` and
`couchbase.subdocument.create.document` so they can be grouped
with the other handler-specific properties.
The old property name is accepted as an alias for the new ones.
Use the new concurrency control scheme to ensure SubDocumentSinkHandler
and N1qlSinkHandler behave correctly (see KAFKAC-261 and KAFKAC-262).
Configure the Maven Javadoc plugin to accept @implNote and friends.
Change-Id: I53075168bdcdb821d4990577582f76c2b973a5ce
Reviewed-on: http://review.couchbase.org/c/kafka-connect-couchbase/+/156380
Tested-by: Build Bot <build@couchbase.com>
Reviewed-by: Michael Reiche <michael.reiche@couchbase.com>