Any Couchbase 4.0 / Kafka / Hadoop Examples

Couchbase Server
1

Hi All,

We like to explore Couchbase / Kafka / Hadoop any examples to follow for Couchbase 4.0 ?

Any suggestions ?

Regards
George

2

Have you looked at our Kafka connector? http://blog.couchbase.com/2015/november/kafka-and-couchbase-up-and-running-in-10-minutes

1 Like
3

@ldoguin

Its a good staring point for me thanks

Any idea what to do if we need to have time to live on document and couchbase goes to delete the document we some how send it to kafka ?

And I am trying to understand the benefit in using the Couchbase Kafka connector vs writing code using Couchbase SDK + Kafka SDK ? Any ideas ?

Regards
George

4

I am guessing docs even with short TTL will be put in a queue to be sent to Kafka. @avsej should be able to tell you more about it and what’s exactly in the Kafka connector.

1 Like
5

Unfortunately, Couchbase Server doesn’t currently have any interfaces that allow you to trigger events on expiration of TTL. One possible approach for something similar would be to regularly run a view query grabbing things just about to expire and move them to Hadoop or Kafka.

This could be done with the Spark Connector, about which I have a blog and we have a repository of samples.

The Kafka Connector uses the Couchbase DCP interface which currently lacks any server side filtering/query capabilities.

2 Likes
6

Hi Matt,

Will do I always enjoy your videos and examples they are very helpful.

Cheers and Thanks
George