Hadoop Couchbase Connector - Lambda Architecture

Other
1

Hi

I have a question regarding the following links:

http://blog.couchbase.com/hortonworks-and-couchbase-nosql-fueled-hadoop

http://www.couchbase.com/bigdata

http://www.couchbase.com/press-releases/couchbase-delivers-real-time-big-data-solutions-to-the-enterprise

They are about the connection between Hadoop and Couchbase. Does anyone of you know if this has anything to do with the Lambda architecture or if and how it could fit into it. Or is it a kind of an own architecture?

Thank you very much in advance.

Regards,
Daniela

2

Hi Daniela,

I would say its kind of its own architecture.

Let’s say the common components are Couchbase, Kafka, Spark / Storm, and Hadoop. They can be arranged in any order, and the data can flow in any direction. Lambda architecture is just one arrangement and flow, and it is a solution to a very specific problem. We see lots of arrangements and flows.

  • Kafka to Storm/Spark to Couchbase to Hadoop
  • Couchbase to Kafka to Hadoop
  • Log to Hadoop to Couchbase
  • Couchbase to Hadoop to Couchbase

The first one is what the big data site focuses on. However, you can create your own architecture by reordered components or add/removing them. That’s the fun part.

More on Lambda here: http://blog.couchbase.com/lamda-architecture-and-beyond-with-nosql

3

Hi Shane

Thank you very much for your explanation.

Unfortunately I do not understand why or in which cases it should be useful to replicate data between Couchbase and Hadoop in both directions. For me it would make sense to see the data flow from Couchbase to Hadoop or vice versa but not in both directions. Could you explain me why and how this could be useful?

Thank you very much in advance.

Regards,
Daniela