Couchbase version: Community Edition 5.0.0 build 3519
Spark Version: 2.2.0
Spark Couch Connector: 2.2.0 with Scala 2.11
We have two buckets like bucket1 and bucket2. bucket one has 1 lack records and bucket 2 has 20 lack records.
When we read the one bucket the reads are happening pretty fast. 6k gets/sec
But when we connect two buckets trying read as below the reads are coming down to 512 gets/sec.
val spark = SparkSession.builder()
.appName(“CouchbaseReadTest”)
.config(“com.couchbase.username”, “")
.config(“com.couchbase.password”, "”)
.config(“spark.couchbase.bucket.bubcket1”, “”)
.config(“spark.couchbase.bucket.bubcket2”, “”)
.getOrCreate()
val optios = Map(
"bucket" -> "bubcket1",
"schemaFilter" -> N1QLRelation.filterToExpression(EqualTo("processed_flag", "N")))
val bubcket1 = spark.read.couchbase(optios)
bubcket1.count()
val bubcket2 = spark.read.couchbase(Map("bucket" -> "bubcket2"))
bubcket2.count
Please help to solve the issue.