Spark Connector 2.1 in Databricks

Spark Connector
,
1

Hi,

I was wondering how in scala to set up my databricks notebook so that I can access multiple buckets for my dataframes? A sample workbook is here Ive been following:

https://docs.databricks.com/spark/latest/data-sources/couchbase.html

However:

a) dbricks doest allow a config key to have an empty value which below indicates

spark.couchbase.bucket.bucket-name

b) this would limit me to one bucket a cluster start up, Id rather do this in the code if possible and be able to access multiple buckets within the same notebook?

Is this possible?

Thanks!

2

Hi,

I think your connector documentation (https://developer.couchbase.com/documentation/server/current/connectors/spark-2.1/spark-intro.html) has everything to answer your questions. On connect you can open more buckets like:

  .config("spark.couchbase.bucket.beer-sample", "") // open the beer-sample bucket with empty password
  .config("spark.couchbase.bucket.foo", "bar") // open the foo bucket with password bar

You can try setting the value of the config as “”, this might work as well. Also, we always recommend to set passwords anyways.

Keep in mind that if you open more than one bucket on each op against couchbase you need to specify he bucket name, since the connector doesn’t know which one you want. See our docs for each method on how to do that.

1 Like