NPE in DAGScheduler.getShuffleDependencies

Spark Connector
1

Hello,

I try to use com.couchbase.spark.japi.CouchbaseSparkContext -> couchbaseView

My Simple program:

SparkConf sparkConf = new SparkConf().
setAppName(“my_app”).
setMaster("local[]").
set(“spark.couchbase.nodes”, “localhost”).
set(“com.couchbase.bucket.myBucket”, “1111”).
set(“com.couchbase.connectTimeout”, String.valueOf(100 * 1000)).
set(“com.couchbase.keepAlive”, String.valueOf(20 1000));

SparkSession spark = SparkSession.builder().config(sparkConf).getOrCreate();

JavaSparkContext jsc = new JavaSparkContext(spark.sparkContext());

CouchbaseSparkContext csc = couchbaseContext(jsc);

long a = csc.couchbaseView(ViewQuery.from(“my_view”,“my_view”).limit(10).reduce(false)).count();

However, it got NullPointerException:

Exception in thread “main” java.lang.NullPointerException
at org.apache.spark.scheduler.DAGScheduler.getShuffleDependencies(DAGScheduler.scala:431)
at org.apache.spark.scheduler.DAGScheduler.getOrCreateParentStages(DAGScheduler.scala:380)
at org.apache.spark.scheduler.DAGScheduler.createResultStage(DAGScheduler.scala:367)
at org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:850)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1677)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1669)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1658)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:630)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2022)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2043)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2062)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2087)
at org.apache.spark.rdd.RDD.count(RDD.scala:1158)
at org.apache.spark.api.java.JavaRDDLike$class.count(JavaRDDLike.scala:455)
at org.apache.spark.api.java.AbstractJavaRDDLike.count(JavaRDDLike.scala:45)

I use the following dependencies:

org.apache.spark
spark-core_2.11
2.2.0


org.apache.spark
spark-sql_2.11
2.2.0


com.couchbase.client
spark-connector_2.11
2.2.0

Thanks!