Java application is faster than the insertion into couchbase

Java SDK
,
1

I have a java application that reads an excel file. While I am processing the file row by row, I am doing a query N1QL to check if the Id (first cell of each row) is already in the couchbase. if not, it inserts it (bucket.insert(doc), otherwise it will upsert it.

Problem: the process is so fast that I insert a row and then if I try to insert the same one, the query doesn’t give any value (I believe it doesn’t have time to update the index)

What I tried so far:

  • Use ScanConsistency.REQUEST_PLUS when I build the N1qlQuery object. - Didn’t work

I am using couchbase-client 1.4.13 and java-client 2.7.9

2

Hey @Fabio_Cardoso

Could you possibly post a code snippet showing what you’re doing?

You may want to consider using the bucket.exists() call instead of N1QL, it will likely be faster as the request can be sent directly to the correct KV node and there will be no query parsing required. You could also do bucket.insert() and handle DocumentAlreadyExistsException - that will be faster still, if most of the time you expect to be inserting.

Update: actually, rereading your description, it sounds like you just want to bucket.upsert() all the time, and skip the N1QL exists check? Upsert will insert if it doesn’t exist, and replace if it does, which sounds like what you want?

1 Like
3

Hey @graham.pople tank you for your reply,

I’m not using the upsert because I am generating each id like UUID, but I am comparing and internal Id from the JSON object itself.

This is the method to query the Id (I removed the ScanConsistency thing):

@Override
	public JsonObject getDocument(String keyName, String keyValue) {

		final String statement = "SELECT META().id,* " + "FROM `" + bucket.name() + "` " + "WHERE " + keyName + " =  '"
				+ keyValue + "'";

		final N1qlQuery query = N1qlQuery.simple(statement);
		final N1qlQueryResult result = bucket.query(query);

		final Optional<N1qlQueryRow> obj = result.allRows().stream().findFirst();
		if (obj.isPresent()) {
			return obj.get().value();
		}
		return JsonObject.create();
	}

then I am just checking if he json is empty or not to know if I need to call upsert or create

final JsonObject dbDocument = getDocument(key, jsonObj.getString(key));
			if (dbDocument.isEmpty()) {
(...)

I am using meta.id to know what document I want to upsert if it is the case.