Get Flags (0x2) indicate non-JSON document for id ..., could not decode

Java SDK
1

When I trie to get record from bucket, I often get the error
Get Flags (0x2) indicate non-JSON document for id …, could not decode
If I visit the doc on web, the flags is not 0x2.

What is the reason of 0x2 occured?
Is there anyway I can get the doc ignor flags?
Thanks!

2

@jiuchang2004, did you use Couchbase v3.0 at some point of time? I guess you’re using Couchbase Java SDK v2.x?

3

Yes, server is 3.0.3, client is java sdk 2.1.2

4

Is that the only version which you’ve used for v3.x? Original v3.0 release had a bug related to byte swapping of document flags and hence we rolled out v3.0.1 containing the fix.

Also could you clarify, since when are you seeing this issue? Was this after some server maintenance or Couchbase java client upgrade?

5

In the cb cluster, there are one 3.0.3 and one 3.0.2.
The data in bucket is OK, its flags=33554432=0x2000000. Sometimes, the client may report an error that flag=0x02.
I record that in my log files.
I tried there times when error occurred, and I may get the correct answer.

6

are you sure the document isn’t replaced somewhere else? what’s the usage and update pattern of this document?
the common flag for JSON is indeed 0x2000000 (which is hexadecimal for what you see in the console).
the fact that the SDK reports a flag of 0x2 (hex for 2) makes me think that another SDK is also mutating the document…

7

The doc may be changed, but the flags is not.
The writing program will modify the content about 10 times, using jdk 2.1.2.
Codes like this:

public static void upsert(Bucket bucket, String key, String json) {
	try {
		if (cluster != null && bucket != null)
			bucket.upsert(RawJsonDocument.create(snt_sha(key), json),
					bucket_write_timeout, TimeUnit.SECONDS);
	} catch (Exception e) {
		logger.error("upsert error, for :" + e.toString() + "|" + key);
	}
}

While the reading programs just read data with the code below:

private Object get(String id, Bucket bucket) {
	RawJsonDocument doc = null;
	int flag = 0; 
	try {
		doc = bucket.get(id, RawJsonDocument.class, timeout,
			TimeUnit.MILLISECONDS);
		if (doc != null)
			return doc;
	} catch (TranscodingException e) {
		logger.warn("Get " + e.getMessage() + " for doc " + id);
		flag = 1;
	} catch (Exception e) {
		logger.warn("Get " + e.getMessage() + " for doc " + id);
		flag = 2;
	}
	return flag;
}
8

ok, I’ll look into it.
on a side note, you can edit your message to make code (or logs) more readable: use the “code block” syntax:

```LANGUAGE-FOR-HIGHLIGHTING
your.code.here;
```

For example the following:

```java
System.out.println("test");
```

gives:

System.out.println("test");
9

I tried to reproduce the issue using SDK 2.1.2 and your code snippet, but couldn’t. I modified it to run upsert then get on key testFlag in a loop, 100 000 times.

what does the snt_sha method do? can you give an example of what parameters you use for a problematic upsert call and the corresponding get call? shouldn’t you use the snt_sha method in get as well?

10

snt_sha is a function encode the key into a sha code string.

	protected static String snt_sha(String snt) {
		return DigestUtils.shaHex(snt);
	}
11

ok, still couldn’t reproduce, can you answer my 2 other questions as well?

12

Glad that I can help.

13

I have another question.
How could I get a document with wong flags use Java client?
The web-console can get the document and decode the content correctly.

14

You could either use the LegacyDocument or implement your own Document/Transcoder that ignores the flags. How do your flags look like, btw?

15

I store docs as JsonDocument, but when I get the doc, the flags maybe 0x2.
If the doc stored use php sdk, the flags maybe other values, like 0x4000000, 0x6000000, etc.

16

can you show the code on how you read and write it, with the exact client versions for each?

17

I implemented the Document/Transcoder, which doesn’t validate the flags only output logs.
I get the following log:
192.168.1.106:7092 22:36:29 INFO [org.pigai.couchbase.CikuuTranscoder:31] - Flags (0x2) indicate non-JSON document for id 845d955dfec831dbcac5c5c37d677df8ba5ccef9, could not decode. content={“0”:{“w”:"^",“l”:"^",“p”:"_decl",“b”:"",“t”:2,“s”:{}},“1”:{“w”:“I”,“l”:“i”,“p”:"_r",“b”:“anim,humn”,“t”:1,“s”:{“1”:"_n"}},“2”:{“w”:“bless”,“l”:“bless”,“p”:"_v",“b”:"",“t”:2,“s”:{}},“3”:{“w”:".",“l”:".",“p”:"_h",“b”:"",“t”:3,“s”:{}}}
The meta data which get using web consol is:

meta: {
    id: "845d955dfec831dbcac5c5c37d677df8ba5ccef9",
    rev: "2-0034940b2b18b1780000000002000000",
    expiration: 0,
    flags: 33554432
},

My program uses Jetty as the web server, run in a multithreads mode. When an http request comes, it will visit the couchbase using the code:

	public static String getRawJson(Bucket bucket, String key, boolean validated) {
		try {
			if (cluster == null || bucket == null || key == null)
				return null;
			if (validated)
				key = snt_sha(key);

			CikuuJsonDocument doc = null;
			synchronized (bucket) {
				doc = bucket.get(key, CikuuJsonDocument.class,
						bucket_read_timeout, TimeUnit.SECONDS);
			}
			return doc == null ? null : doc.content();
		} catch (Exception e) {
			logger.error("getRawJson error, for :" + e.toString() + "|" + key);
		}
		return null;
	}

After process, if the data is not in couchbase, I will store the data using the following code:

	public static void upsert(Bucket bucket, String key, String json) {
		try {
			if (cluster != null && bucket != null)
				bucket.upsert(RawJsonDocument.create(snt_sha(key), json),
						bucket_write_timeout, TimeUnit.SECONDS);
		} catch (Exception e) {
			logger.error("upsert error, for :" + e.toString() + "|" + key);
		}
	}

	protected static String snt_sha(String snt) {
		return DigestUtils.shaHex(snt);
	}

My cluster has there servers, one is 3.0.2-1603 Enterprise Edition (build-1603-rel), and tow are 3.0.3-1716 Enterprise Edition (build-1716-rel)
The Java client is couchbase-java-client-2.1.2.jar