Enterprise Edition 5.0.0 build 3519 on MacOS 10.12.6
When I try to load documents via cbdocloader I get the following error:
Errors occurred during the view creation phase. See logs for details.
All 22 documents appear under ‘items’ of the bucket, they even all show a content sample. However, when clicking on some of the documents (document edit), nothing appears on the screen other than 'Warning: Editing of document with size more than 256 KiB is not allowed".
I had the same issue in the beta versions of 5.0 as well.
Can you please send your log file for your cbdocloader? It should have details on what might have gone wrong.
You can also log a bug and attach your log files too
Yes this helps. I am able to reproduce the same issue on my local as well. Looking at the logs. Are you using a zip file or a directory for your dataset?
When I tried to repro, I loaded the beer-sample data which has views and indexes defined to a single node cluster. But I forgot to add the required services during initialization and hence the following error was thrown:
Couchbases-MacBook-Pro-17:bin a.senthilnathan$ ./cbdocloader -c 127.0.0.1:8091 -u Administrator -p password -b default -d …/samples/beer-sample.zip -m 512 -v
2017/10/26 18:11:00 Bucket default already exists, skipping creation
2017/10/26 18:11:00 Reading view definitions from beer-sample/design_docs/beer.json
2017/10/26 18:11:00 Reading index definitions from beer-sample/design_docs/indexes.json
2017/10/26 18:11:00 ERRO: No available N1QL nodes. – docloader.(*jsonSampleImporter).Queries() at sample_importer.go:204
2017/10/26 18:11:00 ERRO: No available N1QL nodes. – docloader.(*jsonSampleImporter).Queries() at sample_importer.go:204
2017/10/26 18:11:00 Loading data into the default bucket
2017/10/26 18:11:02 Loaded 7303 items into the default bucket
Errors occurred during the index creation phase. See logs for details.
I am going to retry this with services added. Can you also provide some details about your dataset? Does it have any views or indexes defined? Do you have services added?
These JSON files are dumped from an API, never been couchbase, so there’s no indexes or anything like that. They’re loaded from a directory and not a zip file. If I recall correctly, when I was running 5.0 beta and tried the zip file it resulted in the same issue.
By the way, Couchbase 5.0 has a new tool called ‘cbimport’ which can import text files containing JSON, csv files, and files in the cbdocloader format. cbimport is faster than cbdocloader, and it might not have the bug you are running into.
I tried cbimport as well, and while it doesn’t throw an error when importing the data, I’m still seeing symptoms of the issue relating to cbdocloader as well. Even though the documents appear to be loaded, and you can see a content sample of each document, some of them are actually empty when you click on them - see the screenshots below as an example
devnullaby:bin jonernster$ ./cbimport json -c 127.0.0.1:8091 -u Administrator -p couchbase -b nhl -d /Users/jonernster/projects/personal/nhl/nhl-data/ -f sample -v
2017-10-27T11:01:07.402-06:00 (Rest) GET http://127.0.0.1:8091/pools 200
2017-10-27T11:01:07.404-06:00 (Plan) Executing transfer plan
2017-10-27T11:01:07.404-06:00 (Plan) Checking for data movement restrictions between nhl and nhl
2017-10-27T11:01:07.404-06:00 (Plan) Transfering bucket configuration for nhl to nhl
2017-10-27T11:01:07.406-06:00 (Rest) GET http://127.0.0.1:8091/pools/default/buckets 200
2017-10-27T11:01:07.406-06:00 Editing bucket to conform with settings provided
2017-10-27T11:01:07.411-06:00 (Rest) POST http://127.0.0.1:8091/pools/default/buckets/nhl 200
2017-10-27T11:01:07.420-06:00 (Rest) GET http://127.0.0.1:8091/pools/default/buckets/nhl 200
2017-10-27T11:01:07.420-06:00 (Plan) Transfering views definitions for nhl to nhl
2017-10-27T11:01:07.421-06:00 (Rest) GET http://127.0.0.1:8091/pools/default/nodeServices 200
2017-10-27T11:01:07.421-06:00 (Plan) Executing queries against nhl to nhl
2017-10-27T11:01:07.421-06:00 (Rest) GET http://127.0.0.1:8091/pools/default/nodeServices 200
2017-10-27T11:01:07.423-06:00 (Rest) GET http://127.0.0.1:8091/pools/default/buckets 200
2017-10-27T11:01:07.423-06:00 (Plan) Transfering full text index definitions for nhl to nhl
2017-10-27T11:01:07.423-06:00 (Plan) Deciding which key value data to transfer for nhl
2017-10-27T11:01:07.423-06:00 (Plan) Transfering new key value data for nhl to nhl
2017-10-27T11:01:07.425-06:00 (Rest) GET http://127.0.0.1:8091/pools/default/buckets 200
2017-10-27T11:01:07.427-06:00 (Rest) GET http://127.0.0.1:8091/pools/default/nodeServices 200
2017-10-27T11:01:07.430-06:00 (Rest) GET http://127.0.0.1:8091/pools/default/buckets 200
2017-10-27T11:01:07.663-06:00 (Plan) Data transfer completed after 224.732015ms
2017-10-27T11:01:07.663-06:00 (Plan) Transfer of all data complete
2017-10-27T11:01:07.663-06:00 (Plan) Transfer plan finished successfully
Json /Users/jonernster/projects/personal/nhl/nhl-data/ imported to http://127.0.0.1:8091 successfully
Also, I messaged jira administrators to get a jira account to add content to the existing ticket as it doesn’t appear I’m able to currently.
Since you are not able to add content to the Jira ticket, is the document file small enough to email? If so, just use my first name @couchbase.com. I’ll attach it, and see if I can reproduce it.
Jon sent the data file, which is now attached to the Jira ticket. I tried loading it with cbimport into both 5.0GA and 5.1 on my Mac, and it loaded just fine, all 22 records.
I also tried using cbdocloader, and saw the same error messages, but it does appear that all 22 documents were loaded successfully. E.g. if you run the following N1QL query, which picks up a field from either of the two types of documents:
select copyright, total from nhl;
It will show you something for all 22 documents. You can do “select * from nhl”, but it takes longer to display.
Jon - I just realized the problem with the blank document editor. If you look closely at the amber-colored band above the doc editor, there is a warning message saying, “Warning: Editing of document with size more than 256 KiB is not allowed”. So some of your documents are too large for viewing in the doc editor. (We are actually working on an improved doc editor for a future release that should avoid that problem.)
Thanks Eben. It looks like the amber colored band appears in all/most of the documents when you click on them. I’m trying to understand why some are visible and some aren’t.
On my machine, I can view that events file through the document editor, but not the shifts file, which is smaller than the events file. It makes sense that you can’t modify certain documents above a certain size but not being able to view them seems odd; seem like that would make things very difficult for analysis and querying if you can’t view the contents of documents.
@jmernster, it seems that your documents are pushing the limits of our UI. There is definitely a bug in how the document editor handles them, since I also see it showing the warning all the time. The “Schema Insights” panel also has trouble showing the document structure, it gets bogged down but shows it eventually. Even “select * from nhl” in the query workbench shows a warning message, and is slow, though you could do o.k. with ‘select * from nhl limit 2’.
So we have some bugs to fix in handling very complex documents. Thanks for helping expose our flaws!
Got it, thanks Eben. It’s a new dataset for me so I wasn’t sure if there were potential issues/formatting issues with the documents or if we were running into limitations/bugs in the system. Appreciate the help/troubleshooting from all of you.
There is definitely a bug in how the document editor handles them, since I also see it showing the warning all the time. The “Schema Insights” panel also has trouble showing the document structure, it gets bogged down but shows it eventually. Even “select * from nhl” in the query workbench shows a warning message, and is slow, though you could do o.k. with ‘select * from nhl limit 2’.