Cbbackup generates files with question marks in parent directory of destination directory

Couchbase Server
1

We’re using cbbackup to create a complete backup (ie -m full) of our cluster that contains 3 nodes and has about 189GB of data. We’re currently running the following command:

/opt/couchbase/bin/cbbackup http://localhost:8091 -b bucket1 -u xxx -p xx /backup1

What happens:

The backup runs from end to end and when it has completed the directory /backups1 has a few kilobytes of data: 264KB to be exact. This is strange and then at the level above /backups1 there are a ton of files that look like so:

??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????b 
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????b 
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????�????????????????????????????????????????缩

Which when you add up the size of those files it comes to hundred and something GB which would be the contents of our backup.

Setup Details:

CentOS Linux release 7.2.1511 (Core) 
Python 2.7.12 
Linux hostname.here 3.10.0-327.28.3.el7.x86_64 #1 SMP Thu Aug 18 19:05:49 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

I tried attaching the logs from the 3 nodes that were generated by the couchbase UI but it keeps timing out.

2

Hey @rgomes it looks like you might be hitting Loading... which was fixed in 4.5.0. Which version of Couchbase Server are you running?

3

Yeah we’re running 4.1.1

The interesting thing is that I am able to run the backup fine without the same issue and using the same version but a debian based system instead of our cent os 7 image. Do we know if the issue has to do with sqlite or python version ?

4

The issue seems to arise from incorrect character encoding, internally the problem was reproduced on a CentOS machine but not on Debian-based variants, so perhaps that does play a part also. If you have access to the machine performing the backup you could try making the changes in https://github.com/couchbase/couchbase-cli/commit/648a51b3e9ff24ab61491036e70758822876ff28 to see if that resolves the issue.
The file that you would want to change is /opt/couchbase/lib/python/pump_bfd.py.

Hope this helps!