Cbbackup/cbrestore issue

Couchbase Server
1

Hello,

I have an issue with cbbackup/cbrestore on Couchbase Server 6.0.0 CE. I am using backup and restore without issue until I enabled the use of xattrs in Sync Gateway 2.6.0.

I do a backup in a 3 nodes cluster with this command :

/opt/couchbase/bin/cbbackupwrapper --username=${COUCHBASE_USERNAME} --password=${COUCHBASE_PASSWORD} --parallelism=4 http://couchbase-master-0.couchbase-service.default.svc.cluster.local:8091 /backup/couchbase`

and I try to restore it in a single node cluster with :

/opt/couchbase/bin/cbrestorewrapper --path=/opt/couchbase/bin -u ${COUCHBASE_USERNAME} -p ${COUCHBASE_PASSWORD} /dump/ http://localhost:8091

It ends with some errors :

    2019-11-23 09:30:55,570: mt cbrestore...
    2019-11-23 09:30:55,570: mt  source : /dump/200-299
    2019-11-23 09:30:55,570: mt  sink   : http://localhost:8091
    2019-11-23 09:30:55,570: mt  opts   : {'username': '<xxx>', 'verbose': 1, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'dcp_consumer_queue_length': 1000.0, 'data_only': 0.0, 'uncompress': 0.0, 'nmv_retry': 1.0, 'conflict_resolve': 1.0, 'cbb_max_mb': 100000.0, 'report': 5.0, 'mcd_compatible': 1.0, 'try_xwm': 1.0, 'backoff_cap': 10.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'flow_control': 1.0, 'batch_max_size': 1000.0, 'seqno': 0.0, 'design_doc_only': 0.0, 'allow_recovery_vb_remap': 0.0, 'recv_min_bytes': 4096.0}, 'collection': None, 'ssl': False, 'threads': 1, 'to_date': None, 'key': None, 'password': '<xxx>', 'id': None, 'bucket_source': 'parametre', 'silent': False, 'dry_run': False, 'from_date': None, 'bucket_destination': None, 'add': False, 'vbucket_list': None, 'separator': '::'}
    2019-11-23 09:30:55,575: mt Starting new HTTP connection (1): localhost
    2019-11-23 09:30:56,486: mt bucket: parametre
    2019-11-23 09:30:56,556: s0 error: async operation: error: MCSink MC error: 135 on sink: http://localhost:8091(parametre@couchbase-master-0.couchbase-service.default.svc.cluster.local:8091)
    2019-11-23 09:30:56,556: w0   source : /dump/200-299(parametre@couchbase-master-0.couchbase-service.default.svc.cluster.local:8091)
    2019-11-23 09:30:56,557: w0   sink   : http://localhost:8091(parametre@couchbase-master-0.couchbase-service.default.svc.cluster.local:8091)
    2019-11-23 09:30:56,557: w0          :                total |       last |    per sec
    2019-11-23 09:30:56,577: w0   source : /dump/200-299(parametre@couchbase-worker-0.couchbase-service.default.svc.cluster.local:8091)
    2019-11-23 09:30:56,578: w0   sink   : http://localhost:8091(parametre@couchbase-worker-0.couchbase-service.default.svc.cluster.local:8091)
    2019-11-23 09:30:56,578: w0          :                total |       last |    per sec
    2019-11-23 09:30:56,578: w0    batch :                    1 |          1 |       80.0
    2019-11-23 09:30:56,578: w0    byte  :                    0 |          0 |        0.0
    2019-11-23 09:30:56,578: w0    msg   :                    0 |          0 |        0.0
    2019-11-23 09:30:56,600: w0   source : /dump/200-299(parametre@couchbase-worker-1.couchbase-service.default.svc.cluster.local:8091)
    2019-11-23 09:30:56,600: w0   sink   : http://localhost:8091(parametre@couchbase-worker-1.couchbase-service.default.svc.cluster.local:8091)
    2019-11-23 09:30:56,600: w0          :                total |       last |    per sec
    2019-11-23 09:30:56,600: w0    batch :                    1 |          1 |       89.4
    2019-11-23 09:30:56,600: w0    byte  :                    0 |          0 |        0.0
    2019-11-23 09:30:56,600: w0    msg   :                    0 |          0 |        0.0
    error: MCSink MC error: 135

In the server, I have this in the logs :

2019-11-23T09:30:57.170637+00:00 WARNING 122: Invalid format specified for DEL_WITH_META - 135 - closing connection packet:mcbp::header: magic:0x80, opcode:0xa8, keylen:51, extlen:28, datatype:0x4, specific:66, bodylen:79, opaque:0x1000000, rawextras:000000000000000b15d8de773f17000000 
2019-11-23T09:30:57.187145+00:00 WARNING 138: Invalid format specified for DEL_WITH_META - 135 - closing connection packet:mcbp::header: magic:0x80, opcode:0xa8, keylen:50, extlen:28, datatype:0x4, specific:294, bodylen:78, opaque:0x10000000, rawextras:000000000000000de15d8df648a5000000 
2019-11-23T09:30:57.293212+00:00 WARNING 136: Invalid format specified for DEL_WITH_META - 135 - closing connection packet:mcbp::header: magic:0x80, opcode:0xa8, keylen:51, extlen:28, datatype:0x4, specific:762, bodylen:79, opaque:0xa000000, rawextras:000000000000000915d8de782ff000000 
2019-11-23T09:30:57.524271+00:00 INFO 117: Client 127.0.0.1:34846 authenticated as <ud>admin123</ud>
2019-11-23T09:30:57.524449+00:00 INFO 117: HELO XATTR, XERROR [ 127.0.0.1:34846 - 127.0.0.1:11210 (<ud>admin123</ud>) ]
2019-11-23T09:30:57.547122+00:00 INFO 116: Client 127.0.0.1:34888 authenticated as <ud>admin123</ud>
2019-11-23T09:30:57.547541+00:00 INFO 116: HELO XATTR, XERROR [ 127.0.0.1:34888 - 127.0.0.1:11210 (<ud>admin123</ud>) ]
2019-11-23T09:30:57.850114+00:00 INFO 110: Client 127.0.0.1:35132 authenticated as <ud>admin123</ud>
2019-11-23T09:30:57.850231+00:00 INFO 110: HELO XATTR, XERROR [ 127.0.0.1:35132 - 127.0.0.1:11210 (<ud>admin123</ud>) ]
2019-11-23T09:33:29.287453+00:00 INFO 106: Client 127.0.0.1:39541 authenticated as <ud>@ns_server</ud>

I already that an similiar issue exists in couchbase <6.0.0 (https://issues.couchbase.com/browse/MB-31224) but it should be fixed in the version 6.0.0 I am using.
Is there a workaround for this kind of issue ? I do not have backup working anymore…

Thanks