Trouble replicating to SG from remote db

Mobile Sync Gateway
1

I have a load of data in a Cloudant db I want to replicate over to a Couchbase instance via SG.

My request:

curl -H 'Content-Type: application/json' -X POST http://localhost:4985/_replicate -d ' {"source": "https://username:password@myaccount.cloudant.com/mydb", "target": "mydb"} '

I get a response e.g.:

{"type":"replication","replication_id":"","continuous":false,"source":"https://username:password@myaccount.cloudant.com/mydb","target":"http://:4985/mydb","docs_read":0,"docs_written":0,"doc_write_failures":0,"start_last_seq":0,"end_last_seq":"0-g1AAAADfeJzLYWBgYMlgTmGQS0lKzi9KdUhJMtUryC8qSS3SS87JL01JzCvRy0styQGqY0pkSLL___9_ViIDqg4TnDqSHIBkUj02TbityWMBkgwNQAqobz8JtkE0HoBoBNmYBQD_Skp0"}#

Which looks like it should start it off but checking the Couchbase instance there are no new docs coming in. Anything I should check here?

2

After confirming that I am actually connecting to the bucket, trying replication again, I now have some logs to show:

2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] stateFnPreStarted got START event: {REPLICATION_START <nil>}
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] sent notificication: &{REPLICATION_ACTIVE <nil> <nil>}
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] Transition from stateFnActiveFetchCheckpoint -> stateFnActive
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] new state: 0x6e74f0
2017-08-16T14:37:32.814Z HTTP:  #005: GET /sayit-engb/_local/60dc917940f753117432289d8d4a69475d36453c  (ADMIN)
2017-08-16T14:37:32.814Z HTTP: #005:     --> 404 missing  (0.4 ms)
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] resp: &{404 Not Found 404 HTTP/1.1 1 1 map[Content-Type:[application/json] Server:[Couchbase Sync Gateway/1.4.0] Date:[Wed, 16 Aug 2017 14:37:32 GMT] Content-Length:[40]] 0xc42044e500 40 [] false false map[] 0xc42012c870 <nil>}, err: <nil>
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] 404 trying to get checkpoint, continue..
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] stateFnActiveFetchCheckpoint got event: {FETCH_CHECKPOINT_SUCCEEDED {0  }}
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] call fetchChangesFeed()
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] Transition from stateFnActiveFetchCheckpoint -> stateFnActiveFetchChangesFeed
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] new state: 0x6e7df0
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] stateFnActiveFetchChangesFeed
2017/08/16 14:37:32 Replicate: sequenceNumberToString called with: 0 type: int
2017/08/16 14:37:32 Replicate: sequence is an int
2017/08/16 14:37:32 Replicate: sequenceNumberToString called with: 0 type: int
2017/08/16 14:37:32 Replicate: sequence is an int
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] changes feed resp: &{200 OK 200 HTTP/1.1 1 1 map[Cache-Control:[must-revalidate] Content-Type:[application/json] Date:[Wed, 16 Aug 2017 14:37:32 GMT] Etag:["DQ7OKE4TMDSDK9YJIRRLQWSL1"] Server:[CouchDB/2.0.0 (Erlang OTP/17)] X-Couch-Request-Id:[8459ab9f04] X-Content-Type-Options:[nosniff] Via:[1.1 lb1.porter (Glum/1.37.1)] X-Cloudant-Backend:[porter] Strict-Transport-Security:[max-age=31536000]] 0xc42042e1a0 -1 [chunked] false false map[] 0xc4200c03c0 0xc420076790}, err: <nil>
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] event: &{FETCH_CHANGES_FEED_SUCCEEDED {[] 0-g1AAAADfeJzLYWBgYMlgTmGQS0lKzi9KdUhJMtMryC8qSS3SS87JL01JzCvRy0styQGqY0pkSLL___9_ViIDqg4TnDqSHIBkUj02TbityWMBkgwNQAqobz8JtkE0HoBoBNmYBQAAikp2}}
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] stateFnActiveFetchChangesFeed got event: {FETCH_CHANGES_FEED_SUCCEEDED {[] 0-g1AAAADfeJzLYWBgYMlgTmGQS0lKzi9KdUhJMtMryC8qSS3SS87JL01JzCvRy0styQGqY0pkSLL___9_ViIDqg4TnDqSHIBkUj02TbityWMBkgwNQAqobz8JtkE0HoBoBNmYBQAAikp2}}
2017-08-16T14:37:32.881Z HTTP+: #004:     --> 200   (68.6 ms)
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] new state: <nil>
2017/08/16 14:37:32 Replicate: [f650b2f1b88d1949459fb3605fbc1ba5] processEvents() is done

I’m not sure why there are 404 responses here, I’ve confirmed that the source (remote) db responds with a 200 OK to a GET request, and I can do the same with the local target SG DB.

The suspect looks to be:

#005: GET /sayit-engb/_local/60dc917940f753117432289d8d4a69475d36453c  (ADMIN)
#005:     --> 404 missing  (0.4 ms)

And indeed if I try to get that local document I do get a 404. Where/when should that local document have been created?

3

That’s not an error. SG is looking for a pre-existing checkpoint on the source server (Cloudant) and not finding one. This is normal if this is a first-time replication.

1 Like
4

Ah, ok - so the problem is elsewhere. Maybe I’ll drop this for now, as I can also push the data up from a pre-existing script. Replicating would be convenient though.

5

So I tried replicating from Cloudant, and its now pushing up my data, problem solved. Not sure why it wasn’t working the other way round…