Hi,
When using Couchbase Lite and Sync Gateway is it possible to only sync documents to the mobile client that are only n number of (e.g. 7) days old or less?
This will ensure that the client does not end up with too many documents over time.
I do not think this is possible to achieve with channels as we would like the client to have access to documents that are older, but be able to configure (on the client) the maximum age of documents that should be sync’ed to the client.
Cheers,
Anton
No, but you can do this using channels. Define channels for time intervals like a week — you could use a naming scheme something like “2015-11” to mean the 11th week of 2015, for instance. Then have the sync function assign a doc to a weekly channel based on a timestamp property in the doc.
Then set up permissions so all clients have access to these channels.
Now the client can request docs for any combination of weeks that it wants, for example the most recent 2 weeks.
Thanks for the fast response!
The actual duration we likely need is one week. So that would create a lot of channels and the number of channels will only continue to increase with time. Making sure that a user gets access to all the time based channels when they really need access to only one “actual” channel is also a pain and something that only gets slower with time.
Am I right that if the sync interval (i.e. how often a client syncs) is smaller than the “cut off” period for the docs, we can achieve what we are after by purging documents on the client after the initial sync?
The only inefficiency then is the initial sync will need to pull all the documents down, and then purge most of them.
Do you think I am missing something?
Really appreciate your help!
Yes, you can purge the old docs on the client. Generally they won’t come back, unless they’re updated, or the replicator has to reset its state and start over (which would only occur if something weird happens like a client db getting restored from a backup.)
We’ve actually just been talking about the possibility of adding CouchDB-style filter functions to SG, which would let you add arbitrary criteria like “is doc.timestamp within the past week”. They have the potential to add a lot of CPU overhead to the gateway, though (which is their drawback in CouchDB, and why we came up with channels.)
We actually want the purged docs to come back if they have been updated. So that is really good 
I see channels as enforcing security. And they are very good for that!
Filter functions would enable to customise which documents to sync to the device (out of the ones a user has permissions to see). The “past week” needs to come as a parameter from the client, as it could be “past two weeks” or “past month”, etc. We are finding that most users would want to set a different setting.
Briefly thinking about it, I think it is possible to back “filter functions” by a view and allow the client to supply the view lookup criteria. I am guessing this would be more efficient. What do you think?
This is something slightly like the way channels work. It’s actually very complex to do.
Then filter functions you are suggesting is our best bet.
Do you know if this feature is scheduled at the moment? Is this something that is likely to be done in the next 6 months?
So far it’s just something that’s been talked about in the hallway, I think. You could look at the Github issue tracker for sync_gateway and file an issue requesting it if there isn’t one already (or add a “+1” comment if there is.)
I have created https://github.com/couchbase/sync_gateway/issues/836 for enhancement,