How to sync related documents?

Mobile Sync Gateway
1

Been mashing my brain on this. I have a multi-tenant application with many different types of content, and groups that have different permissions for each type. Because of the multi-tenant aspects I have channels for each tenant+type+group, and when I process each document, I check metadata (tenant, target groups, etc) on the document and publish it to the various correct channels. All this seems to work fine.

The problem is that I want to be able to link comments to the original content items. When processing the comment document in the sync function though, how do I access information about the original content item to ensure that the comments are only synched in the same channels as the referenced original content?

How do you handle joins between documents in the Sync Function?

2

Hi @chrismbeckett,

You can’t access other docs in the sync function. But you can have a property on the comment documents that references the original content items.

Then, in the Sync Function, map the original content and comments to the same channel.

If you change the channel of one document though, the other documents won’t be remapped to the new channel. You will have to create a new revision for those documents as well. This is a fairly common pattern when doing migrations that change the data model and potentially the access rules as a result.

Does that answer your original question?

James

3

I have been stuck on this now for a couple of days. I am sorry but I should have provided an example - I know these sorts of problems can be conceptually challenging.

First I have a group:

{"_id": “grp-xxx”, “type”: “group”, “tenant”: “tenant1”, “group”: “Test1”, “members”: [“userA”, “userb”]}

So in my sync function I create a channel for the tenant/group and grant the members of the group access to it. Anything I put into that group’s channel will only be seen by that group.

Then I have record that should only be seen by this group:

{"_id": “rec-xxx”, “type”: “record”, “tenant”: “tenant1”, “groups”:[“Test1”] …}

So in my sync function, I add the record to the tenant/group channel. Then I have a note on the record:

{"_id": “note-xxx”, “type”: “note”, “tenant”: “tenant1”, “rec-ref”: “rec-xxx” … }

So in my sync function, the note doesn’t know what groups the original record was assigned to, so I have no way of knowing how to add it to the appropriate group channel(s).

I had thought about creating a channel for each record itself, so that all notes could be added to the record channel, but the record document doesn’t know about the members of the group, so I have no way to assign the correct members to the channel if I create it for the record instead of the group. [This problem could be overcome if my group was an actual Sync Gateway role, because then I could just assign the role to the channel, but since groups are dynamic, and roles can only be created by administrators on the admin REST API on the server, outside the Sync function, my groups are not currently roles].

My app has a lot of different groups, access rights, types of documents, and references between documents like this.

I don’t really see that I am doing anything that isn’t common practice in NoSQL storage design, but the whole application is very dynamic: dynamic tenants, dynamic groups, dynamic permission sets, dynamic logical storage containers (with individual security permissions) etc. All in all, pretty standard stuff for an enterprise business application.

Trying to figure out how to apply a security model is becoming mental gymnastics and I am having to re-design my whole storage model over and over as I come across more and more scenarios like this I can’t figure out.

Probably the biggest underlying issue is the limitations on roles. You can’t dynamically create them in the Sync function, you can’t get the list of members of a role in the Sync Function, and you can’t assign roles to other roles.

Where I am headed now is re-factoring to make my groups actual Sync Gateway roles. This creates a lot more work for me, and introduces more user scenarios to my mobile app that won’t work offline (if I try to solve this with a REST API). I am trying to get my head around whether I can make group documents a stateful workflow with server-side agents to provision the group, but I have not gotten that far yet…

4

There are two solutions that come to mind for this from me:

  1. Copy the groups from the record to the note before you push it up to sync gateway

  2. Embed the notes directly into the record as an array

Once you start depending on external state in the sync function things become unpredictable very quickly. For example, what if the record groups changed at a later date? Or what if the sync function were to be run again as the result of a _resync call? For these reasons the sync function can rely only on the state of the revision being examined to ensure identical results for every run.

So let me know if one of those two solutions is useful for you.

5

I really appreciate the suggestions. Here was my final solution:

  • Each group maps to a Sync Gateway role, and I have a server-side agent watch for changes to groups, and update the users assigned to the role.

  • A Folder document identifies the application storage containers for content. Each folder becomes a channel, and the Folder document has a groups array property with the groups allowed to access the folder. The sync gateway grants the Group role access to the Folder channel.

  • Content documents (like records) and any related Comments are stamped with the Folder ID. They sync function adds the document to the Folder channel.

It creates an almost dynamic model that maps to the basic behavior of the Sync Gateway with roles granted access to channels, and documents published to channels. Because roles cannot be created dynamically like channels, to make it work my Groups had to become actual Sync Gateway roles, and that required a server side agent to do the actual role provisioning.

Still working on getting it all coded and tested, but I think this should work.