Scope & collection limit/performance impact

Couchbase Server
1

Hey, I wanted to bump this topic ( Scopes and Collection limits and sync support for them ), as I am facing similar questions:

I am using couchbase as a gameservice backend and store various different kinds of datatypes. Based on the description here ( How Scopes & Collections Simplify Multi-Tenant App Deployments on Couchbase ). I assumed creating a ‘table’ (collection) for each type of data would be fitting. Then for supporting multiple instances of the service I would use a scope per instance. That setup worked very well for my service, as I need to load a lot of the data completely into memory of the gameservice and then stream changes to CB over time.

But based on the 1000 collection limit (in my test about 1200) and the fact that creation of collections slow down significantly reaching the limit I have some concerns:

  1. What can we assume about the limit in the future? Will it increase in the next versions or is it a hardlimit (based on metadata-size limitations maybe)?
  2. I assume the way I am using scopes & collections is not the intended use. Lets say I want to setup 20 instances of the same service, each with 150 ‘Tables’ of data. How should I approach this correctly? Scopes for the instances seemed like a nice way of encapsulating independent data

Thanks
Kademlia

grafik
grafik1315×1050 67.8 KB

2

@Kademlia Your example in #2 (20 scopes each with 150 collections) is an expected use. This is a pretty typical way of implementing multitenancy.

3

@Kademlia as Kevin also said your usage is pretty typical multi-tenancy and seems an appropriate use of Scopes and Collections.
What sort of slowness in collection creation are you observing? It is not an expected behavior (at least not noticably) for collection creation to slow down as you get close to 1000. Can you provide more details:

  • how are you creating the collections? REST API, SDK?
  • how much time does it take to create your collections (min. max)?
  • logs if you can provide

We may consider increasing the 1000 limit at a future point.

Thanks
–Shivani