Couchbase - best practice for storing huge listts

Couchbase Server
, ,
1

Hi,
I’m trying to evaluate Couchbase capabilities.

Is Couchbase capable of storing multiple lists, each holds between 100,000-100,000,000 records?
The records are stored in a “data series” fashion (or delayed queue) and queried accordingly.

Example
List dataset structure:

  • id
  • list_id # the list the record belongs
  • next_check timestamp
  • status
  • some other fields…

Typical use case:
Select All records that have next_check in the past and a specific status.

SELECT * FROM RECORDS
WHERE next_check < now()
  AND status = X
  limit, offset

Then I can perform several actions:

  • Update the record with a new next_check/status values.
  • OR delete the record and insert a new one.

Questions

What I’m trying to understand is this:

  1. If Couchbase can handle such huge dataset
    • Huge lists of 100,000 and up to 100,000,000.
    • We don’t have a lots of parallel reads, max 10 reads per list (for palatalization).
    • Once a record is read, its going to be updated with a new status/next_check or completely deleted.
  2. What is the best way to store and query such structure?
  3. and finally, is there any Couchbase limitation I need to pay attention to (i.e don’t use more then 1000 buckets)?

Thanks!

2

@eldad87 ,

Sounds like you are new to Couchbase and you are asking some basic and some advanced questions.
I would recommend you take some of the free online training here:

I think the above classes will short circuit your learning curve.