Change part of document if some logic

Python SDK
1

Dears,

Im trying to use sub document mutating and it seams very easy.
But :slight_smile:
I have questions:
If I can find/get some sub document sub element (list, dict) , get some value from it(add some value), then modify it with sub document mutate.

For example:
I want for given document find “credit” where is_covered = false, get “amount” and subtract some value , then modify with sub document mutating, to not submit whole document.

{
  "id": "812db46dc6a1462ea8272de30d102483",
  "credits": [
    {
      "credit_type": "credit1",
      "id": "fdccc057343f4e96ac2cae8e522d6482",
      "amount": 200,
      "commission": 40,
      "is_covered": false,
      "cover_date": null,
      "create_date": "2018/06/06T16:48:32.523491",
      "left_chunk_quantity": 4,
      "chunks": []
    },
    {
      "credit_type": "credit1",
      "id": "18e0abd72a624b32bdcabe55e560b555",
      "amount": 200,
      "commission": 40,
      "is_covered": true,
      "cover_date": null,
      "create_date": "2018/06/06T16:49:18.261654",
      "left_chunk_quantity": 4,
      "chunks": []
    }
  ],
  "black_list_history": [],
  "black_list_current_status": false,
  "subscriber_create_date": "2018/06/06T16:48:32.520692"
}
2

Hi @asakurajano, normally you would achieve this using a N1QL query - see this command description for some examples:

https://developer.couchbase.com/documentation/server/5.1/n1ql/n1ql-language-reference/update.html#topic_11_8

However, I’m not sure if this rewrites the whole document or just does SD updates on the changed parts. Is it essential for it to be a sub-document operation?

If you want to guarantee that you are using only SD changes, you could do the conditional logic by querying for the relevant documents where is_covered = false, and then performing the relevant SD operations from within Python. Although that might be a lot slower than using the N1QL query to do the mutations.

Thanks,

Ellis

3

Thank you for quick reply.

This is essential because this document could be very big and I need to change only little part of it.

Problem is that I need to query for specific list item in given document, which is dict, get some values from this dict than change it and submit only changed dict.

for example in document

{credit : [
{this:true, value:5},
{this:false,value6}
]}

For given document I need to find documents “credit” keys - value (list) , and from this list get dict which “this” argument == True, get value 5 , add 5 and save 10 , in such way to only change this part of document.

4

Hi @asakurajano, there are many tradeoffs in performance between different approaches, depending (for example) on the location of the data nodes and the query nodes.

What scale are these queries at? How many documents total? And how many likely to be affected in a single run of this process?

Will it be run in a large batch, or on small subsets sporadically?

If the frequency of changes is small, then the overhead from N1QL rewriting an entire document will probably be dwarfed by the performance advantage of running the mutations directly within N1QL.

If the frequency of changes is large and the data set is large, then there may be more benefit from a hybrid approach, i.e. using N1QL to find the fields you need to change, and then using subdoc operations to make the changes.

I’d recommend prototyping using the first approach, as it’s much simpler, then experimenting with the latter approach if you need more performance. It’s a good idea to benchmark this in detail to see where the bottlenecks are.

Thanks,

Ellis

5

Hi @ellis.breen

Query will be run only for one document, but the problem is that in list “credits” may be 1000 of list items.
So if I get document and change something in one list item, for this I need to upsert whole document.

If there is some way to find list item that I filter with some logic change it and upsert only this list item.

As I get what I can do is:

  1. Get whole document
  2. find list item I want in “credits” list save it in variable
  3. That with function mutate_in go to this list change values I want and upsert it.

Am I right?

6

Hi @asakurajano - yes, for one document, this is about the best approach. You could use N1QL to find where the items you want to change are within the document, but this would mean an extra transfer of the whole document to the Query service, which would potentially slow things down quite a lot if it’s large.

Hope that helps.

Many thanks,

Ellis

7

Many thanks @ellis.breen.

1 Like