Some documents are not indexed

SQL++
1

Hi,

I have about 100~120 docs with the following data format

{
    "id": 1,
    "form": "product",
    "meta": {
    
    }
    "values": {
        "field1": "value",
        "field2": "value"
    }
}

I created an index to cover id, meta, and values.

Covering index

create index `test_cover` on bucket(id, meta, `values`) where form="product"

Regular index

create index `test_form` on bucket(form) where form="product"

I’m seeing unexpected behavior with covering index.

The following query does not return all docs that should be in the index

select * from bucket use index (`test_cover`) where form = 'product'

but non-covering index does

select * from bucket  where form = 'product'

Is this a bug…?

Update1: Is there size limitation on covered field? Only diff I see between indexed doc and unindexed doc is that unindexed doc has larger values field.

2

There is limit on index key size. If that limit exceeds it will not index that document and entry will be logged in indexer.log . This is been expected.

You have 2 options

  1. Reduce the number of index keys or size of index key
  2. Increase index keysize by changing settings. cc @deepkaran.salooja
3

Ah…that makes sense.

What settings should I change? Is there a doc on it?

Thanks!

4

curl -X POST http://<ip_address>:9102/settings -u username:password -d ‘{“indexer.settings.max_seckey_size”: <new_size_in_bytes>}’

5

Thank you. Is there a way to see the current value?

6

curl http://<ip_address>:9102/settings -u username:password

you can look for the setting value in the returned json.

1 Like
7

Can this be used in 4.5.1?

I just posted the setting, but I don’t see it coming back from /settings response.

8

Yes, this setting works in 5.0 and above. For 4.5.1, you’ll have to limit the secondary key to less than 4k.

9

Thanks! In that case…I guess I have limit the key for now.