Range() response failed `Index scan timed out`

SQL++
1

Hi,

We’ve noticed very slow response from n1ql.

When I check /opt/couchbase/var/lib/couchbase/logs/query.log, I see the following errors.

2016-04-26T19:52:02.028+00:00 [Warn] Trying scan again for index 16015133227865254038 (true true): Index scan timed out ...
2016-04-26T19:52:02.939+00:00 [Error] [GsiScanClient:""] Range() response failed `Index scan timed out`

I actually do not even understand what that error means. Could someone please give me a hint on it?

2

Hi,

This means indexer hits its default scan timeout during indexer scan.

Couple of questions:

  1. Can you please share the query? How many documents are expected to return per query?
  2. What’s the size of the expected response for query?
  3. How many documents are there in the index? Can you please share index definition as well?

Thanks,
Qi

3

How can I find out which query is causing it? We are running many queries.
Let me try my best…

  1. This is one of the queries that takes longer than other queries.
select * from mybucket bucket1 where bucket1.clientId = 'aValue' and bucket1.form = 'aValue' and bucket1.`values`.`slug` = 'aValue'

Explain of the query:

{
    "requestID": "02ac34a7-6dfb-4b31-b8a8-18b5dbc597c5",
    "signature": "json",
    "results": [
        {
            "#operator": "Sequence",
            "~children": [
                {
                    "#operator": "IntersectScan",
                    "scans": [
                        {
                            "#operator": "IndexScan",
                            "index": "idx_product_type_clientId",
                            "keyspace": "catalog",
                            "namespace": "default",
                            "spans": [
                                {
                                    "Range": {
                                        "High": [
                                            "\"aValue\""
                                        ],
                                        "Inclusion": 3,
                                        "Low": [
                                            "\"aValue\""
                                        ]
                                    }
                                }
                            ],
                            "using": "gsi"
                        },
                        {
                            "#operator": "IndexScan",
                            "index": "idx_product_type_slug",
                            "keyspace": "catalog",
                            "namespace": "default",
                            "spans": [
                                {
                                    "Range": {
                                        "High": [
                                            "\"aValue\""
                                        ],
                                        "Inclusion": 3,
                                        "Low": [
                                            "\"aValue\""
                                        ]
                                    }
                                }
                            ],
                            "using": "gsi"
                        },
                        {
                            "#operator": "IndexScan",
                            "index": "idx_product_type_form",
                            "keyspace": "catalog",
                            "namespace": "default",
                            "spans": [
                                {
                                    "Range": {
                                        "High": [
                                            "\"aValue\""
                                        ],
                                        "Inclusion": 3,
                                        "Low": [
                                            "\"aValue\""
                                        ]
                                    }
                                }
                            ],
                            "using": "gsi"
                        }
                    ]
                },
                {
                    "#operator": "Parallel",
                    "~child": {
                        "#operator": "Sequence",
                        "~children": [
                            {
                                "#operator": "Fetch",
                                "as": "bucket1",
                                "keyspace": "catalog",
                                "namespace": "default"
                            },
                            {
                                "#operator": "Filter",
                                "condition": "((((`bucket1`.`clientId`) = \"aValue\") and ((`bucket1`.`form`) = \"aValue\")) and (((`bucket1`.`values`).`slug`) = \"avalue\"))"
                            },
                            {
                                "#operator": "InitialProject",
                                "result_terms": [
                                    {
                                        "expr": "self",
                                        "star": true
                                    }
                                ]
                            },
                            {
                                "#operator": "FinalProject"
                            }
                        ]
                    }
                }
            ]
        }
    ],
    "status": "success",
    "metrics": {
        "elapsedTime": "91.843686ms",
        "executionTime": "91.770049ms",
        "resultCount": 1,
        "resultSize": 4296
    }
}

Indexes for the query

CREATE INDEX `idx_product_type_clientId` ON `mybucket`(`clientId`) WHERE (`form` = "product_type") USING GSI

CREATE INDEX `idx_product_type_slug` ON `mybycjet`((`values`.`slug`)) WHERE (`form` = "product_type") USING GSI

CREATE INDEX `idx_product_type_form` ON `mubucket`(`form`) WHERE (`form` = "product_type") USING GSI
  1. For this particular query, doc is less than 50KB
  2. just one.

Update: We rebuilt all indexes. We are not getting the error anymore, but I would like to know what caused this error. It was such an unexpected behavior. Is there a way to prevent it next time?

We currently have 34 million documents in our bucket.

4

Hi, could you share your indexer.log - I am at cihan@couchbase.com. it may be due to an issue in the indexer. Did you have multiple copies of the index or do you have a single copy of the index?

5

Hi @cihangirb

We actually lost log files. Everything has been running smoothly since then. I will make sure to get the file if it happens again.

Thank you!

6

thanks - let me know if you hit the issue again.