Couchbase 6.0 FTS Index definition - is it correct?

Full Text Search
1

Hello,
Could you please help me to figure out what is correct definition for my FTS indexes?
In my couchbase DB I have 2 types of documents let’s say:
{“id”: “cr_Tbqja”, “type”: “car”, “description” : “Volvo”}
and
{“id”: “bk_abqja”, “type”: “bike”, “description” : “Mountain bike”}

I have defined 2 FTS indexes (I am not sure if I suppose to have 2).
For both of them I selected:
image

also first index is “car_fts_index”, having under “Type Mappings” - “car” (selected), also checkbox " only index specified fields" selected.

and second index is “bike_fts_index” with similar definition, having under “Type Mappings” - “bike” (selected), also checkbox “only index specified fields” selected.

What appears strange to me is for both of the indexes I see same number of doc count.
Does this mean it is not indexing separately cars and bikes (by type actually)?
Also is it correct to keep 2 separate FTS indexes, each of them having different type mapping or should I create 1 bigger index having 2 types inside it?

Thanks is advance.

2

Please any idea, I am totally confused with doc count. When specifying a concrete mapping it suppose to index only the documents, that have type=car (or in different case bike), but why for both indexes the doc count is the same???
In real life we are planning to have ~15 FTS indexes, each of them containing 1 type mapping and our biggest concern is performance. Our couchbase server is using 75-80% of memory even when has very little number of operations. So we are worried may be because of our FTS indexes. Should we have just 1 FTS index with all 15 types mappings? It will be difficult to manipulate it.
Waiting for any hints.
Thanks.

3

@bertynat doc_count is indicative of the number of documents in the bucket that the index has processed. It does not represent the number of the documents indexed. We’ve added a tool tip in the upcoming release indicating this to avoid the confusion.

You do have the option to build an index that maps documents of several types. However we’d be able to give you a better suggestion if you can let us know how you intend to use your indexes, as in - what you’d typically be searching for?

4

Thanks Abhinav for your answer.
We have a pretty big entity persons that contains ~90 000 entries. Every person has
{“type”: “person”,
“createdAt”: “2019-09-09T19:21:50.023”,
“updatedAt”: “2019-09-09T19:21:50.023”,
“firstName”: “Steven”,
“middleName”: null,
“lastName”: “Jackson”,
“displayName”: “Steven Jackson”,
“personTypes”: [
“Director”,
“Manager”
]
}
and we have index defined for it:
…
“types”: {
“person”: {
“dynamic”: true,
“enabled”: true,
“properties”: {
“personTypes”: {
“enabled”: true,
“dynamic”: false,
“fields”: [
{
“analyzer”: “keyword-tolower”,
“include_in_all”: true,
“include_term_vectors”: true,
“index”: true,
“name”: “personTypes”,
“store”: true,
“type”: “text”
},
{
“analyzer”: “standard-no-stop”,
“include_in_all”: true,
“include_term_vectors”: true,
“index”: true,
“name”: “personTypesMatch”,
“store”: true,
“type”: “text”
}
]
},
“displayName”: {
“enabled”: true,
“dynamic”: false,
“fields”: [
{
“analyzer”: “keyword-tolower”,
“include_in_all”: true,
“include_term_vectors”: true,
“index”: true,
“name”: “displayName”,
“store”: true,
“type”: “text”
},
{
“analyzer”: “standard-no-stop”,
“include_in_all”: true,
“include_term_vectors”: true,
“index”: true,
“name”: “displayNameMatch”,
“store”: true,
“type”: “text”
}
]
}
…
}
}

Usually we are searching by displayNameMatch or personTypes… But it takes a lot of time - each request ~ 1 sec, maybe somehow I can increase the performance.??

Other our indexes are pretty similar, each entity has name in it, so usually each entity type we are searching by name.

5

@bertynat Having 2 mappings for the same field (displayName) seems a bit wasteful. If you were to just set up a custom analyzer with the toLower filter and the no-stop-word filter, “Steven Jackson” would be returned as a hit for the following search terms …

  • Steven Jackson
  • Steven
  • Jackson
  • steven jackson
  • steven
  • jackson

The same should be applicable for personTypes as well. Try setting up a single mapping with the custom analyzer to do what you’re looking for.

Rather than setting separate indexes for each type mapping - you could setup all mappings within the same index … you would want to set up a default index mapping for that index and index all the required fields within it, along with the “type” field. Now during search, should you want to match all documents that are of a particular type, say person or car - you could explicitly request for {“field”: “type”, “match”: “person”} or {“field”: “type”, “match”: “car”} in conjunction to other search criteria.

6

@bertynat,

1 sec query latency is pretty huge.

whats the memory quota you set for FTS? Is it the default 512mb?
If so, its worth to bump it to a few GBs or 50-60% percent of the RAM available in the node. (provided its an FTS only node)

-Sreekanth