Hi All,
I tried to add an index, but the index was not added and it stuck at 69%.
Couchbase Version is 5.0. so i checked the processlist, and the query is closed.
Item count of target bucket is 87,638,870. additional write is not exist. Memory Size of Index node is enough.
In which cases can I stop adding indexes?
@notemusic, what is the index storage mode in the cluster(Standard or Memory Optimized)?
You can check the indexer.log for any errors or collect the logs(UI->Logs) and share it here.
If its stuck for a long time, you may want to drop and recreate it to see if that helps.
@deepkaran.salooja,
i allocated 512M for Indexer. you can refer below log.
2017-12-01T09:37:05.968+09:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81000 Milliseconds Bucket catalog-creation-request Vbucket 983
2017-12-01T09:37:05.991+09:00 [Info] bucket_name/index_name/Mainstore Plasma: Warning: not enough memory to hold records in memory. Stats:{
“memory_quota”: 483183820,
“punch_hole_support”: true,
“count”: 0,
“compacts”: 0,
“purges”: 0,
“splits”: 0,
“merges”: 0,
“inserts”: 0,
“deletes”: 0,
“compact_conflicts”: 0,
“split_conflicts”: 0,
“merge_conflicts”: 0,
“insert_conflicts”: 0,
“delete_conflicts”: 0,
“swapin_conflicts”: 0,
“memory_size”: 34007573,
“memory_size_index”: 33324773,
“allocated”: 13273934344,
“freed”: 13239926771,
“reclaimed”: 13239906621,
“reclaim_pending”: 20150,
“allocated_index”: 33324773,
“freed_index”: 0,
“reclaimed_index”: 0,
“num_pages”: 253443,
“items_count”: 87638870,
“num_rec_allocs”: 122464062,
“num_rec_frees”: 122464062,
“num_rec_swapout”: 116340601,
“num_rec_swapin”: 0,
“bytes_incoming”: 0,
“bytes_written”: 45219010,
“write_amp”: 0.00,
“write_amp_avg”: 0.00,
“lss_fragmentation”: 30,
“lss_data_size”: 2404543741,
“lss_used_space”: 3451164214,
“lss_num_reads”: 186881,
“lss_read_bs”: 761282376,
“lss_gc_num_reads”: 0,
“lss_gc_reads_bs”: 0,
“cache_hits”: 0,
“cache_misses”: 0,
“cache_hit_ratio”: 0.00000,
“rcache_hits”: 0,
“rcache_misses”: 0,
“rcache_hit_ratio”: 0.00000,
“resident_ratio”: 0.00000,
“mvcc_purge_ratio”: 1.32750,
“currSn”: 720006,
“gcSn”: 720005,
“gcSnIntervals”: “[0 3 4 69 720006]”,
“purger_running”: false,
“mem_throttled”: true,
“lss_throttled”: false
}
2017-12-01T09:37:06.137+09:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81000 Milliseconds Bucket catalog-creation-request Vbucket 862
2017-12-01T09:37:06.170+09:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 147000 Milliseconds Bucket catalog-creation-request Vbucket 225
2017-12-01T09:37:06.183+09:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 246000 Milliseconds Bucket catalog-creation-request Vbucket 388
2017-12-01T09:37:06.220+09:00 [Warn] Indexer::MutationQueue Waiting for Node Alloc for 81000 Milliseconds Bucket catalog-creation-request Vbucket 795
@notemusic, 512M memory quota looks to be very small to index 87M items in the bucket. You can try to increase it to something like 2GB and see how it goes.