GSI Indexes growing indefinitely

Hello,

We’re trying out Couchbase Enterprise Edition 7.6.4 build 5146 but we have an issue that we cannot figure out. The indexes keeps growing until they occupy the whole disk.

The disk is ext4, main bucket contains around 300K documents.

[root@server-instance-v2 @2i]# du -cksh * | sort -rh
138G    total
2.9G    hrs_adv_companyId_meta_id_bookingResult_data_flightOffers0_chargeAmount_5957735829133141493_0.index
2.9G    hrs_adv_companyId_meta_id_8850357005463079428_0.index
2.9G    hrs_adv_combined_bookingRate_userData_exceedsPolicyLimit_companyId_meta_id_policy_id_count_10758791702350556098_0.index
2.9G    hrs_adv_bookingResult_data_flightOffers0_travelerPricings0_userData_1506729552_15380352469523833507_0.index
2.8G    hrs_idx_push_tokens_consolidated_4325257141410386655_0.index
2.8G    hrs_airBookingsConsolidatedIndex_6225744916456278318_0.index
2.6G    shards
2.5G    hrs_idx_hrs_type_11873868772694642082_0.index
2.5G    hrs_idx_emailCode_2_10040514299481959355_0.index
2.5G    hrs_idx_emailCode_1_11150257696180397076_0.index
2.5G    hrs_adv_referenceNumber_class_8653831852766080384_0.index
2.5G    hrs_adv_lower_bookingResult_data_associatedRecords_0_reference_lower238877905_2762152148865054283_0.index
2.4G    hrs_users_stripe_customer_id_12625997587250320491_0.index
2.4G    hrs_userDiscounts_16910555900617828125_0.index
2.4G    hrs_netMarginDiscount_15427557061242532879_0.index
2.4G    hrs_idx_userCode_1628518528997876783_0.index
2.4G    hrs_idx_userCode_1_16627839491205714634_0.index
2.4G    hrs_idx_teams_paymentMethodsIds_2_10578410297871822345_0.index
2.4G    hrs_idx_hrs_usersType_16309675261269761163_0.index
2.4G    hrs_idx_hrs_leadUsers_lower_email_10266024752475512595_0.index
2.4G    hrs_idx_hotel_reviews_1375712546913230184_0.index
2.4G    hrs_idx_companyBookings_dashboard_1765721268087681507_0.index
2.4G    hrs_idx_cheque_payouts_2883791984655483029_0.index
2.4G    hrs_adv_user_company_lower_email_class_7208430063741474232_0.index
2.4G    hrs_adv_status_requesterUserId_teamId_class_16166525188914122845_0.index
2.4G    hrs_adv_status_approverUserId_teamId_teamId_class_11724080686382897949_0.index
2.4G    hrs_adv_status_approverUserId_teamId_class_6603431194532156443_0.index
2.4G    hrs_adv_rewardReceived_checkoutDate_rewardAmount_bookingStatus_class_14572776169283098619_0.index
2.4G    hrs_adv_policyId_class_4450221895644977100_0.index
2.4G    hrs_adv_membership_companyMembership_companyId_membership_companyMem4172941942_408480753678787040_0.index
2.4G    hrs_adv_membership_companyMembership_companyId_lower_lastName_lower_firstName_lower_email_class_14497299893194371315_0.index
2.4G    hrs_adv_membership_companyMembership_companyId_class_membership_companyMembership_inactive_5024682432900706953_0.index
2.4G    hrs_adv_lower_email_class_745936991935331327_0.index
2.4G    hrs_adv_hrsReferenceNumber_class_7517329847179874753_0.index
2.4G    hrs_adv_DISTINCT_approvers_class_10743874032715545476_0.index
2.4G    hrs_adv_display_class_4153957644746654263_0.index
2.4G    hrs_adv_companyId_substr0_creationDate010_class_4243627340668778395_0.index
2.4G    hrs_adv_companyId_class_5648528200537993434_0.index
2.4G    hrs_adv_companyDbId_class_1584265190778598230_0.index
2.4G    hrs_adv_class_substr0_creationDate_010_1_6004944597352805415_0.index
2.4G    hrs_adv_class_companyId2_2171755314841477836_0.index
2.4G    hrs_adv_bookingStatus_userId_tripStartDateTime_class_17998866608919609012_0.index
2.4G    hrs_adv_bookingStatus_checkoutDate_userId_travelerId_class_7112224790140134332_0.index
2.4G    hrs_adv_bookingStatus_checkoutDate_class_10393906880420592856_0.index
2.2G    hrs_adv_inactive_substr0_creationDate010_lower_name_lower_domain_class_7411110214209367795_0.index
1.8G    hrs_#primary_7786860651755249870_0.index
1.7G    hrs_adv_bookingStatus_travelerId_userId_class_4719570424774471264_0.index
1.5G    hrs_adv_lower_companyId_chargeAmount_class_6644134054308316684_0.index
1.1G    hrs_static_content_idx_triphop_mappings_2_751117558843196495_0.index
1001M   hrs_static_content_idx_hrs_static_content_type_4308206713101905357_0.index
955M    hrs_static_content_idx_triphop_properties_2_14104791410250913923_0.index
933M    hrs_static_content_idx_triphop_properties_name_13823893235523116983_0.index
909M    hrs_static_content_hotel_ids_name_country_city_state_lower_13024777128863729841_0.index

If I drop an index and create it again, it drops down to KBytes, and over time it will grow indefinitely again.

If someone could assist it would be much appreciated.

Thank you,

Hi @lenix - did this work on a different version?

@mreiche I think it didnt happen on the older version we had before updating to 7.6.4

What other information is needed to diagnose this?

Much appreciated.