We are developing an api platform and using couchbase as our database. While developing generally everything was ok, inserts, selects, making N1QL indexes and new views were fast and efficient.
Then at one point we migrated the existing database of the product into couchbase. We got about 7.5 milion records all together. Then everrything seemed to have slowed down, constant server crashes on GSI index creation, and very slow responses from the database in general.
We thought the main problem was the quantity of documents.
So we deleted them, yet it seems the problem has persisted. Index creation takes very long and sometimes doesnt even finish. We have rebooted couchbase several times, and the whole server several times. The bucket now has just 150k records.
Do you have any suggestions as to why this would happen? Any way to make it “forget” that it had 7 million records and go back to the state it was before?
Our setup is an aws instance, 2proc 8gb ram, centos7
cb Version: 4.0.0-4051 Community Edition (build-4051)