Problem when adding new server to cluster

Couchbase Server
1

I’ve created a new EC2 server on AWS and I tried to add it to my cluster but when I hit rebalance it fails, this is what I see. in the logs:
Rebalance exited with reason {badmatch,
{leader_activities_error,
{default,{ns_janitor,“api”,apply_config}},
{no_quorum,
[{required_quorum,
{all,
{set,5,16,16,8,80,48,
{,,,,,,,,,,,,,
,,},
{{[‘ns_1@10.10.200.100’,
‘ns_1@10.10.100.50’],
,,
[‘ns_1@10.10.100.93’,
‘ns_1@10.10.100.133’],
,,
[‘ns_1@10.10.200.166’],
,,,,,,,,}}}}},
{leases,
[‘ns_1@10.10.100.50’,‘ns_1@10.10.200.100’,
‘ns_1@10.10.100.93’,
‘ns_1@10.10.100.133’]}]}}}

in addition there are weird errors in the logs when I connect via ssh, I see a lot:
Projector.log - [Error] PROJ[:9999] ##2 acquireFeed(): projector.topicMissing

indexer.log:
2020-09-21T16:01:44.872+00:00 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 10.100.1.145:55610. Error = EOF. Kill Pipe.
2020-09-21T16:01:44.872+00:00 [Error] LeaderSyncProxy.updateAcceptEpochAfterQuorum(): Error encountered = Server Error : SyncProxy.listen(): channel closed. Terminate
2020-09-21T16:01:44.872+00:00 [Error] LeaderServer:startProxy(): Leader Fail to synchronization with follower (TCP conn = 10.100.1.145:55610)
2020-09-21T16:01:44.872+00:00 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 10.100.1.145:55611. Error = EOF. Kill Pipe.
2020-09-21T16:01:44.872+00:00 [Error] LeaderSyncProxy.updateAcceptEpochAfterQuorum(): Error encountered = Server Error : SyncProxy.listen(): channel closed. Terminate
2020-09-21T16:01:44.872+00:00 [Error] LeaderServer:startProxy(): Leader Fail to synchronization with follower (TCP conn = 10.100.1.145:55611)

error.log:
[ns_server:error,2020-09-21T15:09:07.177Z,ns_1@10.10.200.149:service_status_keeper_worker<0.450.0>:rest_utils:get_json_local:63]Request to (indexer) getIndexStatus failed: {error,
{econnrefused,
[{lhttpc_client,send_request,1,
[{file,
“/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/lhttpc/lhttpc_client.erl”},
{line,220}]},
{lhttpc_client,execute,9,
[{file,
“/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/lhttpc/lhttpc_client.erl”},
{line,169}]},
{lhttpc_client,request,9,
[{file,
“/home/couchbase/jenkins/workspace/couchbase-server-unix/couchdb/src/lhttpc/lhttpc_client.erl”},
{line,92}]}]}}

I’ve created and added new servers before, not sure what happened al of the sudden I’m using the same method as before, any help?

thank you very much

2

Problem still happening, please has someone ever seen these errors?

3

At the end we needed to create another cluster set it as xdcr of the original cluster and when all the data returned we switched all the applications to to work with the xdcr cluster…