"Only the default Scope is supported by Memcached Buckets" Error

.NET SDK
1

Hello,

I have created a Couchbase Docker container and I am connecting it with 127.0.0.1:8091. Most of the time there are no errors and everything works as expected. But sometimes I get this error when I call DefaultCollectionAsync on bucket object. What might be wrong here?

I am using Couchbase .NET Client Version: 3.1.7.
Couchbase Server Version in Docker is: 6.5.1.

Here is the stack trace of the error:

Error Message:
   System.NotSupportedException : Only the default Scope is supported by Memcached Buckets
  Stack Trace:
     at Couchbase.MemcachedBucket.get_Item(String scopeName)
   at Couchbase.MemcachedBucket.Scope(String scopeName)
   at Couchbase.Core.BucketBase.ScopeAsync(String scopeName)
   at Couchbase.Core.BucketBase.DefaultScopeAsync()
   at Couchbase.Core.BucketBase.CollectionAsync(String collectionName)

I am connecting to Couchbase via Dependency Injection:

services.AddCouchbase(client =>
    {
        client.WithConnectionString($"couchbase://127.0.0.1");
        client.EnableTls = false;
        client.UserName = "Username";
        client.Password = "Password";
        client.KvTimeout = TimeSpan.FromMilliseconds(90000);
        client.MaxHttpConnections = 20;
    })
    .AddCouchbaseBucket<ICustomBucketProvider>("MyBucket");

Here is my code that causes the exception to be thrown:

private async Task<IGetResult> Get(string documentId)
{
    var bucket = await GetBucket();
    var defaultCollection = await GetDefaultCollection(bucket);
    var result = await defaultCollection.GetAsync(documentId);
    return result;
}

private static async Task<ICouchbaseCollection> GetDefaultCollection(IBucket bucket)
{
    var defaultCollection = await bucket.DefaultCollectionAsync();
    return defaultCollection;
}

private async Task<IBucket> GetBucket()
{
    var bucket = await _purchaseOrderDraftBucketProvider.GetBucketAsync();
    return bucket;
}

Thanks.

2

@dogac_akyildiz -

What server version are you using?

Jeff

3

Hi @jmorris

I included in my post,

Couchbase Server Version in Docker is: 6.5.1.

Thanks.

4

Oops, missed that :slight_smile:

So, this combo (3.1.7/6.5.1) should work fine - can you enable logging and seeing what causes the NotSupportedException? I am guessing that error is just a side effect of some other error as you mentioned it was intermittent. It would be helpful to isolate the root cause here.

IIRC I ran into this last week and I believe it was that after provisioning the docker image - I ran my app and the cluster wasn’t yet ready. I created a ticket for tracking.

Jeff

5

No problem :slight_smile:

I enabled logging and was able to reproduce the error, here are the error logs:

[17:22:32 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[17:22:32 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[17:22:32 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[17:22:32 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[17:22:35 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[17:22:35 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[17:23:02 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[17:23:02 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[17:23:32 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[17:23:32 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.

It looks like Couchbase Client did not complete the initialization because I see no info logs. Normally, there are a lot of information logs when Couchbase Client is starting up.

Thanks.

6

@dogac_akyildiz -

Sounds like a configuration issue with docker/couchbase. Check to see if the correct ports are exposed to the consuming application. SDK doctor may help in isolating the problem as well.

Jeff

7

But this happens only once in while and I double checked the configuration and it is correct. This error happens approximately 1 in 100 times.

Here is my Docker configuration:

public class CouchbaseContainer
    {
        private readonly CouchbaseTestcontainer _couchbaseTestContainer;

        public CouchbaseContainer()
        {
            var dockerHost = Environment.GetEnvironmentVariable("DOCKER_HOST");
            if (string.IsNullOrEmpty(dockerHost))
            {
                dockerHost = "unix:/var/run/docker.sock";
            }

            _couchbaseTestContainer = new TestcontainersBuilder<CouchbaseTestcontainer>()
                .WithDockerEndpoint(dockerHost)
                .WithDatabase(new CouchbaseTestcontainerConfiguration
                {
                    Username = "Username",
                    Password = "Password",
                    BucketName = "BucketName"
                })
                .WithImage("mustafaonuraydin/couchbase-testcontainer:6.5.1")
                .WithName("couchbase-testcontainer")
                .WithExposedPort(8091)
                .WithExposedPort(8093)
                .WithExposedPort(11210)
                .WithPortBinding(8091, 8091)
                .WithPortBinding(8093, 8093)
                .WithPortBinding(11210, 11210)
                .Build();
        }

        public async Task StartAsync()
        {
            await _couchbaseTestContainer.StartAsync();
        }

        public async Task StopAndDisposeAsync()
        {
            await _couchbaseTestContainer.StopAsync();
            await _couchbaseTestContainer.DisposeAsync();
        }
    }

Should I expose and bind 8092, 8094, 8095, 8096, 11211 and try again?

8

@jmorris

I exported other ports (8092, 8095, 8096, 12111) too and the occasional error persists in our tests.

Dogac

9

@dogac_akyildiz -

Your SDK error logs should be more complete and should show what specifically is happening. Make sure your in DEBUG mode and you see the SDK trying to bootstrap - this should help isolate the issue.

For example:

2021-07-05T11:25:18.0237391-07:00  [INF] There was an error attempting to resolve hosts using DNS-SRV - "Non-Existent Domain" (9daf8ee2)
2021-07-05T11:25:18.0265966-07:00  [DBG] Bootstrapping with node "localhost" (98ca0e33)
2021-07-05T11:25:18.1682692-07:00  [DBG] Setting TCP Keep-Alives using SocketOptions - enable keep-alives True, time 00:01:00, interval 00:00:01. (d66a37aa)
2021-07-05T11:25:18.1686863-07:00  [DBG] Setting TCP Keep-Alives using SocketOptions - enable keep-alives True, time 00:01:00, interval 00:00:01. (d66a37aa)
2021-07-05T11:25:18.2081782-07:00  [DBG] Starting connection initialization on server "[::1]:11210". (5c5836f0)
2021-07-05T11:25:18.2081782-07:00  [DBG] Starting connection initialization on server "[::1]:11210". (5c5836f0)
2021-07-05T11:25:18.5049581-07:00  [DBG] Executing op Helo on "[::1]:11210" with key "{\"i\":\"73f10a7b60147cea/0000000000000001\",\"a\":\"couchbase-net-sdk/3.0.0.0 (clr/.NET 5.0.1-servicing.20575.16) (os/Microsoft Windows 10.0.19042)\"}" and opaque 4. (ebdc59ba)

...

Jeff

10

Hi @jmorris

I set minimum level to debug and reproduced the issue. Here is the logs:

[00:13:14 DBG] Waiting for 00:00:02.5000000 before polling.
[00:13:14 INF] There was an error attempting to resolve hosts using DNS-SRV - Non-Existent Domain
[00:13:14 DBG] Bootstrapping with node 127.0.0.1
[00:13:14 DBG] Setting TCP Keep-Alives using SocketOptions - enable keep-alives True, time 00:01:00, interval 00:00:01.
[00:13:14 DBG] Setting TCP Keep-Alives using SocketOptions - enable keep-alives True, time 00:01:00, interval 00:00:01.
[00:13:14 DBG] Starting connection initialization on server 127.0.0.1:11210.
[00:13:14 DBG] Starting connection initialization on server 127.0.0.1:11210.
[00:13:14 DBG] Executing op Helo on 127.0.0.1:11210 with key {"i":"c6f2bd4c8e33b939/0000000000000001","a":"couchbase-net-sdk/3.1.7.0 (clr/.NET 5.0.2) (os/Linux 4.4.0-186-generic #216-Ubuntu SMP Wed Jul 1 05:34:05 UTC 2020)"} and opaque 3.
[00:13:14 DBG] Executing op Helo on 127.0.0.1:11210 with key {"i":"c6f2bd4c8e33b939/0000000000000002","a":"couchbase-net-sdk/3.1.7.0 (clr/.NET 5.0.2) (os/Linux 4.4.0-186-generic #216-Ubuntu SMP Wed Jul 1 05:34:05 UTC 2020)"} and opaque 4.
[00:13:14 DBG] Completed executing op 127.0.0.1:11210 on Helo with key {"i":"c6f2bd4c8e33b939/0000000000000001","a":"couchbase-net-sdk/3.1.7.0 (clr/.NET 5.0.2) (os/Linux 4.4.0-186-generic #216-Ubuntu SMP Wed Jul 1 05:34:05 UTC 2020)"} and opaque 3
[00:13:14 DBG] Completed executing op 127.0.0.1:11210 on Helo with key {"i":"c6f2bd4c8e33b939/0000000000000002","a":"couchbase-net-sdk/3.1.7.0 (clr/.NET 5.0.2) (os/Linux 4.4.0-186-generic #216-Ubuntu SMP Wed Jul 1 05:34:05 UTC 2020)"} and opaque 4
[00:13:14 DBG] Executing op GetErrorMap on 127.0.0.1:11210 with key  and opaque 6.
[00:13:14 DBG] Executing op GetErrorMap on 127.0.0.1:11210 with key  and opaque 8.
[00:13:14 DBG] Completed executing op 127.0.0.1:11210 on GetErrorMap with key  and opaque 8
[00:13:14 DBG] Completed executing op 127.0.0.1:11210 on GetErrorMap with key  and opaque 6
[00:13:14 INF] v=IzisxXr8CvxGOUOef/8hE0x+X/o=
[00:13:14 DBG] Connection for 127.0.0.1:11210 has been started.
[00:13:14 INF] v=uDdQVz0PolXt+4ehaxeNSb4rcvE=
[00:13:14 DBG] Connection for 127.0.0.1:11210 has been started.
[00:13:14 DBG] Starting connection pool monitor on 127.0.0.1:11210, idle timeout 00:01:00, back pressure threshold 8
[00:13:14 DBG] Connection pool for 127.0.0.1:11210 initialized with 2 connections.
[00:13:14 DBG] Executing op GetClusterConfig on 127.0.0.1:11210 with key  and opaque 14.
[00:13:14 DBG] Completed executing op 127.0.0.1:11210 on GetClusterConfig with key  and opaque 14
[00:13:14 INF] NetworkResolution [default] using default 127.0.0.1
[00:13:14 DBG] Adding node 127.0.0.1:11210 to [].
[00:13:14 DBG] Added node 127.0.0.1:11210 to ["127.0.0.1:11210-af7f1cf8-ebb6-4282-a7d2-0de75e583145"]
[00:13:14 INF] Enabling Enhanced Prepared Statements
[00:13:14 DBG] Executing op SelectBucket on 127.0.0.1:11210 with key PurchaseOrderItemDrafts and opaque 15.
[00:13:14 DBG] Executing op SelectBucket on 127.0.0.1:11210 with key PurchaseOrderItemDrafts and opaque 16.
[00:13:14 DBG] Server 127.0.0.1:11210 returned KeyNotFound for op SelectBucket with key PurchaseOrderItemDrafts and opaque 16.
[00:13:14 DBG] Server 127.0.0.1:11210 returned KeyNotFound for op SelectBucket with key PurchaseOrderItemDrafts and opaque 15.
[00:13:14 DBG] Op failed: Couchbase.Core.IO.Operations.Authentication.SelectBucket
Couchbase.Core.Exceptions.KeyValue.DocumentNotFoundException: Exception of type 'Couchbase.Core.Exceptions.KeyValue.DocumentNotFoundException' was thrown.
   at Couchbase.Core.ClusterNode.ExecuteOp(Func`4 sender, IOperation op, Object state, CancellationTokenPair tokenPair)
-----------------------Context Info---------------------------
{"DispatchedFrom":null,"DispatchedTo":null,"DocumentKey":"PurchaseOrderItemDrafts","ClientContextId":"16","Cas":0,"Status":1,"BucketName":null,"CollectionName":null,"ScopeName":null,"Message":"KV Error: {Name=\"KEY_ENOENT\", Description=\"Not Found\", Attributes=\"item-only\"}","OpCode":"SelectBucket"}
[00:13:14 DBG] Op failed: Couchbase.Core.IO.Operations.Authentication.SelectBucket
Couchbase.Core.Exceptions.KeyValue.DocumentNotFoundException: Exception of type 'Couchbase.Core.Exceptions.KeyValue.DocumentNotFoundException' was thrown.
   at Couchbase.Core.ClusterNode.ExecuteOp(Func`4 sender, IOperation op, Object state, CancellationTokenPair tokenPair)
-----------------------Context Info---------------------------
{"DispatchedFrom":null,"DispatchedTo":null,"DocumentKey":"PurchaseOrderItemDrafts","ClientContextId":"15","Cas":0,"Status":1,"BucketName":null,"CollectionName":null,"ScopeName":null,"Message":"KV Error: {Name=\"KEY_ENOENT\", Description=\"Not Found\", Attributes=\"item-only\"}","OpCode":"SelectBucket"}
[00:13:14 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[00:13:14 ERR] The Bucket [PurchaseOrderItemDrafts] could not be selected. Either it does not exist, is unavailable or the node itself does not have the Data service enabled.
[00:14:44 DBG] Connection for 127.0.0.1:11210 could not be started.
[00:14:44 DBG] ExistingEndpoints: ["127.0.0.1:11210"], revision 24.
[00:14:44 DBG] RemovedEndpoints: [], revision 24
[00:14:44 DBG] Fetching scope _default
Getting logs for current test failure.
System.NotSupportedException: Only the default Scope is supported by Memcached Buckets
   at Couchbase.MemcachedBucket.get_Item(String scopeName)
   at Couchbase.MemcachedBucket.Scope(String scopeName)
   at Couchbase.Core.BucketBase.ScopeAsync(String scopeName)
   at Couchbase.Core.BucketBase.DefaultScopeAsync()
   at Couchbase.Core.BucketBase.CollectionAsync(String collectionName)

Thank you,
Dogac

11

Can you confirm that you are bootstrapping to a node with the data service installed?

12

Of course, here’s my Dockerfile for creating a Couchbase Container:

FROM couchbase:6.5.1

ENV SERVICES "data,index,query,fts,analytics,eventing"

ENV CLUSTER_RAMSIZE 1024
ENV CLUSTER_INDEX_RAMSIZE 512
ENV CLUSTER_EVENTING_RAMSIZE 256
ENV CLUSTER_FTS_RAMSIZE 256
ENV CLUSTER_ANALYTICS_RAMSIZE 1024

ENV INDEX_STORAGE_SETTING "memopt"

ENV BUCKET_NAME "Sample"
ENV BUCKET_TYPE "couchbase"
ENV BUCKET_RAMSIZE 128

ENV USERNAME "Administrator"
ENV PASSWORD "password"

ENV REST_PORT 8091
ENV CAPI_PORT 8092
ENV QUERY_PORT 8093
ENV FTS_PORT 8094
ENV MEMCACHED_SSL_PORT 11207
ENV MEMCACHED_PORT 11210
ENV SSL_REST_PORT 18091

COPY entrypoint.sh /config-entrypoint.sh

ENTRYPOINT ["/config-entrypoint.sh"]

I believe it has a single node since it is running locally.

13

@dogac_akyildiz

In my experience, your problem is most likely timing-related differences in your environments. When you start Couchbase its startup process is asynchronous. If you try to bootstrap the SDK in another container before Couchbase is done with startup, creating buckets, etc, you’ll get errors like this.

What I’ve typically done is use shared files between the containers so that when my configuration script (like your “config-entrypoint.sh”) is complete it writes a file, then have the startup on my application wait for that file before bootstrapping.

You can see an example here:

https://github.com/brantburnett/couchbasefakeit/blob/main/scripts/configure-node.sh#L30

This is a shared Docker image I maintain that does similar things to the one you’re building, configuring Couchbase as it starts up.

14

Hi @btburnett3

Thanks for your reply, that is a good point. We already check for the final log in our test container, in our case it is specifically “couchbase-dev started” log.

The repository of the Dockerfile we are using is this: GitHub - mustafaonuraydin/couchbase-docker-image-for-testcontainers: This project is provides pre initialized Couchbase over docker with environment variables for testcontainers.

And here’s the “couchbase-dev started” log: couchbase-docker-image-for-testcontainers/entrypoint.sh at master · mustafaonuraydin/couchbase-docker-image-for-testcontainers · GitHub

And this is repository we are using for .NET TestContainers library:

And the configuration that sets the wait strategy for the final log is here: testcontainers-dotnet/src/DotNet.Testcontainers/Containers/Configurations/Databases/CouchbaseTestcontainerConfiguration.cs at aa3d573129045a26de08e1fe57ccc180b1e04108 · testcontainers/testcontainers-dotnet · GitHub

Dogac

15

Hello again,

It turns out that I was using CouchbaseNetClient, Version 3.1.7, and Couchbase.Extensions.DependencyInjection, Version 3.1.6.
I downgraded CouchbaseNetClient to Version 3.1.6 and this issue is resolved.

But there are another subtle errors that I encounter occasionally with this combination (both versions 3.1.6), but this is a subject of a different post.

Thanks,
Dogac

1 Like
16

@dogac_akyildiz -

Have you updated to 3.2.0 yet? It would be nice to see if this has been resolved as the work-around involved down-grading to 2.1.6 and there were a number of collections/scopes bugs fixed in 3.2.0.

Jeff

17

Hello, I am also having the same problem. I am using version 3.2.2
I’m getting a Memcached error even though I’m using a Couchbase bucket.

Couchbase .NET Client Version: 3.2.2
Couchbase Server Version 6.6.2 build 9600

This is my stack trace of error;

Only the default Scope is supported by Memcached Buckets

Couchbase.MemcachedBucket.get_Item(string scopeName)
   at async ValueTask<ICouchbaseCollection> Couchbase.Core.BucketBase.CollectionAsync(string collectionName)
   at async ValueTask<ICouchbaseCollection>

Thanks.

18

@aytuncc -

Have you checked the SDK logs to see what is happening while bootstrapping? I suspect its failing and the last bucket type it tries to bootstrap against is a Memcached bucket as the client doesn’t actually know the bucket type until it attempts to connect to it.

Jeff

19

@jmorris @btburnett3

I’m getting the same error and trying to figure out what might be reason for the exception…

I see that you expect from MemcachedBucket instance to have “_default” scope added in the “Scope” dictionary but don’t see any part of code that’s supposed to add “_default” scope to the dictionary. Also, LoadDefaultScope method(BucketBase.cs) is never called at all.

Thank you!

20

@antotoki

It’s somewhat misleading, but the most common cause of this error is simply that the client is unable to connect to the server, or at least was briefly unable.

The bootstrap protocol requires that the client connects to a bucket first as a full-featured Couchbase bucket. If this fails for any reason, it does a fallback and attempts to connect as an old-style Memcached bucket instead. This is for backward compatibility.

However, I’ve seen cases where the first connection hiccups but then the second connection succeeds, despite the fact it’s really a full-featured Couchbase bucket. At this point, if you try to acquire a scope/collection it will fail with the error message noted because the SDK thinks it’s a Memcached bucket.

I’d recommend reviewing logs to see why the first bootstrap attempt failed. Feel free to post the logs here for more feedback. You might also try using the SDK Doctor to verify your network connectivity.

1 Like