I am in the process of migrating my code from the 2.8 to the 3.1 .net SDK. I have an issue I cannot figure out.
I have a document whose meta().id is User~8169d289-4f47-4973-ac2f-cd6e62698eb9
If I query the cluster for that document it is found (using " SELECT meta().id, * FROM Core WHERE meta().id = ‘User~8169d289-4f47-4973-ac2f-cd6e62698eb9’").
However, if I call ExistsAsync (on the bucket’s default collection), false is returned. And if I call GetAsync on that same collection it throws a DocumentNotFoundException.
So why can I not access this document using the SDK even though it definitely exists?
If it helps, the document was created using a 2.x version of the SDK (or even possibly via Couchbase.Lite).
Would you mind posting the code that returns false for ExistsAsync and the GetAsync that throws the exception (including the code to connect, get the bucket, get the collection)? (It shouldn’t matter if it was created through 2.x or through Sync Gateway+Couchbase Lite, so long as the key is accurate).
var settings = new JsonSerializerSettings {
ContractResolver = new DefaultContractResolver(),
PreserveReferencesHandling = PreserveReferencesHandling.Objects
};
var options = new ClusterOptions {
ConnectionString = $ "couchbase://{ServerUri.Host}",
Compression = true,
UserName = "user",
Password = "pwd",
Serializer = new DefaultSerializer(settings, settings)
};
var theCluster = await Cluster.ConnectAsync(serverOptions).ConfigureAwait(false);
theCluster.ConfigureAwait(false);
await theCluster.WaitUntilReadyAsync(TimeSpan.FromSeconds(1)).ConfigureAwait(false);
var bucket = await Manager.TheCluster.BucketAsync(BucketName).ConfigureAwait(false);
bucket.ConfigureAwait(false);
var collection = Bucket.DefaultCollection();
var result = await Database.Current.TheCluster.QueryAsync < dynamic > ("SELECT meta().id, * FROM Core WHERE meta().id = 'User~8169d289-4f47-4973-ac2f-cd6e62698eb9'");
var rows = await result.Rows.ToArrayAsync(); // has one row with all of the right information
var userExists = await Database.Current.CoreCB.Collection.ExistsAsync("User~8169d289-4f47-4973-ac2f-cd6e62698eb9"); // userExists.Exists==false
var userResult = await Database.Current.CoreCB.Collection.GetAsync(user.Id).ConfigureAwait(false); // throws not found exception
Thanks, @blimkemann, I can’t quite try that code as-is, mainly because I’m not sure what’s behind Database, Database.Current, etc. Also, there’s a Bucket (property, I’m assuming), but also a bucket local variable (which is ignored?). There are some other issues too, like your code defining options but using serverOptions instead in ConnectAsync. Finally, there’s a reference to user.Id, but I don’t see a user object anywhere in your code.
So, below is my attempt at a reproduction. I also created a single document with an ID of User~8169d289-4f47-4973-ac2f-cd6e62698eb9 in a bucket called ‘Core’ and a single primary index on that bucket. In my example, ExistsAsync and GetAsync return true and the correct result, respectively. I suspect, therefore, that there is some mix-up in which bucket is being used for which operation, and/or whatever is in the user.Id field in your code. Take a look at my code and let me know where you think we might differ:
var settings = new JsonSerializerSettings
{
ContractResolver = new DefaultContractResolver(),
PreserveReferencesHandling = PreserveReferencesHandling.Objects
};
var BucketName = "Core";
var serverOptions = new ClusterOptions
{
ConnectionString = $"couchbase://localhost",
Compression = true,
UserName = "Administrator",
Password = "password",
Serializer = new DefaultSerializer(settings, settings)
};
var theCluster = await Cluster.ConnectAsync(serverOptions).ConfigureAwait(false);
theCluster.ConfigureAwait(false);
await theCluster.WaitUntilReadyAsync(TimeSpan.FromSeconds(1)).ConfigureAwait(false);
var bucket = await theCluster.BucketAsync(BucketName).ConfigureAwait(false);
bucket.ConfigureAwait(false);
var collection = bucket.DefaultCollection();
var result = await theCluster.QueryAsync<dynamic>("SELECT meta().id, * FROM Core WHERE meta().id = 'User~8169d289-4f47-4973-ac2f-cd6e62698eb9'");
var rows = await result.Rows.ToArrayAsync(); // has one row with all of the right information
var userId = "User~8169d289-4f47-4973-ac2f-cd6e62698eb9";
var userExists = await collection.ExistsAsync(userId); // userExists.Exists==false
var userResult = await collection.GetAsync(userId).ConfigureAwait(false); // throws not found exception
var user = userResult.ContentAs<dynamic>();
await theCluster.DisposeAsync();
@matthew.groves : Sorry for the code sample. I was trying to distill my code into something simple and missed a few things.
When I put your code into NUnit and change a few strings to get it to connect to my environment it works fine.
I have verified that my code is indeed grabbing the right collection (named _default and the private variable _bucket matches the name of the bucket I’m using) calls to the collection still think that document does not exist.
Is a collection thread-specific? In other words, do I need to get the collection each and every time I use it? I have wrapped my bucket opening logic into an async method and store a reference to my wrapper class in a static variable so that it is always available. This has worked perfectly in the past, but do I need to re-think my design?
Since a query to the cluster works and the collection appears to be correct I’m out of other ideas…
@matthew.groves : I enabled Debug logging and found something very odd. I failed to mention before (because I thought it irrelevant) that I am opening several buckets using a Task.WhenAll that calls my OpenForServerAsync on each bucket (see below). When I debug, the collection is referencing the right bucket (Core) but the debug log says Couchbase is actually using the bucket named Working instead. (see log output). If the debugger says the collection belongs to Core but Working is being used instead, something must be getting stepped on.
public virtual async Task OpenForServerAsync()
{
Bucket = await Manager.TheCluster.BucketAsync(BucketName).ConfigureAwait(false);
Bucket.ConfigureAwait(false);
Collection = Bucket.DefaultCollection();
}
and is called like this:
var tasks = new List<Task>();
tasks.Add(db1.OpenForServerAsync());
tasks.Add(db2.OpenForServerAsync());
await Task.WhenAll(tasks);
Log output (from the QueryAsync to ExistsAsync):
dbug: Couchbase.Query.QueryClient[0]
Sending query 619a2962-11a9-480e-9e6c-64357679ac17 to node http://domain.com:8093/query.
dbug: Couchbase.Query.QueryClient[0]
Request 619a2962-11a9-480e-9e6c-64357679ac17 has succeeded.
dbug: Couchbase.Core.ClusterNode[0]
CB: Current state is Closed.
dbug: Couchbase.Core.ClusterNode[0]
Executing op GetMeta on 104.7.7.51:11210 with key User~8169d289-4f47-4973-ac2f-cd6e62698eb9 and opaque 41.
dbug: Couchbase.Core.ClusterNode[0]
Server 104.7.7.51:11210 returned KeyNotFound for op GetMeta with key User~8169d289-4f47-4973-ac2f-cd6e62698eb9 and opaque 41.
dbug: Couchbase.Core.ClusterNode[0]
Op failed: Couchbase.Core.IO.Operations.GetMeta
Couchbase.Core.Exceptions.KeyValue.DocumentNotFoundException: Exception of type ‘Couchbase.Core.Exceptions.KeyValue.DocumentNotFoundException’ was thrown.
at Couchbase.Core.ClusterNode.ExecuteOp(Func`4 sender, IOperation op, Object state, CancellationToken token)
-----------------------Context Info---------------------------
{“DispatchedFrom”:null,“DispatchedTo”:null,“DocumentKey”:“User~8169d289-4f47-4973-ac2f-cd6e62698eb9”,“ClientContextId”:“41”,“Cas”:0,“Status”:1,“BucketName”:“Working”,“CollectionName”:“_default”,“ScopeName”:null,“Message”:“KV Error: {Name="KEY_ENOENT", Description="Not Found", Attributes="item-only"}”}
UPDATE: the Task.WhenAll is the culprit. When I change my code to call each OpenForServerAsync in a loop it now works perfectly. So something in your SDK is not playing nicely with a WhenAll
public class DatabaseBase{
public DatabaseBase(DatabaseManager manager){
Manager = manager;
}
public DatabaseManager Manager{get;}
public IBucket bucket{ get; protected set;}
public ICouchbaseCollection TheCollection{get;protected set;
public string BucketName{get;set;}
public async Task OpenForServerAsync(){
Bucket = await Manager.TheCluster.BucketAsync(BucketName).ConfigureAwait(false);
Bucket.ConfigureAwait(false);
Collection = Bucket.DefaultCollection();
}
}
public class DatabaseManager{
public ICluster TheCluster { get; protected set; }
public List<DatabaseBase> Databases { get; } = new List<DatabaseBase>();
public Uri ServerUri { get; }
public async Task OpenAsync(ClusterOptions serverOptions = null){
TheCluster = await Cluster.ConnectAsync(serverOptions).ConfigureAwait(false);
TheCluster.ConfigureAwait(false);
await TheCluster.WaitUntilReadyAsync(TimeSpan.FromSeconds(1)).ConfigureAwait(false);
// GETS WRONG COLLECTION IN database (looks like they all point to the last database)
//var openDbTasks = new List<Task>();
//foreach (var database in Databases)
//openDbTasks.Add(database.OpenForServerAsync());
//await Task.WhenAll(openDbTasks).ConfigureAwait(false); // SDK gets stepped on by doing this
// WORKS!
foreach (var database in Databases)
await database.OpenForServerAsync().ConfigureAwait(false);
}
}
This feels like a race condition bug of some kind, probably something which needs a mutex. Thanks for your effort in tracking down the details. I’ve filed a bug for tracking.
I believe that issue was more specifically related to the DI system’s ClusterProvider bootstrapping multiple clusters at the same time. Though there may be some overlap.