How to do .Get<T>(List keys) in SDK 3.x in C#?

.NET SDK
1

Hi,
In 2.x you can get multiple documents with a list of keys, but how do you do this in the C# SDK 3.x?

I am only seeing one K/V method for get?

`Task<IGetResult> GetAsync(string id, GetOptions? options = null);`

So question is how do we do the same operation with SDK 3.x ?

Thanks,

Brian Davis

2

@Brian_Davis that method actually had some issues with regard to how efficiently it parallelized the operations in 2.x, and is not part of the 3.x design. Part of the problem is that the most efficient way to parallelize is going to vary widely by use case. Things like expected document size, number of data nodes, network latency, and configured connection pool size can all affect how many operations can be efficiently run in parallel.

My recommendation is to parallelize the operations yourself. For a small set of documents (say, a couple dozen), you don’t need to worry about anything fancy.

var keys = new[] {"document_1", "document_2", "document_3"};
var tasks = keys.Select(p => bucket.GetAsync(p)).ToList();

await Task.WhenAll(tasks); // Note: This line will throw an exception if ANY of the gets fail

var results = tasks.Select(p => p.Result).ToList();

If you’re trying to get a large number of documents, you may want to implement a limiter to keep the number of in-flight operations under control. For very large numbers of documents, I recommend grouping the requests into parallel batches for the best efficiency.

If you have more specifics about your use case, I’m happy to provide more details.

2 Likes
3

Hi Brant,
How can I use “Id” from the results?

image
image1744Ă—599 30.4 KB

When Results are return it is showing the Id through?

image
image716Ă—199 16.1 KB

But at the same time the property does not exist?

image
image1307Ă—581 29.3 KB

But, when convert data to JSON and then that JSON to an object it works.

image
image1283Ă—466 25.1 KB

There must be something I am missing?

There must be a better way to be able to use the Id of a Get Result?

The other properties seems to work ok, just issue with Id.

Thanks,

Brian Davis

4

@Brian_Davis

The issue is that the internal concrete implementation of GetResult has the Id property, but the IGetResult interface exposed to you does not. I see three possibilities.

  1. The SDK could be changed to expose Id on IResult or IGetResult in a future release
  2. You could use reflection to get to the Id property (theoretically unsafe, as the Id property, being internal, is subject to change in future SDK releases)
  3. Alter the logic a bit to keep the Id pair with the task in something like a ValueTuple
var keys = new[] {"document_1", "document_2", "document_3"};
List<(string Id, Task<IGetResult> Task)> tasks = keys.Select(p => (p, bucket.GetAsync(p))).ToList();

await Task.WhenAll(tasks.Select(p => p.Task)); // Note: This line will throw an exception if ANY of the gets fail

List<(string Id, IGetResult Result)> results = tasks.Select(p => (p, p.Result)).ToList();
5

As this was something discussed when the RFC was written and it is by design, it might make sense to make an extensions library for handling use-cases like this that are outside of the RFC.

-Jeff

6

@btburnett3

With example code you gave getting error that .Result is not a property.

image
image1642Ă—335 20 KB

Result not available in p.

image
image863Ă—346 12.7 KB

7

I got it working with this:

Had to put a check for strings and use custom transcoder for that was done with the “Get” Method was working on before.

using (var scope = Datadog.Trace.Tracer.Instance.StartActive(method, null, EnvName))
                {

                    List<(string Id, Task<IGetResult> Task)> tasks;

                    var typeCode = Type.GetTypeCode(typeof(T));

                    // Work around for now for "strings" it is a bug in SDK 3.x which is going to be fixed in next release.
                    if (typeCode == TypeCode.Char || typeCode == TypeCode.String)
                    {
                        var options = new GetOptions();

                        options.Transcoder(_LegacyTranscoder);

                        // SDK 3.X Wrapper
                        tasks = keys.Select(p => (p, Get(p,options))).ToList();
                    }
                    else
                    {
                        // SDK 3.X Wrapper
                        tasks = keys.Select(p => (p, Get(p))).ToList();
                    }

                    await System.Threading.Tasks.Task.WhenAll(tasks.Select(p => p.Item2)); // Note: This line will throw an exception if ANY of the gets fail

                    List<(string Id, IGetResult result)> results = tasks.Select(p => (p.Id, p.Item2.Result)).ToList();

                    return results;
                }