Custom serialization and LINQ

.NET SDK
1

Hi,

I am using a custom serializer and was surprised that the N1Ql queries did not use it. Looking through some of the code, I see a separate DataMapper option for QueryRequest. That option seems to however not been exposed for LINQ. Is there a way to override serialization across the bucket and the queries (both LINQ and N1QL) in one location?

2

@RentierM

You should be able to use a custom serializer with LINQ. Just make sure you implement IExtendedTypeSerializer, which has some extra features needed by LINQ. More details here: Linq2Couchbase/docs/custom-serializers.md at master · couchbaselabs/Linq2Couchbase · GitHub

Then just put the custom serializer into you client configuration when creating the cluster, and LINQ should respect it.

If you want to use Change Tracking, there are some additional requirements which are document at the end of this file: https://github.com/couchbaselabs/Linq2Couchbase/blob/master/docs/change-tracking.mdhttps://github.com/couchbaselabs/Linq2Couchbase/blob/master/docs/change-tracking.md

Sorry about the formatting, we need to go through and update for GitHub’s new markdown renderer.

If you have any further questions, please let me know.

Thanks,
Brant

3

Hi,

When calling

var serializer = context.Configuration.Serializer();

I see the correct serializer being created. But when I call

var result = context.Query().FirstOrDefault(w => w.Name == entity.Name);

The serializer is not used. I am setting the function to create the serializer on the configuration at start up. When I use a QueryRequest and set the DataMapper, it works.

4

@RentierM

Can you provide the configuration code you’re using during startup to set the serializer? Also, what version of the .Net SDK are you currently using?

Setting the DataMapper on the QueryRequest should not be required to use the custom serializer. That’s an extension point that we added to allow you to use custom serializer per-request (to support some special LINQ functions behind the scenes). But if left null it should use the configured serializer.

5

Hi,

using CouchbaseNetclient 2.4.6, Linq2Couchbase 1.3.2

 var config = new ClientConfiguration
{
            Servers = clusterServers,
            UseSsl =  settings.UseSsl,
            BucketConfigs = [//removed create dictionary with default bucket] 
        };
config.Serializer = () =>new CustomSerializer();
ClusterHelper.Initialize(config);
6

@RentierM

I suspect I see the issue. Try adding UseStreaming(false) after calling Query(), and see if that affects it.

Linq2Couchbase 1.3 turned on the new streaming query result function by default, which is a more memory-efficient way of processing large query results that was added recently to the main SDK. However, the streaming function in the main SDK doesn’t support custom serializers, it’s built around Newtonsoft’s JSON reader.

If this is the issue, we’ll probably need to add a more user-friendly way to disable streaming in LINQ globally for custom serializers.

7

@RentierM

Were you able to confirm if UseStreaming(false) fixed your issue?

8

Hi, I am facing the same issue as decribed in opening comment. Adding UseStreaming(false) on Query() doesn’t help either…

9

What version of the SDK and Linq2Couchbase are you using? Can you post an example?

10

“CouchbaseNetClient” Version=“2.4.7”

Example:

  1. Define custom serializer:

    public class CustomSerializer : DefaultSerializer
  {
    private static readonly JsonSerializerSettings jsonSettings = new JsonSerializerSettings
    {
      TypeNameHandling = TypeNameHandling.All 
    };

    public CustomSerializer() 
      : base(jsonSettings, jsonSettings) { }
  }

  2. SDK initialization with custom serializer

        this.clusterConfiguration.Serializer = () => new CustomSerializer();
    this.cluster = new Cluster(this.clusterConfiguration);

  3. Try to query it. Query results cotains base resources only ($type attribute not respected)

    var queryRequest = new QueryRequest().Statement($“SELECT * FROM {bucketName} WHERE Archived=false”).UseStreaming(false);
    var result = await StorageClient.QueryAsync(queryRequest);

11

@simonmsims

The issue may be the fact that SELECT * returns the result object nested inside a property with the name of the extent (in this case the bucket. As an experiment, can you try this query instead? It will remove the extra level of nesting.

SELECT {bucketName}.* FROM {bucketName} WHERE Archived=false
12

I’ve tried with the provided query and can confirm that it working now as expected. Loos like that extra level of nesting was causing the observed problems.