Couchbase Lite Sync Protocol Examination via Wireshark

During the course of development it is often useful to examine what exactly is sent over the network between machines.  This helps to perform tasks like finding the source of an issue or just simply understanding the conversation.  Often times the conversation uses a well known protocol like HTTP.  However, for Couchbase Mobile 2.0 and higher, the conversation takes place using a sync protocol called BLIP, transported via web socket.  You might think that this makes the conversation entirely opaque, but actually wireshark is able to analyze BLIP messages.  Let’s take a quick look at how to do this.

Before we get into that, however, I need to disclaim something.  The information here is presented purely for educational purposes.  You should not rely on this format always being the same.  We at Couchbase have no contract with consumers about it, and it is subject to change.  However, sometimes it is still nice to be able to see what is going on.

Filtering On BLIP

This post will assume that you know the basics of how to use wireshark.  Any version 3.0.0 or higher is able to be used, but earlier versions have a bug that can cause a crash on large compressed messages so 3.2.7+ / 3.0.14+ is recommended.  You should be familiar with how to start a network capture.  This post starts assuming that you have already made one.  When we open your capture to examine, you may see something like the following image.

This is the view you will probably see when you open a new trace

This is because we have not set any filters yet.  Every packet of every type is currently visible. If you notice, near the top is a text box with the hint text “Apply a display filter” in it.  BLIP is a supported packet format, and you can set a filter to show only BLIP messages.

Now things look more manageable, showing only the BLIP conversation.

One caveat to this is that since BLIP uses web sockets, it cannot be decoded if the entire conversation is not present.  In other words, the initial HTTP request for the web socket must be present.  For example, if you change the filter to blip || http then you can see the first HTTP message in this conversation.

Note the two green entries at the top, which show a request/response HTTP message for a web socket.

Message Details

So now that a BLIP conversation is here, what is inside of a BLIP message?  To answer that question, let’s examine the contents of a typical BLIP message:

The contents of a typical BLIP message.

From top to bottom we have the following items:

• Message Number – Basically the ID of the message being referred to.  This is important for the next section.
• Frame Flags – Some metadata information about the message (In this example, the body is compressed and it is urgent.  This means that it will be handled out of order.)
• Properties Length – The length of the properties of the message (The properties are encoding as a length entry, followed by the data of the given length)
• Properties – The properties of the message (A collection of key value pairs)
• Message Body – The actual content of the message.  It occupies the entire rest of the message, except for the last 4 bytes.
• Checksum – A checksum value is written to ensure that the message is received in the order it was sent, and that the packet contents have not been malformed

There are 6 types of messages that can be sent, indicated by the way they are marked in the list of packets:

• MSG – A new message
• RPY – A [normal] reply to a received message
• ERR – An error reply to a received message
• ACKMSG – Progress acknowledgement for receiving a long multi packet message
• ACKRPY – Progress acknowledgement for receiving a long multi packet reply

Following a Conversation

For the purposes of following a given conversation, an important thing to take into account is that both sides can initiate a message.  This will show up in the list of messages in wireshark as MSG#{NUM}.  The way to tell which side sent the message is to look at the source and destination IP addresses.  For example, two diagrams up you can see MSG#2 as the third message in the list.  You can also see MSG#2 as the second to last message in the list.  The difference is that the first entry has the source IP as 192.168.33.1 and the destination IP as 192.168.33.11, and the latter is reversed.  This indicates the first message is a message that Couchbase Lite sent to Sync Gateway because in my setup the Sync Gateway instance is running at 192.168.33.11.  The second is the opposite, a message that Sync Gateway sent to Couchbase Lite.

After receiving a message one of three things will occur:

1. The message is received, executed, and finishes normally.  In this case, a reply message will be sent with the same message number as the received message.  So for example, MSG#2 will get a RPY#2 (with the source and destination IP reversed).
2. The message is received, but cannot complete normally.  In this case, an error message will be sent with the same conditions as #1.
3. The message has a NoReply flag in its flags.  In this case, no response is sent.

Using the above, we can see that in the message list, the message immediately following the first MSG#2 is RPY#2 and thus it is the reply to the message.  Similarly, the first MSG#1 has an ERR#1 reply, indicating that it finished with an error.  This method is applicable to both sides of the conversation, so it should be easy to track the back and forth messages in a given BLIP conversation.