The Internet of Things (IoT) is becoming all the rage lately. Being able to craft any mechanical device, for example watches, televisions, thermostats, and have them communicate over the internet is the modern age. In this modern age, one thing that remains consistent is the need for transferring and storing data. How do you do this from an IoT device?
Couchbase Server exists for storing massive amounts of enterprise data and Couchbase Mobile exists for storing data locally on mobile devices and synchronizing it to Couchbase Server when capable. Where does IoT fit into this? IoT devices are not servers and they are technically not mobile devices.
Did you know that many IoT devices are capable of running Java applications? There is actually a Couchbase Lite SDK for Java. This is not an Android or iOS SDK. It is an SDK for general Java applications. With this, we can use it for IoT.
The Project Scope
Now that we know it is possible, let’s think of a cool IoT example and execute it. Let’s go with an iBeacon example under the following scenario.
Let’s say you want to track your pet while you’re away from your home. You want to understand where your pet goes throughout your home and at what times of the day. So you decide to attach an iBeacon to your pet’s collar and set up a few IoT scanners around your home.
So with your iBeacon attached and your IoT gateways in place, the IoT devices can continuously scan for iBeacons. When an iBeacon comes into range, a timestamp along with the location and beacon information can be saved and uploaded to your server. You can later create your own dashboard with a heat map to better make sense of your data.
So what is it going to take to try out this project?
The Requirements
There are a few requirements in the realm of software as well as hardware. They can be seen below:
- Minimum one (1) Intel IoT Gateway
- Minimum one (1) Gimbal proximity beacon
- Java 1.7+
- Maven
- Couchbase Sync Gateway
While there is a hardware requirement, the hardware brand is a little flexible. I listed both the Intel IoT Gateway and Gimbal proximity beacons because not only are they incredibly affordable, but they are what I used when building my application. To get the most out of this example, it is best to have multiple beacons and gateways, but one will do fine for prototyping.
Below is how this project will come together.
The full source code to this project can be found on GitHub.
Saving Your Data with Couchbase
Before we get into the IoT and beacon work it would be a good idea to define our data model and build our Java application. In this scenario, which is one of many, the Java application will not be scanning for beacons. It will only be responsible for saving data.
The Couchbase Beacon Data Model
If you’re unfamiliar with iBeacons, they offer nothing but a few string and integer values in their transmission. They cannot be connected to, and they cannot see other devices. All they do is broadcast. That said, the values broadcasted are as follows:
- UUID
- Major
- Minor
- Power
The UUID, Major, and Minor offer uniquness information about a particular beacon. The three values can form a composite key that is useful when querying later on.
Let’s think about how we’re going to store the beacon data every time one is detected.
We could create a new JSON document every time a beacon is detected. The individual documents might look something in the realm of:
|
1 2 3 4 5 6 7 8 9 |
{ "uuid": "32342342", "major": 1, "minor": 0, "createdAt": 1932847298, "gatewayDevice": "kitchen" } |
There is nothing wrong with the above approach. However, you could end up with massive amounts of documents depending on how many beacons you have circulating. Again, Couchbase was designed to handle this so it is a matter of preference. I actually prefer keeping all beacon transactions for a particular beacon in the same document like so:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
{ "uuid": "23423423", "major": 1, "minor": 0, "beaconStatus": [ { "createdAt": 234234321234, "gatewayDevice": "kitchen" }, { "createdAt": 12312323544, "gatewayDevice": "garage" } ] } |
In the second scenario, every time a beacon is discovered, Couchbase would be queried based on the composite key and then the status data would be added to the status array.
Adding the Maven Dependencies
Since this Java project is going to be Maven based, the pom.xml file needs certain dependencies satisfied. Add the following dependency to your Maven file to include the Couchbase Lite Java SDK in your project:
|
1 2 3 4 5 6 7 |
com.couchbase.lite couchbase-lite-java 1.2.0 |
The above dependency will allow you to store Couchbase documents locally on your IoT device and have them synchronize with Couchbase Sync Gateway.
Creating a Beacon Class
Since we will be working with iBeacons, it makes sense to make a Java Beacon class. This class should be responsible for saving beacon data as well as loading it in the event of a query. Let’s start by taking a look at what loading is like:
|
1 2 3 4 5 6 7 8 9 10 |
private Document load(Database database) { Document document = null; try { View beaconView = database.getView("beacons"); beaconView.setMap(new Mapper() { @Override public void map(Map<String, Object> document, Emitter emitter) { List<object> keys = new ArrayList<object>(); keys.add(document.get("uuid")); keys.add(document.get("major")); keys.add(document.get("minor")); emitter.emit(keys, document.get("beaconStatus")); } }, "1"); Query query = beaconView.createQuery(); List<object> keys = new ArrayList<object>(); List<object> key = new ArrayList<object>(); key.add(this.uuid); key.add(this.major); key.add(this.minor); keys.add(key); query.setKeys(keys); QueryEnumerator result = query.run(); for (Iterator it = result; it.hasNext(); ) { QueryRow row = it.next(); document = row.getDocument(); } } catch (Exception e) { e.printStackTrace(); } return document;}A lot is happening in the above load so let's take a look at it.Since Couchbase Lite queries based on MapReduce views we first create a view. Remember we plan to use a composite key which is why we create a list of keys. The only data we need returned if a match was found is the status array that contains timestamp and gateway information.With the view in place, a query can be constructed. Since we are in a custom Java class, let's assume that we've captured the <strong>uuid</strong>, <strong>major</strong>, and <strong>minor</strong> values at some point in time. We'll construct the key lists out of the values and set the query to use them with the <code>query.setKeys</code> function.After the query runs we'll loop through the results. We are only expecting one result which can further be insured by using a limit function before executing the query. The result returned from the query, if any, will be returned to whatever calls the load function. In this case the save function calls it before saving.Now let's take a look at the save function of our beacon class. <pre><code>public String save(Database database) { String docId = ""; Map<String, Object> properties = new HashMap<String, Object>(); ArrayList beaconStatusList = new ArrayList(); Document document = this.load(database); if(document != null) { properties.putAll(document.getProperties()); beaconStatusList = (ArrayList) properties.get("beaconStatus"); } else { document = database.createDocument(); properties.put("uuid", this.uuid); properties.put("major", this.major); properties.put("minor", this.minor); } beaconStatusList.add(this.beaconStatus); properties.put("beaconStatus", beaconStatusList); try { docId = document.putProperties(properties).getDocument().getId(); } catch (Exception e) { e.printStackTrace(); } return docId;} |
In the save function we first want to load documents to see if they first exist. If a document exists for the beacon information, we want to add to it, otherwise we can just create a new document. After we’ve reconstructed our Couchbase document, whether it be from scratch or from existing document data, we save it via the putProperties function.At this point beacon data can be saved and read from the local Couchbase Lite database.
Synchronizing with Couchbase Sync Gateway
More than likely you’re going to be working with more than one IoT scanning gateway. Because of this, the beacon data (in the embedded document model) will need to be synchronized between IoT devices. Now we can take our saving and loading code to the next level.Take a look at the following code:
|
1 |
try { manager = new Manager(new JavaContext("data"), Manager.DEFAULT_OPTIONS); database = manager.getDatabase("iot-project"); URL url = new URL("http://192.168.1.174:4984/beacons-iot/"); final Replication push = database.createPushReplication(url); Replication pull = database.createPullReplication(url); pull.setContinuous(false); push.setContinuous(false); pull.addChangeListener(new Replication.ChangeListener() { @Override public void changed(Replication.ChangeEvent event) { if(event.getSource().getStatus() == Replication.ReplicationStatus.REPLICATION_STOPPED) { beacon.save(database); push.start(); } } }); push.addChangeListener(new Replication.ChangeListener() { @Override public void changed(Replication.ChangeEvent event) { if(event.getSource().getStatus() == Replication.ReplicationStatus.REPLICATION_STOPPED) { System.exit(1); } } }); pull.start();} catch (Exception e) { e.printStackTrace();} |
This code would exist in another class, preferrably your class that contains your main function. Essentially it will initialize connection to your local database, configure the push and pull replicators to a remotely running instance of Sync Gateway, and start the save and synchronization process.To be more specific, the replication process will only happen one time. When we run the application, first we will pull down any relevent beacon documents from the server. By using change listeners on the replicators we can wait to save only after we’ve finished downloading any changes. To prevent the Java application from remaining open after pushing, we add a listener that will close the application upon completion.
Resolving the SQLite Dependency Errors
When building this application with Maven, there shouldn’t be any issues. However, depending on the IoT device you deploy to there could be a library dependency issue. There are many different architectures in circulation. An example error might look like the following:
|
1 |
Library not found: /native/linux/i386/libsqlite3.so |
This can easily be resolved by extracting the JAR archive, renaming one of the directories, and then packaging it again into a JAR. To be more specific, to extract your JAR file, execute the following command:
|
1 |
jar xvf [filename].jar |
From the extracted documents and files, rename the /native/linux/x86 directory to /native/linux/i386. With this done you can repackage your JAR file and redeploy it.
|
1 |
jar cvfm [filename].jar META-INF/MANIFEST.MF . |
The above command will repackage the JAR file for you.
Scanning for iBeacons and Tracking Their Status
Like mentioned previously, the Java application is not responsible for detecting iBeacons. It is only responsible for saving the data. Instead we’re going to make use of a few tools that come pre-installed on a Linux operating system.
Scanning with Linux Tools
Most Linux distributions ship with the hcitool and hcidump applications. The hcitool will let you scan for bluetooth devices that are within range. You would run something like the following:
|
1 |
hcitool lescan |
The above would return basic information about devices found in a scan. You would then use the hcidump tool to show anything and everything about the bluetooth data that was discovered. Something like this would show the raw data:
|
1 |
hcidump --raw |
The problem is that data is very raw. It is not something we can work with in its current form. We wouldn’t be able to use it to make sense of iBeacon data.
Simplify the Process with a Tool by Radius Networks
This is where a script by Radius Networks comes into play. There is a script called iBeacon Scan that will make use of hcitool and hcidump, but parsed out and clean. It can be seen below:
|
1 |
#!/bin/bash# iBeacon Scan by Radius Networks# Modified by Nic Raboy at Couchbaseif [[ $1 == "parse" ]]; then packet="" capturing="" count=0 while read line do count=$[count + 1] if [ "$capturing" ]; then if [[ $line =~ ^[0-9a-fA-F]{2} [0-9a-fA-F] ]]; then packet="$packet $line" else if [[ $packet =~ ^04 3E 2A 02 01 .{26} 02 01 .{14} 02 15 ]]; then UUID=`echo $packet | sed 's/^.{69}(.{47}).*$/1/'` MAJOR=`echo $packet | sed 's/^.{117}(.{5}).*$/1/'` MINOR=`echo $packet | sed 's/^.{123}(.{5}).*$/1/'` POWER=`echo $packet | sed 's/^.{129}(.{2}).*$/1/'` UUID=`echo $UUID | sed -e 's/ //g' -e 's/^(.{8})(.{4})(.{4})(.{4})(.{12})$/1-2-3-4-5/'` MAJOR=`echo $MAJOR | sed 's/ //g'` MAJOR=`echo "ibase=16; $MAJOR" | bc` MINOR=`echo $MINOR | sed 's/ //g'` MINOR=`echo "ibase=16; $MINOR" | bc` POWER=`echo "ibase=16; $POWER" | bc` POWER=$[POWER - 256] # Launch Couchbase Java Application To Save Beacon Transaction java -jar iot-couchbase-project.jar $UUID $MAJOR $MINOR $POWER fi capturing="" packet="" fi fi if [ ! "$capturing" ]; then if [[ $line =~ ^> ]]; then packet=`echo $line | sed 's/^>.(.*$)/1/'` capturing=1 fi fi doneelse sudo hcitool lescan --duplicates 1>/dev/null & sudo hcidump --raw | ./$0 parse $1fi |
Full disclosure that this script was written by Radius Networks with the exception of one line:
|
1 |
java -jar iot-couchbase-project.jar $UUID $MAJOR $MINOR $POWER |
We’re taking the information parsed by the script and piping it into our Java application to be saved. This script will run continuously until manually stopped.
Above is a sample animation of this project in action.
Conclusion
That wasn’t so bad right? You just created a simple Internet of Things (IoT) project that scans iBeacons and saves the information into Couchbase. The Couchbase Lite Java SDK is near identical to the Couchbase Lite Android SDK. With it we can take our Java applications to pretty much anything that supports Java.A full working example project can be seen on GitHub.
