When I develop applications I always find myself running into a scenario where I need to search for a particular set of text within a chunk of data. I, like many others, often find myself taking the easy way out and doing a wildcard query. However, making use of wildcards in a query can have significant performance drains and inefficiencies within your application or database. This is where full text search (FTS) comes into play.
Full text search operates on full text indexes and is far more efficient than working with wildcards when it comes to search for words and phrases in a database. Up until now to make this possible you’d have to use other search software such as Solr or ElasticSearch. Starting in Couchbase 4.5, FTS is available as developer preview to bring it into a single platform.
We’re going to take a look at how to make use of full text search in a Node.js application using the Node.js SDK for Couchbase Server. What comes next assumes that you already have Node.js installed and ready for development. It also assumes that you are using Couchbase Server 4.5 or higher.
To keep things simple we’re going to work on a new Node.js project with a simple story. Let’s start by creating a new project somewhere on our machine. I’m going to call this project resume. As you can probably guess from the project name, we’re going to create a parsing type application for job applicant resumes. We can use FTS to scan for keywords or phrases that are relevant for certain job positions. Since FTS offers search scoring, we can see which candidates might be more qualified for the job.
Creating a New Node.js Project
Using a Terminal (Mac and Linux) or Command Prompt (Windows), execute the following with the project as the current active directory:
|
1 2 3 |
npm init -y |
The above command will initialize a new Node.js project. Going forward, all Terminal and Command Prompt activity will be done with the project as the current working directory.
We’re going to create more of a Node.js script rather than a RESTful application. That said, there is only one dependency requirement, and that is the Couchbase Node.js SDK. To install it, execute the following from the Command Prompt or Terminal:
|
1 2 3 |
npm install couchbase --save |
Although we won’t start coding yet, we want to create our main Node.js file. Create app.js at the root of your project.
Creating a Full Text Search Index
Before you can start using full text search, you must create a special index. This can be done from the Indexes -> Full Text tab of the Couchbase administrative dashboard.
From this section you’ll want to choose New Full Text Index and choose the name and bucket to apply it on. For this example we’ll be using the default bucket and an index of resume-search. Our index will be very basic, so I encourage you to check out the documentation so you can best meet your needs.
After the index has been created, click the refresh button. At this point you can test it out via the dashboard or your own code.
Developing the Node.js Application
Just to re-iterate, this project will be kept rather simple. You can certainly expand upon this idea and make a full on resume evaluation application. For us, we’re only scratching the surface of possibilities.
Open the project’s app.js file and include the following JavaScript code:
|
1 2 3 4 5 6 |
var Couchbase = require("couchbase"); var cluster = new Couchbase.Cluster("couchbase://localhost") var bucket = cluster.openBucket("default"); |
In the above, we’re importing the dependency that we downloaded and we’re establishing a connection to the Couchbase cluster of our choosing. In this case it is a single node cluster on my local machine.
Before we start performing full text search operations, let’s think about our data model. Let’s say that each resume is a document in the database. For example, here is my resume:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
{ "firstname": "Nic", "lastname": "Raboy", "skills": [ "java", "node.js", "golang", "nosql" ], "summary": "", "social": { "github": "https://www.github.com/nraboy", "twitter": "https://www.twitter.com/nraboy" }, "employment": [ { "employer": "Couchbase", "title": "Developer Advocate", "location": "San Francisco" } ] } |
Yes, of course above is not my real resume. It is only an example to see what we’re going to be working with. Now let’s say we want to find all resumes where the person has prior developer advocate experience. In Node.js we might create the following:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
var SearchQuery = Couchbase.SearchQuery; var query = SearchQuery.new("resume-search", SearchQuery.match("developer advocate")); bucket.query(query, function(error, result, meta) { if(error) { return console.log("ERROR: ", error); } for (var i = 0; i < result.length; i++) { console.log({"id": result[i].id, "score": result[i].score}); } return; }); |
Since I listed I was a Developer Advocate in my employment history, my record will appear. However, the above FTS query is rather vague. As long as “developer advocate” in any property of the document, we’ll know about it. Maybe we want to narrow it down, to specifically the employment history. In this case we can change our SearchQuery to the following:
|
1 2 3 |
var query = SearchQuery.new("resume-search", SearchQuery.match("developer advocate").field("employment.title")); |
Notice the use of the field function? You can also use fields and pass an array of fields to search in.
There are plenty of other ways to search for data using FTS and the Node.js SDK. For some information on using FTS, check out the developer documentation or the Node.js API documentation.
Conclusion
If you ever find yourself needing to query for wildcard data in an efficient manner, Couchbase and the Node.js SDK has you covered with its support for full text search (FTS). There are far more complex scenarios to full text search than the ones I gave. For example maybe you want to start adding conditions to the search or view facet information. All of this can be read about in the Couchbase Developer Portal.
