Libevent io plugin on windows

amitkul111 · February 16, 2016, 8:54pm

Can I use libevent io plugin on windows. If yes how to enable it?

ingenthr · February 17, 2016, 1:08am

Are you referring to use with libcouchbase? Could you describe a bit more what you’d like to do?

amitkul111 · February 17, 2016, 2:48pm

Yes @ingenthr I am using libcouchbase C SDK client with couch base server 3.0. I would like to achieve callback using events and not lcb_wait on windows platform. By default libevent plugin is not present on windows and even throwing error while tried to add the configuration (./configure enable-embedded-libevent-plugin). So please tell me which is the best way to achieve this?

ingenthr · February 18, 2016, 12:00am

I believe not using the lcb_wait() and the choice of event backend are orthogonal. You’ll still need an event backend.

To solve your compilation problem, I think you’ll need to get a source distribution of libevent if that’s the backend you want to use. On windows, I believe it will use IOCP by default and you don’t need to pass anything to configure. @mnunberg may be able to confirm.

Generally speaking, you’ll want to implement the interface in iops.h. That’s a volatile interface, as covered in the docs.

mnunberg · February 18, 2016, 2:16am

On Windows libcouchbase does indeed use IOCP by default, while an option is also available for using select().

The libevent plugin has not been tested on Windows. It may or may not work on Windows, likely you would have to change some #include s and #ifdefs, but it should otherwise work.

I am unclear on what type of “events” you are expecting to receive. Do you mean Windows events (as in the Windows Event API (https://msdn.microsoft.com/en-us/library/windows/desktop/ms686915(v=vs.85).aspx) or do you simply mean your application is non-blocking and is already using libevent?

amitkul111 · February 18, 2016, 2:38am

I just want my application non-blocking and I am free to use any io plugin. But I noticed that there is no code for libevent and libuv comes with libcouchbase. Is libcouchbase supports libevent and libuv on Windows? Or select is the only option ?

Also is there any samples for libuv and select calls?

mnunberg · February 18, 2016, 2:48am

Of the I/O plugins in libcouchbase, the select and IOCP plugins are self-contained, meaning they are not intended for integration with other applications in a non-blocking manner.

The libev, libevent and libuv plugins are intended to allow integration with other applications performing I/O other than libcouchbase.

libuv is indeed supported and tested on Windows. A small example of its usage (which isn’t really an example per-se): https://github.com/couchbase/libcouchbase/blob/master/plugins/io/libuv/libuv_io_opts.h#L54 and another one: https://github.com/couchbase/couchnode/blob/master/src/binding.cc#L140

It is also possible to write your own I/O backend (that is what the Python SDK does, for example)

amitkul111 · February 24, 2016, 3:23am

Thanks @mnunberg for the quick reply. I am using libuv as a io plugin and able to create an instance, and connect to bucket. I am using following code for that.

options.v.v0.loop = uv_default_loop();
options.v.v0.startsop_noop = 1;

lcb_error_t error = lcb_create_libuv_io_opts(0, &ioops, &options);
if (error != LCB_SUCCESS) {
    return false;
}

char connectionString[BUFFLEN_LARGE] = {'\0'};
memset( connectionString, 0, sizeof(connectionString) );
GLSsnprintf(connectionString, sizeof(connectionString), "couchbase://%s/%s%s", hostname, bucketname, "?bootstrap_on=cccp");
       
struct lcb_create_st creatOptions;
memset(&creatOptions, 0, sizeof (creatOptions));
creatOptions.version = 3;
creatOptions.v.v3.connstr = connectionString;
creatOptions.v.v3.username = bucketUser;
creatOptions.v.v3.passwd = bucketPassword;
creatOptions.v.v3.io = ioops;
creatOptions.v.v3.type = LCB_TYPE_BUCKET;

error = lcb_create(&this->instance, &creatOptions);
if (error != LCB_SUCCESS) {
    return false;
}

// Register us as the thread callback
threadObject.Register(this);

// Start the thread
threadObject.StartThread(); 

lcb_set_configuration_callback(instance, onConfigCallback);
lcb_set_bootstrap_callback(instance, onBootstrapCallback);
lcb_set_error_callback(instance, onErrorCallback);
error = lcb_connect(instance);
if (error != LCB_SUCCESS ) {
    return false;
}

void CouchbaseClientWrapper::onErrorCallback(lcb_t instance, lcb_error_t error, const char *errinfo)
{
    if (errinfo) {
        cout << "Error: " << errinfo << endl;
    }
    return;
}

void CouchbaseClientWrapper::onConfigCallback(lcb_t instance, lcb_configuration_t config)
{
    if(config != LCB_CONFIGURATION_NEW) {
        cout << "Error: Failed to receive configurations" << endl;
    }else {
        cout << "Success: Received new configurations" << endl;
    }

    return;
}

void CouchbaseClientWrapper::onBootstrapCallback(lcb_t instance, lcb_error_t error)
{
    if(error != LCB_SUCCESS) {
        cout << "Error: " << error << endl;
    } else {
        cout << "Success: Received bootstrap callback" << endl;
    }

    return;
}

Once the connection is done I application calls lcb_get to get data from the couch base cluster.

lcb_set_get_callback(instance, onGetCallback);
lcb_get_cmd_t cmd = NULL;
const lcb_get_cmd_t *cmdlist[] = { &cmd };
cmd.v.v0.key = documentKey.c_str();
cmd.v.v0.nkey = documentKey.length();
cmd.v.v0.lock = true;

lcb_error_t error = lcb_get(instance, &clientData, 1, cmdlist);

But lcb_get fails with error as LCB_CLIENT_ETMPFAIL.

Have you any inputs why it is failing even after successful connection?

amitkul111 · February 24, 2016, 3:25am

Also I am using following function to run default uv loop in a separate thread

void CouchbaseClientWrapper::Run()
{
bool exisThread = false;
if ( !isShuttingDown() )
{
uv_run(uv_default_loop(), UV_RUN_DEFAULT);
}

return;

}

mnunberg · February 24, 2016, 4:01am

Do not use lcb_set_error_callback or lcb_set_configuration_callback. These two are older deprecated APIs and are not particularly useful. (Can you tell me where you found out about them, since they have been deprecated for almost two years).

The connection is done only when the bootstrap callback (lcb_set_bootstrap_callback) indicates that it was successful. lcb_connect() merely schedules the connection.

amitkul111 · February 24, 2016, 4:17am

lcb_set_error_callback and lcb_set_configuration_callback are get it from one of the example in couchbase client C SDK.

Anyway I will remove these callbacks But In my case bootstrap callback (lcb_set_bootstrap_callback) indicates it is successful. andstill lcb_get fails with error as LCB_CLIENT_ETMPFAIL.

Have you any inputs?

mnunberg · February 24, 2016, 4:49am

What is the sequence of events that you’re using? Note that before the bootstrap callback is invoked, you cannot schedule any operations

amitkul111 · February 24, 2016, 4:53am

As mentioned in the code in my previous post I am using a following sequence of code

lcb_create_libuv_io_opts
lcb_create
lcb_set_bootstrap_callback
lcb_connect

Once the lcb_connect is complete & bootstrap callback (lcb_set_bootstrap_callback) indicates that it was successful

lcb_get

if you need complete code I can send it to you.

mnunberg · February 24, 2016, 4:27pm

Yes, some code (or better yet, some code I can actually reproduce and run) would be good.

Have you looked into our libevent example? while this does use libevent and not libuv, the use concept should be the same:

github.com

couchbase/libcouchbase/blob/master/example/libeventdirect/main.c

/* -*- Mode: C; tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- */
/*
 *     Copyright 2012 Couchbase, Inc.
 *
 *   Licensed under the Apache License, Version 2.0 (the "License");
 *   you may not use this file except in compliance with the License.
 *   You may obtain a copy of the License at
 *
 *       http://www.apache.org/licenses/LICENSE-2.0
 *
 *   Unless required by applicable law or agreed to in writing, software
 *   distributed under the License is distributed on an "AS IS" BASIS,
 *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *   See the License for the specific language governing permissions and
 *   limitations under the License.
 */

/**
 * gcc -levent -lcouchbase main.c
 *

This file has been truncated. show original

(note, you need libcouchbase >= 2.5.6 to make this example work without any modifications).

(Looking at your code agian, I also hope and assume that CouchbaseClientWrapper::onBootstrapCallback is a static function!)

amitkul111 · February 25, 2016, 2:41am

Yes @mnunberg I am using onBootstrapCallback as a static function.

Also I tried with latest 2.5.6 version of C SDK library but response is same. Also I observed that some times there is no response from the server not sure why?

here i am sharing a link of my code used as a wrapper classes to call C SDK library functionality for your reference.

https://drive.google.com/file/d/0B1gOZhJ_E8TLOS1GZkdHS2JaYXM/view?usp=sharing

if you have any inputs please let me know.

amitkul111 · February 25, 2016, 2:47am

For this I am using C SDK client 2.5.6 and couchbase server 3.1.3 on windows platform

mnunberg · February 25, 2016, 3:19am

Thank you for sharing your code. It would seem you’re still using the synchronous programming model; furthermore it seems you’re never actually waiting anywhere for the bootstrap process to complete - more to the point, if you were running a non-blocking callback driven model, your bootstrap callback (I thought it was just stub code, but now I see the callback indeed does nothing) would signal to somewhere else in your application that the client is connected, and it is therefore now safe to start scheduling operations.

If you need to use synchronous code, then the actual I/O plugin does not matter (because the event loop is self-contained) and you should be using lcb_wait.

amitkul111 · February 25, 2016, 3:28am

Actually I am sending request for any scheduled operation only after lcb_connect is successful. which means my another application send an request to get document or store document only after I am seeing a successful connetion. In my case which I am logging in a log file for temporary purpose but planned to set an event in call function and check for set event in main function for each operation. But my problem is even I am sending get document request after seeing successful message in a log file I am getting error on lcb_get call which is LCB_CLIENT_ETMPFAIL

mnunberg · February 25, 2016, 3:31pm

What exactly are you seeing in the logs?

amitkul111 · February 25, 2016, 5:20pm

I was get an message as “In onBootstrapCallback Success” which is printed in onBootstrapCallback of the library callback function.

Anyway I developed a small executable which basically connect to the cluster and get the document data from the cluster. I am hereby sharing a code and built binary for the same.

https://drive.google.com/open?id=0B1gOZhJ_E8TLUkpXc0p2VWh5QWs

I am using same io plugin (libuv) and interested only in asynchronous mechanism.

In this case I never able to connect to cluster as callback never gets called and hence failed to connect.
Not sure why?
I am able to connect to the same cluster using another library (which is in-house developed by a team) but failed using libcouchbase C SDK with libuv io plugin.

Topic		Replies	Views
Libev vs libevent C SDK	8	5697	January 9, 2018
Trying to use libevent C SDK	4	3435	September 30, 2013
How to build an IO operations plugin? C SDK	1	1851	May 7, 2013
C API - Are callbacks done synchronously or asynchronously? C SDK	3	4341	February 15, 2013
Reason for lcb_update_server_timer C SDK	1	1873	July 30, 2013

Libevent io plugin on windows

Related topics